A comprehensive reference guide for Docker covering essential concepts, configurations, best practices, and solutions to common problems. This guide includes tips for registry management, networking, multi-architecture builds, and more.

Build Optimization

Using Kaniko for Faster Docker Builds

When running GitLab CI/CD in Kubernetes (AKS), Kaniko provides significant performance improvements over Docker-in-Docker (DiND) for building container images. Adding these flags to your Kaniko configuration can further optimize build times:

--snapshotMode=redo
--use-new-run

With these optimizations, we’ve seen build times reduce from approximately 5.5 minutes with DiND to just 3.25 minutes with Kaniko.

Dockerfile Instructions

Understanding ARG vs ENV

ARG Instruction:

• Only accessible during image build time
• Not available when running containers
• Can be passed using the --build-arg flag during build
• Example: docker build --build-arg VERSION=1.0 .

ENV Instruction:

• Available during both build time and container runtime
• Can be overridden when running containers using the --env flag
• Cannot be passed during image build time
• Example: docker run --env MODE=production my-image

AWS Docker Repository login

#+begin_src shell $ aws ecr get-login-password —region us-west-2 | docker login —username AWS —password-stdin <ACCOUNT_ID>.dkr.ecr.us-west-2.amazonaws.com [Aws]({{< relref “20210808103426-aws.md” >}})#+end_src

Docker local repository cache

docker run -it —rm —entrypoint cat registry:2 /etc/docker/registry/config.yml > `pwd`/config.yml

version: 0.1
log:
  fields:
    service: registry
storage:
  cache:
    blobdescriptor: inmemory
  filesystem:
    rootdirectory: /var/lib/registry
http:
  addr: :5000
  headers:
    X-Content-Type-Options: [nosniff]
health:
  storagedriver:
    enabled: true
    interval: 10s
    threshold: 3

proxy:
  remoteurl: https://registry-1.docker.io

or using aliyun docker mirror

proxy:
  # remoteurl: https://registry-1.docker.io
  remoteurl: https://<MIRROR_ID>.mirror.aliyuncs.com

docker run -d —restart=always -p 5000:5000 —name docker-registry-proxy -v `pwd`/config.yml:/etc/docker/registry/config.yml registry:2

dockerd —registry-mirror=http://localhost:5000

or update file directly:

/etc/docker/daemon.json

{
  "registry-mirrors": ["http://localhost:5000"]
}

docker pull nginx

curl http://localhost:5000/v2/_catalog

/lib/systemd/system/docker.service
ExecStart=/usr/bin/dockerd --registry-mirror=http://localhost:5000 -H fd:// --containerd=/run/containerd/containerd.sock

sudo systemctl daemon-reload sudo service docker restart sudo docker system info

https://hub.docker.com/v2/repositories/library/ubuntu/tags?page_size=1000

config.yml

version: 0.1
log:
  fields:
    service: registry
storage:
  cache:
    blobdescriptor: inmemory
  filesystem:
    rootdirectory: /var/lib/registry
http:
  addr: :5000
  headers:
    X-Content-Type-Options: [nosniff]
health:
  storagedriver:
    enabled: true
    interval: 10s
    threshold: 3
proxy:
    remoteurl: https://registry-1.docker.io

start-docker-registry.sh

docker run -d \
  -p 5000:5000 \
  --restart=always \
  --name docker-registry-proxy \
  -v `pwd`/config.yml:/etc/docker/registry/config.yml \
  -v "$(pwd)"/auth:/auth \
  -v "$(pwd)"/registry:/var/lib/registry \
  -e "REGISTRY_AUTH=htpasswd" \
  -e "REGISTRY_AUTH_HTPASSWD_REALM=Registry Realm" \
  -e REGISTRY_AUTH_HTPASSWD_PATH=/auth/passwd \
  registry:2

htpasswd

sudo apt update sudo apt install apache2-utils

htpasswd -cB auth/passwd docker <password>

docker login localhost:5000

docker container ip

docker inspect “container-name” |grep IPAddress

Publish Docker repository proxy server

# /etc/nginx/conf.d/registry.example.com.conf
 server {
     listen 80;

     server_name registry.example.com;  # Replace with your actual domain

     location / {
         proxy_set_header Host $host;
         proxy_set_header X-Real-IP $remote_addr;
         proxy_set_header X-Forwarded-For $remote_addr;
         proxy_set_header X-Forwarded-Proto $scheme;

         # Replace this INTERNALIPADDRESS:PORT with the IP and port
         # combination for the server on your network handling the
         # above-specified domain name.
         proxy_pass http://localhost:5000/;
     }
 }

sudo vi /etc/nginx/nginx.conf
client_max_body_size 100M;
sudo systemctl restart nginx

Search for this variable: client_max_body_size. If you find it, just increase its size to 100M, for example. If it doesn’t exist, then you can add it inside and at the end of http

Add ssl

sudo apt install nginx python3-certbot-nginx

sudo certbot —nginx -d registry.example.com

add user to docker group

sudo usermod -aG docker <user>

Docker in apple m1 sillicon

System requirements You must install Rosetta 2 as some binaries are still Darwin/AMD64. To install Rosetta 2 manually from the command line, run the following command:

$ softwareupdate —install-rosetta

Docker aliyun registry mirror

find aliyun registry mirror url in your aliyun account ecr configuration page

update /etc/docker/daemon.json

{
  "registry-mirrors": ["https://<MIRROR_ID>.mirror.aliyuncs.com"]
}

multi arch builder

Enable Docker Experimental

// daemon.json

{ “builder”: { “gc”: { “defaultKeepStorage”: “20GB”, “enabled”: true } }, “debug”: true,

• “experimental”: true

}

$ docker buildx ls NAME/NODE DRIVER/ENDPOINT STATUS PLATFORMS desktop-linux * docker desktop-linux desktop-linux running linux/amd64, linux/arm64, linux/riscv64, linux/ppc64le, linux/s390x, linux/386, linux/arm/v7, linux/arm/v6 default docker default default running linux/amd64, linux/arm64, linux/riscv64, linux/ppc64le, linux/s390x, linux/386, linux/arm/v7, linux/arm/v6

$ docker buildx create —name mybuilder

$ docker buildx ls

$ docker buildx use mybuilder

$ docker buildx inspect —bootstrap

$ docker buildx build —platform linux/amd64,linux/arm64,linux/arm/v7 -t username/demo:latest —push .

k8s docker secret

kubectl create secret docker-registry regcred --docker-server=<your-registry-server> --docker-username=<your-name> --docker-password=<your-pword> --docker-email=<your-email>

Install docker-ce

sudo apt-get install docker.io

sudo systemctl status docker

sudo usermod -aG docker ${USER}
su - ${USER}
groups

Docker host ip access from container

#!/bin/sh

hostip=$(ip route show | awk '/default/ {print $3}')
echo $hostip

When running Docker natively on Linux, you can access host services using the IP address of the docker0 interface. From inside the container, this will be your default route.

For example, on my system:

$ ip addr show docker0 7: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 valid_lft forever preferred_lft forever inet6 fe80::f4d2:49ff:fedd:28a0/64 scope link valid_lft forever preferred_lft forever And inside a container:

default via 172.17.0.1 dev eth0 172.17.0.0/16 dev eth0 src 172.17.0.4 It’s fairly easy to extract this IP address using a simple shell script:

#!/bin/sh

hostip=$(ip route show | awk ‘default {print $3}’) echo $hostip You may need to modify the iptables rules on your host to permit connections from Docker containers. Something like this will do the trick:

This would permit access to any ports on the host from Docker containers. Note that:

iptables rules are ordered, and this rule may or may not do the right thing depending on what other rules come before it.

you will only be able to access host services that are either (a) listening on INADDR_ANY (aka 0.0.0.0) or that are explicitly listening on the docker0 interface.

Make sure extra_hosts is direct child of php service:

php: extra_hosts: host.docker.internal: host-gateway

network connect

docker network list docker network create demo docker network connect demo mailpit docker network connect demo keycloak-keycloak-1

docker network inspect myNetwork

change to local user instead of root

COPY --from=builder /go/src/mikefarah/yq/yq /usr/bin/yq

WORKDIR /workdir

RUN set -eux; \
  addgroup -g 1000 yq; \
  adduser -u 1000 -G yq -s /bin/sh -h /home/yq -D yq

RUN chown -R yq:yq /workdir

USER yq

ENTRYPOINT ["/usr/bin/yq"]

You don’t have enough free space in var/cache/apt/archives.

clean up system

docker system prune

The issue was because Docker had a large number of unused containers that had to be cleared, the following fixed it:

docker rmi $(docker images -q)
docker rm -v $(docker ps -qa)

or possible:

docker image prune

Copy file out of container

Copy the files out of the container with docker cp <container>:<container_folder>/* <host_folder>

Move docker data directory to different folder

https://www.guguweb.com/2019/02/07/how-to-move-docker-data-directory-to-another-location-on-ubuntu/

1. Stop the docker daemon

sudo service docker stop

1. Add a configuration file to tell the docker daemon what is the location of the data directory

Using your preferred text editor add a file named daemon.json under the directory /etc/docker. The file should have this content:

{ “data-root”: “/path/to/your/docker” }

1. Copy the current data directory to the new one

sudo rsync -aP var/lib/docker /path/to/your/docker

1. Rename the old docker directory

sudo mv /var/lib/docker /var/lib/docker.old

1. Restart the docker daemon

sudo service docker start

FAQs

https://docs.docker.com/desktop/faqs/macfaqs/

Where is the disk image file? To locate the disk image file, select Settings from the Docker Dashboard then Advanced from the Resources tab.

The Advanced tab displays the location of the disk image. It also displays the maximum size of the disk image and the actual space the disk image is consuming. Note that other tools might display space usage of the file in terms of the maximum file size, and not the actual file size.

Alpine

echo http://mirrors.aliyun.com/alpine/v3.10/main/ > etc/apk/repositories echo http://mirrors.aliyun.com/alpine/v3.10/community >> /etc/apk/repositories apk update && apk upgrade

Golang project

FROM golang:1.20.5-alpine AS build

RUN apk update && \
    apk add --update openntpd && \
    ntpd && \
    apk upgrade && \
    apk add --no-cache alpine-sdk git make

WORKDIR /app

# Cache go mod dependencies
COPY go.mod ./
RUN go mod download

COPY . .

RUN make

FROM golang:1.20.5-alpine

WORKDIR /app

COPY --from=build /app/dist/ /app/

RUN chmod a+x /app/*

# CMD ["/app/main"]

Install docker

sudo apt update
sudo apt install apt-transport-https ca-certificates curl software-properties-common
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg
echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt update
apt-cache policy docker-ce
sudo apt install docker-ce
sudo systemctl status docker

sudo usermod -aG docker ${USER}
su - ${USER}
groups
sudo usermod -aG docker username