Docker Handbook
Search…
Docker Handbook
Introduction
Basic Concepts
Containerization History
Docker Containers
Docker Images
Docker Storage and volumes
Docker Networking
Docker Security
Docker Compose
Docker cheat sheet
Docker Compose cheat sheet
Powered By GitBook
Docker Images

What's a docker image?

An image is an executable package that includes everything needed to run an application - the code, a runtime, libraries, environment, variables, and configuration file.
A container is a runtime instance of an image.

container images are big

Container images can be pretty big (though some are really small, like alpine linux is 2.5MB). Ubuntu 16.04 is about 27MB, and the Anaconda Python distribution is 800MB to 1.5GB.
Every container you start with an image starts out with the same blank slate, as if it made a copy of the image just for that container to use. But for big container images, like that 800MB Anaconda image, making a copy would be both a waste of disk space and pretty slow. So Docker doesn’t make copies – instead it uses layering technique called overlay.

how overlays work

Overlay filesystems, also known as “union filesystems” or “union mounts” let you mount a filesystem using 2 directories: a “lower” directory, and an “upper” directory.
Basically:
  • the lower directory of the filesystem is read-only
  • the upper directory of the filesystem can be both readable and writable
When a process reads a file, the overlayfs filesystem driver looks in the upper directory and reads the file from there if it’s present. Otherwise, it looks in the lower directory.
When a process writes a file, overlayfs will just write it to the upper directory.

Understanding Layering with Docker Images

Images are made up of multiple read-only layers. Multiple containers are typically based on the same image. When an image is instantiated into a container, a top writable layer is created. (which is deleted when the container is removed)
Docker uses storage drivers to manage the content of the image layers and the writable container layer.Each storage driver handles the implementation differently, but all drivers use stackable image layers and the copy-on-write(CoW) strategy.

The copy-on-write (CoW) strategy

Copy-on-write is a strategy of sharing and copying files for maximum efficiency. If a file or directory exists in a lower layer within the image, and another layer (including the writable layer) needs read access to it, it just uses the existing file. The first time another layer needs to modify the file (when building the image or running the container), the file is copied into that layer and modified. This minimizes I/O and the size of each of the subsequent layers.

The storage location of Docker images and containers

A Docker container consists of network settings, volumes, and images. The location of Docker files depends on your operating system. Here is an overview for the most used operating systems:
  • Ubuntu: /var/lib/docker/
  • Fedora: /var/lib/docker/
  • Debian: /var/lib/docker/
  • Windows: C:\ProgramData\DockerDesktop
  • MacOS: ~/Library/Containers/com.docker.docker/Data/vms/0/
use docker info | grep -i root command to findout:
1
[[email protected] ~]# docker info | grep -i root
2
Docker Root Dir: /var/lib/docker
Copied!

Docker File

You might create your own images or you might only use those created by others and published in a registry. To build your own image, you create a Dockerfile with a simple syntax for defining the steps needed to create the image and run it. Each instruction in a Dockerfile creates a layer in the image. When you change the Dockerfile and rebuild the image, only those layers which have changed are rebuilt. This is part of what makes images so lightweight, small, and fast, when compared to other virtualization technologies. A Dockerfile is executed by the docker build command
Lets take a look at sample Dockerfile:
1
#Nginx
2
#
3
# VERSION 0.0.1
4
FROM ubuntu
5
LABEL Description="test image" Vendor="ACME Products"
6
RUN apt-get update && apt-get install -y nginx openssh-server
Copied!

Dockerfile instructions:

  • FROM: defines the base image; the FROM instruction must be the first instruction in Dockerfile.
  • LABEL: it's a Description about any thing you want to define about this image,
  • ADD: copies a file into the image but supports tar and remote URL
  • COPY: copy files into the image, preferred over ADD.
  • VOLUME: creates a mount point as defined when the container is run.
  • ENTRYPOINT: the executable runs when the container is run.
  • EXPOSE: documents the ports that should be published.
  • The CMD instruction has three forms:
    • CMD ["executable","param1","param2"] (exec form, this is the preferred form)
    • CMD ["param1","param2"] (as default parameters to ENTRYPOINT)
    • CMD command param1 param2 (shell form)
    There can only be one CMD instruction in a Dockerfile. If you list more than one CMD then only the last CMD will take effect.
  • ENV: used to define environmental variables in the container.
  • MAINTAINER: (while deprecated), MAINTAINER is used to document the author of the Dockerfile (typically an email address)
  • ONBUILD: only used as a trigger when this image is used to build other images; will define commands to run "on build"
  • RUN: runs a new command in a new layer.
  • WORKDIR: defines the working directory of the container.
Now to build an image from this Dockerfile, we'll use the build command. The generic syntax for the command is as follows:
1
docker build <build context>
Copied!
The build command requires a Dockerfile and the build's context. The context is the set of files and directories located in the specified location. Docker will look for a Dockerfile in the context and use that to build the image.
Open up a terminal window inside that directory and execute the following command:
1
[[email protected] sandbox]# ls
2
Dockerfile
3
​
4
[[email protected] sandbox]#docker build .
Copied!
Do not use your root directory, /, as the PATH as it causes the build to transfer the entire contents of your hard drive to the Docker daemon.
We're passing . as the build context which means the current directory. If you put the Dockerfile inside another directory like /src/Dockerfile, then the context will be ./src. The build process may take some time to finish:
1
[[email protected] sandbox]# docker build .
2
Sending build context to Docker daemon 14.85kB
3
Step 1/3 : FROM ubuntu
4
---> adafef2e596e
5
Step 2/3 : LABEL Description="test image" Vendor="ACME Products"
6
---> Running in 043a4ae90eb9
7
Removing intermediate container 043a4ae90eb9
8
---> bdeb04e9a931
9
Step 3/3 : RUN apt-get update && apt-get install -y nginx openssh-server
10
---> Running in ae0d0c698212
11
Get:1 http://archive.ubuntu.com/ubuntu focal InRelease [265 kB]
12
Get:2 http://security.ubuntu.com/ubuntu focal-security InRelease [107 kB]
13
Get:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease [111 kB]
14
Get:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease [98.3 kB]
15
Get:5 http://archive.ubuntu.com/ubuntu focal/universe amd64 Packages [11.3 MB]
16
Get:6 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [187 kB]
17
Get:7 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [1078 B]
18
.
19
.
20
.
21
Successfully built fc32da11d651
Copied!
If everything goes fine, you should see something like Successfully built fc32da11d651 at the end. This random string is the image id and not container id. try docker image inspect <image id> to get information about this image , also to see layers which our image includes try docker image history <image id> :
1
[[email protected] sandbox]# docker image history fc32
2
IMAGE CREATED CREATED BY SIZE COMMENT
3
fc32da11d651 53 seconds ago /bin/sh -c apt-get update && apt-get install… 159MB
4
bdeb04e9a931 2 minutes ago /bin/sh -c #(nop) LABEL Description=test im… 0B
5
adafef2e596e 2 weeks ago /bin/sh -c #(nop) CMD ["/bin/bash"] 0B
6
<missing> 2 weeks ago /bin/sh -c mkdir -p /run/systemd && echo 'do… 7B
7
<missing> 2 weeks ago /bin/sh -c set -xe && echo '#!/bin/sh' > /… 811B
8
<missing> 2 weeks ago /bin/sh -c [ -z "$(apt-get indextargets)" ] 1.01MB
9
<missing> 2 weeks ago /bin/sh -c #(nop) ADD file:cf87af1f0e27aa6ff… 72.9MB
Copied!

Listing images

For listing local images, use the following syntax:
1
docker image ls
Copied!
we can also use docker images which is deprecated somehow.
1
[[email protected] sandbox]# docker image ls
2
REPOSITORY TAG IMAGE ID CREATED SIZE
3
<none> <none> fc32da11d651 About a minute ago 233MB
4
redis latest 50541622f4f1 4 days ago 104MB
5
ubuntu latest adafef2e596e 2 weeks ago 73.9MB
6
nginx latest 9beeba249f3e 2 months ago 127MB
7
hello-world latest bf756fb1ae65 6 months ago 13.3kB
Copied!
The image we have recently built is showing up in the first line. We haven't tagged out image during build process ,we will talk about tagging images later in this section.

Pulling an image from default registry

To download a particular image, or set of images, use docker pull :
1
docker pull <image name>
Copied!
1
[[email protected] sandbox]# docker pull debian
2
Using default tag: latest
3
latest: Pulling from library/debian
4
31dd5ebca5ef: Pull complete
5
Digest: sha256:68f4e2259032a4e6f5035804e64438b52af8dd5889528b305b9059183ea4cd2a
6
Status: Downloaded newer image for debian:latest
Copied!
As we mentioned Docker images can consist of multiple layers. In the example above, the image consists of two layers;

Remove one or more specific images

Use the docker images command to locate the ID of the images you want to remove. When you’ve located the images you want to delete, you can pass their ID or tag to docker rmi:
1
docker rmi <image1> <image2>
Copied!
for example lets remove debian image :
1
[[email protected] sandbox]# docker rmi debian
2
Untagged: debian:latest
3
Untagged: [email protected]:68f4e2259032a4e6f5035804e64438b52af8dd5889528b305b9059183ea4cd2a
4
Deleted: sha256:ae8514941ea4f23d4948150debf0f92a427c136aa4e7fb85f9c56bba09452572
5
Deleted: sha256:6086e1b289d997dfd19df1ec9366541c49f5545520f9dc65ebd4cd64071497b4
Copied!
you can not remove an image which is used by a stop container, you can use --force for removing that but the stopped container(s) will be removed too!
1
[[email protected] sandbox]# docker ps -a | grep "hello-world"
2
c41d97e86738 hello-world "/hello" 4 days ago Exited (0) 4 days ago flamboyant_allen
3
​
4
[[email protected] sandbox]# docker rmi --force hello-world
5
Untagged: hello-world:latest
6
Untagged: [email protected]:49a1c8800c94df04e9658809b006fd8a686cab8028d33cfba2cc049724254202
7
Deleted: sha256:bf756fb1ae65adf866bd8c456593cd24beb6a0a061dedf42b26a993176745f6b
8
​
9
[[email protected] sandbox]# docker ps -a | grep "hello-world"
Copied!

Remove dangling images

Docker images consist of multiple layers. Dangling images are layers that have no relationship to any tagged images. They no longer serve a purpose and consume disk space. They can be located by adding the filter flag, -f with a value of dangling=true to the docker images command. When you’re sure you want to delete them, you can use the docker images purge command:
Note: If you build an image without tagging it, the image will appear on the list of dangling images because it has no association with a tagged image. You can avoid this situation by providing a tag when you build, and you can retroactively tag an images with the docker tag command.
1
#List:
2
docker images -f dangling=true
3
​
4
#Remove:
5
docker images prune
Copied!
As we haven't tag our image lets tag it before purging dangling images.
docker image prune -a will remove all images with out at least one container associate with them, the good news about this is that if you have images that are being used by containers those images won't be deleted.

tagging images

In simple words, Docker tags adds useful information about a specific image version/variant. They are aliases to the ID of your image which often look like this: f1477ec11d12. It’s just a way of referring to your image. A good analogy is how Git tags refer to a particular commit in your history.
The two most common cases where tags come into play are:
  1. 1.
    When building an image, we use the following command:
1
docker build -t image_name:tag_name .
Copied!
It tells the Docker daemon to fetch the Docker file present in the current directory (that’s what the . at the end does). Next, we tell the Docker daemon to build the image and give it the specified tag.
1
[[email protected] sandbox]# docker images
2
REPOSITORY TAG IMAGE ID CREATED SIZE
3
<none> <none> fc32da11d651 21 minutes ago 233MB
4
redis latest 50541622f4f1 4 days ago 104MB
5
ubuntu latest adafef2e596e 2 weeks ago 73.9MB
6
nginx latest 9beeba249f3e 2 months ago 127MB
7
hello-world latest bf756fb1ae65 6 months ago 13.3kB
8
​
9
[[email protected] sandbox]#
10
[[email protected] sandbox]# docker build -t myapp:final .
11
Sending build context to Docker daemon 14.85kB
12
Step 1/3 : FROM ubuntu
13
---> adafef2e596e
14
Step 2/3 : LABEL Description="test image" Vendor="ACME Products"
15
---> Using cache
16
---> bdeb04e9a931
17
Step 3/3 : RUN apt-get update && apt-get install -y nginx openssh-server
18
---> Using cache
19
---> fc32da11d651
20
Successfully built fc32da11d651
21
Successfully tagged myapp:final
22
[[email protected] sandbox]#
23
​
24
[[email protected] sandbox]# docker images
25
REPOSITORY TAG IMAGE ID CREATED SIZE
26
myapp final fc32da11d651 21 minutes ago 233MB
27
redis latest 50541622f4f1 4 days ago 104MB
28
ubuntu latest adafef2e596e 2 weeks ago 73.9MB
29
nginx latest 9beeba249f3e 2 months ago 127MB
30
hello-world latest bf756fb1ae65 6 months ago 13.3kB
Copied!
If you need to push your image to a registry use docker build -t username/image_name:tag_name . :
1
[[email protected] sandbox]# docker build -t borosan/myapp:final .
2
Sending build context to Docker daemon 14.85kB
3
Step 1/3 : FROM ubuntu
4
---> adafef2e596e
5
Step 2/3 : LABEL Description="test image" Vendor="ACME Products"
6
---> Using cache
7
---> bdeb04e9a931
8
Step 3/3 : RUN apt-get update && apt-get install -y nginx openssh-server
9
---> Using cache
10
---> fc32da11d651
11
Successfully built fc32da11d651
12
Successfully tagged borosan/myapp:final
13
​
14
[[email protected] sandbox]#
15
[[email protected] sandbox]# docker image ls
16
REPOSITORY TAG IMAGE ID CREATED SIZE
17
borosan/myapp final fc32da11d651 23 minutes ago 233MB
18
myapp final fc32da11d651 23 minutes ago 233MB
19
redis latest 50541622f4f1 4 days ago 104MB
20
ubuntu latest adafef2e596e 2 weeks ago 73.9MB
21
nginx latest 9beeba249f3e 2 months ago 127MB
22
hello-world latest bf756fb1ae65 6 months ago 13.3kB
Copied!
2.Explicitly tagging an image through the tag command:
1
docker tag SOURCE_IMAGE[:TAG] TARGET_IMAGE[:TAG]
Copied!
It just creates an alias (a reference) by the name of the TARGET_IMAGE that refers to the SOURCE_IMAGE. That’s all it does. It’s like assigning an existing image another name to refer to it. Notice how the tag is specified as optional here as well, by the [:TAG] :
1
[[email protected] sandbox]# docker tag myapp:final myapp:original
2
[[email protected] sandbox]# docker image ls
3
REPOSITORY TAG IMAGE ID CREATED SIZE
4
borosan/myapp final fc32da11d651 28 minutes ago 233MB
5
myapp final fc32da11d651 28 minutes ago 233MB
6
myapp original fc32da11d651 28 minutes ago 233MB
7
redis latest 50541622f4f1 4 days ago 104MB
8
ubuntu latest adafef2e596e 2 weeks ago 73.9MB
9
nginx latest 9beeba249f3e 2 months ago 127MB
10
hello-world latest bf756fb1ae65 6 months ago 13.3kB
Copied!

What happens when you don’t specify a tag?

Alright, now let’s uncover what happens when you don’t specify a tag while tagging an image. This is where the latest tag comes into the picture. Whenever an image is tagged without an explicit tag, it’s given the latest tag by default. It’s an unfortunate naming choice that causes a lot of confusion. But I like to think of it as the default tag that’s given to images when you don’t specify one.

storing images in Docker Registry

A docker registery is a stateless, highly scalable application that stores and lets you distribute Docker images. Registries could be local (private) or cloud-base (private or public).
Examples of Docker Registries:
  1. 1.
    Docker Registry (local open-source registry)
  2. 2.
    Docker Trusted Registry(DTR) [Available in Docker Enterprise Edition]
  3. 3.
    Docker Hub [Default Registry]
The first thing to remember is any time you are going to use a registry you need to first log in to that registry:
You need to create an account in Docker Hub first.
1
[[email protected] sandbox]# docker login
2
Login with your Docker ID to push and pull images from Docker Hub. If you don't have a Docker ID, head over to https://hub.docker.com to create one.
3
Username: borosan
4
Password:
5
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
6
Configure a credential helper to remove this warning. See
7
https://docs.docker.com/engine/reference/commandline/login/#credentials-store
8
​
9
Login Succeeded
Copied!
If we had a docker local registry then it would be docker login localhost:5000 .
and when you finish your job , logout:
1
[[email protected] sandbox]# docker logout
2
Removing login credentials for https://index.docker.io/v1/
3
[[email protected] sandbox]#
Copied!

pushing an image to the Default Registery

Use docker push to Push an image or a repository to a registry
1
docker push [OPTIONS] NAME[:TAG]
Copied!
1
[[email protected] sandbox]# docker image ls
2
REPOSITORY TAG IMAGE ID CREATED SIZE
3
borosan/myapp final fc32da11d651 2 hours ago 233MB
4
myapp final fc32da11d651 2 hours ago 233MB
5
myapp original fc32da11d651 2 hours ago 233MB
6
redis latest 50541622f4f1 4 days ago 104MB
7
ubuntu latest adafef2e596e 2 weeks ago 73.9MB
8
nginx latest 9beeba249f3e 2 months ago 127MB
9
hello-world latest bf756fb1ae65 6 months ago 13.3kB
10
[[email protected] sandbox]#
11
[[email protected] sandbox]# docker login
12
Login with your Docker ID to push and pull images from Docker Hub. If you don't have a Docker ID, head over to https://hub.docker.com to create one.
13
Username: borosan
14
Password:
15
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
16
Configure a credential helper to remove this warning. See
17
https://docs.docker.com/engine/reference/commandline/login/#credentials-store
18
​
19
Login Succeeded
20
[[email protected] sandbox]# docker push borosan/myapp:final
21
The push refers to repository [docker.io/borosan/myapp]
22
3f8b89a55ad4: Pushed
23
544a70a875fc: Pushed
24
cf0f3facc4a3: Pushed
25
132bcd1e0eb5: Pushed
26
d22cfd6a8b16: Pushed
27
final: digest: sha256:c8fcece97935d8babb195f4ab3c9be38a091e259c5e750b84151a48351192fa0 size: 1364
Copied!

Searching for an image

Whether you are using a public or a private registry you can search that registry to find the image that you need. And that is what docker search command does for us:
1
docker search [OPTIONS] TERM
Copied!
docker search has a very useful filtering option, you can filter output based on these conditions:
  • - stars=<numberOfStar>
  • - is-automated=(true|false)
  • - is-official=(true|false)
1
[[email protected] sandbox]# docker search --filter "stars=90" --filter "is-official=true" ubuntu
2
NAME DESCRIPTION STARS OFFICIAL AUTOMATED
3
ubuntu Ubuntu is a Debian-based Linux operating sys… 11152 [OK]
4
ubuntu-upstart Upstart is an event-based replacement for th… 110 [OK]
Copied!
above command searches for official ubuntu images which have more that 90 stars. The --limit flag limits the maximum number of results returned by a search.

Saving and loading images

Pushing to Docker Hub is great, but it does have some disadvantages:
  1. 1.
    Bandwidth - many ISPs have much lower upload bandwidth than download bandwidth.
  2. 2.
    Unless you’re paying extra for the private repositories, pushing equals publishing.
  3. 3.
    When working on some clusters, each time you launch a job that uses a Docker container it pulls the container from Docker Hub, and if you are running many jobs, this can be really slow.
Solutions to these problems can be to save the Docker container locally as a a tar archive, and then you can easily load that to an image when needed.
To save a Docker image after you have pulled, committed or built it you use the docker save command. For example, lets save a local copy of the myapp docker image we made:
1
[[email protected] sandbox]# docker save myapp
2
myapp myapp:final myapp:original
3
[[email protected] sandbox]# docker save myapp:original > myapp-original.tar
4
[[email protected] sandbox]# ls
5
Dockerfile myapp-original.tar
Copied!
Docker supports two different types of methods for saving container images to a single tarball:
  • docker save - saves a non-running container image to a file
  • docker export - saves a container’s running or paused instance to a file
If we want to load that Docker container from the archived tar file in the future, we can use the docker load command:
1
[[email protected] sandbox]# docker load --input myapp-original.tar
2
Loaded image: myapp:original
Copied!

Difference between loading a saved image and importing an exported container as an image

Loading an image using the load command creates a new image including its history. Importing a container as an image using the import command creates a new image excluding the history which results in a smaller image size compared to loading an image.

Commiting changes to an image

When working with Docker images and containers, one of the basic features is committing changes to a Docker image. When you commit to changes, you essentially create a new image with an additional layer that modifies the base image layer.
1
docker commit [OPTIONS] CONTAINER [REPOSITORY[:TAG]]
Copied!
For example let run a container based on nginx image :
1
[[email protected] ~]# docker run -d --name my_nginxtemp nginx
2
bc741086bb8bd8375ff03f14c699927e9659560ab6e653fe614f68843c6e4859
Copied!
Now lets attach to it and modify index.html:
1
[[email protected] ~]# docker exec -i -t my_nginxtemp bash
2
[email protected]:/# cd /usr/share/nginx/html/
3
[email protected]:/usr/share/nginx/html# ls
4
50x.html index.html
5
[email protected]:/usr/share/nginx/html# cat > index.html
6
<h1>MY-NGINX</h1>
7
[email protected]:/usr/share/nginx/html# ls
8
50x.html index.html
9
[email protected]:/usr/share/nginx/html# cat index.html
10
<h1>MY-NGINX</h1>
Copied!
Now lets ctrl+p and then ctlr+q to exit from the container without stopping that.
1
[[email protected] ~]# docker ps
2
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
3
bc741086bb8b nginx "/docker-entrypoint.…" 9 minutes ago Up 9 minutes 80/tcp my_nginxtemp
Copied!
and finally lets creating a new image from this running container using commit command:
1
[[email protected] ~]# docker commit my_nginxtemp my_nginx
2
sha256:6e7586d227a5ad56906d8a2f14070621249a8ed7e53c4ee16275c2781ba35e96
Copied!
and see the result:
1
[[email protected] ~]# docker image ls | grep my_nginx
2
my_nginx latest 6e7586d227a5 3 minutes ago 133MB
Copied!
Now we can run as many containers as we like from this image.
.
.
Previous
Docker Containers
Next
Docker Storage and volumes
Last modified 8mo ago
Copy link
Contents
What's a docker image?
container images are big
how overlays work
Understanding Layering with Docker Images
The storage location of Docker images and containers
Docker File
Dockerfile instructions:
Listing images
Pulling an image from default registry
Remove one or more specific images
Remove dangling images
tagging images
storing images in Docker Registry
pushing an image to the Default Registery
Searching for an image
Saving and loading images
Commiting changes to an image