Sunday, 3 March 2019

3 . Docker Series - Deep Dive Images and Containers




            Deep Dive Images and Containers


For premium DevOps and AWS Courses: https://imojo.in/2cox7em
  1. AWS with the project 
  2. DevOps Real-Time 
  3. DevOps with the project 
  4. AWS Real-Time
  5. Linux Admin

Image Layer





Union Mounts

Ability to mount multiple file systems on top of each other.
If we are having same file with different data in multiple layers, then top layer data overwrites all.


When we run a container, whatever we write it forms the uppermost layer.
IMAGE Union mounts 2










Image Layer and OverlayFS


#docker pull ubuntu
Using default tag: latest
latest: Pulling from library/ubuntu
d54efb8db41d: Pull complete
f8b845f45a87: Pull complete
e8db7bf7c39f: Pull complete
9654c40e9079: Pull complete
6d9ef359eaaa: Pull complete
Digest: sha256:dd7808d8792c9841d0b460122f1acf0a2dd1f56404f8d1e56298048885e45535
Status: Downloaded newer image for ubuntu:latest


Each image layer has its own directory under /var/lib/docker/overlay/. This is where the contents of each image layer are stored.
The output of the command below shows the five directories that store the contents of each image layer just pulled. However, as can be seen, the image layer IDs do not match the directory names in             .

[root@docker overlay]# ls -ltr
total 0
drwx------. 3 root root 18 Apr  1 07:44 0e2c57fb6e308cbef7250967f66fd14c8d49f4422e758ce9cd0b2d3f05b3e771
drwx------. 3 root root 18 Apr  1 07:44 2d0d4233891dd532127336c470ec8cb41d0b95d140ee946a73b336530a742d5f
drwx------. 3 root root 18 Apr  1 07:44 0179b7a9394b83d47f88fe23abe0e9fa529c245506434fc32f1ea39179898737
drwx------. 3 root root 18 Apr  1 07:44 43696dfe7b0b4d5d77e94a3747435498a3819fecec846e91c2f48c34bd1c2428
drwx------. 3 root root 18 Apr  1 07:44 90c0ed5454b33d404b5600ef02c5efa78dc5b470cf03b2005a34ca489883f772


Now make a change in a container and find it on the overlayFS.

#docker run -it 0ef2e08ed3fa
0ef2e08ed3fa #apt-get update
0ef2e08ed3fa #apt-get install vim
0ef2e08ed3fa #vi /tmp/testfile
This is a test file.
Now press “Ctrl P Q”

# mount | grep overlay
/dev/mapper/cl-root on /var/lib/docker/overlay type xfs (rw,relatime,seclabel,attr2,inode64,noquota)
overlay on /var/lib/docker/overlay/0cab383e563082ef352c77e2eb186f136fdd484b6a98a0c4edb8e7caac231f07/merged type overlay (rw,relatime,seclabel,lowerdir=/var/lib/docker/overlay/90c0ed5454b33d404b5600ef02c5efa78dc5b470cf03b2005a34ca489883f772/root,upperdir=/var/lib/docker/overlay/0cab383e563082ef352c77e2eb186f136fdd484b6a98a0c4edb8e7caac231f07/upper,workdir=/var/lib/docker/overlay/0cab383e563082ef352c77e2eb186f136fdd484b6a98a0c4edb8e7caac231f07/work)



The “lower-id” file contains the ID of the top layer of the image the container is based on. This is used by OverlayFS as the “lowerdir”.
The “upper” directory is the containers read-write layer. Any changes made to the container are written to this directory.
The “merged” directory is effectively the containers mount point. This is where the unified view of the image (“lowerdir”) and container (“upperdir”) is exposed. Any changes written to the container are immediately reflected in this directory.
The “work” directory is required for OverlayFS to function. It is used for things such as copy_up operations.

# cat /var/lib/docker/overlay/0cab383e563082ef352c77e2eb186f136fdd484b6a98a0c4edb8e7caac231f07/merged/tmp/testfile
this is a testfile





Copying Image to other hosts


Now stop the above container
# docker ps
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES
d956f56f2ac5        0ef2e08ed3fa        "/bin/bash"         19 minutes ago      Up 19 minutes                           angry_wilson

Take the container id from above and stop it

# docker stop d956f56f2ac5
d956f56f2ac5

After stopping commit it and give it a name.
# docker commit d956f56f2ac5 testfile

sha256:fa914368ee56dae3b5b0c843ac407dc6ed2481b6116aad665df092b1348cf668

# docker images
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
testfile            latest              fa914368ee56        8 seconds ago       228 MB
centos              latest              98d35105a391        2 weeks ago         192 MB
ubuntu              latest              0ef2e08ed3fa        4 weeks ago         130 MB

So, the size of testfile is increased by 36MB, as we have updated the server.

Run the history command to check it.

# docker history testfile
IMAGE               CREATED             CREATED BY                                      SIZE                COMMENT
fa914368ee56        10 minutes ago      /bin/bash                                       97.9 MB
0ef2e08ed3fa        4 weeks ago         /bin/sh -c #(nop)  CMD ["/bin/bash"]            0 B
<missing>           4 weeks ago         /bin/sh -c mkdir -p /run/systemd && echo '...   7 B
<missing>           4 weeks ago         /bin/sh -c sed -i 's/^#\s*\(deb.*universe\...   1.9 kB
<missing>           4 weeks ago         /bin/sh -c rm -rf /var/lib/apt/lists/*          0 B
<missing>           4 weeks ago         /bin/sh -c set -xe   && echo '#!/bin/sh' >...   745 B
<missing>           4 weeks ago         /bin/sh -c #(nop) ADD file:efb254bc677d66d...   130 MB

So, we started the container by /bin/bash which is getting reflected here. And what was the size of the change.

Now save the image as tar ball. Here, even if we don’t mention file name.tar, docker will create tar ball by default.

# docker save -o /tmp/tesfile.tar testfile
# ls -ltr /tmp/tesfile.tar
-rw-------. 1 root root 235798528 Apr  1 09:56 /tmp/tesfile.tar

#du -sh /tmp/tesfile.tar
225M    /tmp/tesfile.tar


Transfer the file to other Ubuntu server
# scp /tmp/tesfile.tar 192.168.56.226:/tmp/


To check all the file inside the tar
# tar -tf /tmp/tesfile.tar

# docker images
REPOSITORY                           TAG                 IMAGE ID            CREATED             SIZE

Load the image

# docker load -i /tmp/tesfile.tar
9d3227c1793b: Loading layer  121.3MB/121.3MB
a1a54d352248: Loading layer  15.87kB/15.87kB
511ddc11cf68: Loading layer  11.78kB/11.78kB
08f405d988e4: Loading layer  5.632kB/5.632kB
73e5d2de6e3e: Loading layer  3.072kB/3.072kB
7738492b326d: Loading layer  97.87MB/97.87MB
Loaded image: testfile:latest


# docker images
REPOSITORY                           TAG                 IMAGE ID            CREATED             SIZE
testfile                             latest              00a5cfa72dbd        10 minutes ago      213MB

# docker run -it testfile /bin/bash

root@4880f241a8f1:/# cat /tmp/testfile
this is a testfile





More about Containers


Image: Build time construct

Container: Run time construct





  • Rootfs is read only
  • Kernel is shared by all the containers
  • Thin writable layer



One process per Container

We can check all the top processes running on a container
# docker run -it ubuntu /bin/bash
# docker ps
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES
72f512e7e091        ubuntu              "/bin/bash"         9 seconds ago       Up 9 seconds                            dazzling_hypatia
# docker top 72f512e7e091
UID                 PID                 PPID                C                   STIME               TTY                 TIME                CMD
root                10816               10805               0                   10:44               pts/1               00:00:00            /bin/bash

#docker inspect 72f512e7e091




Click below to view more :