×
Community Blog Practical Exercises for Docker Compose: Part 3

Practical Exercises for Docker Compose: Part 3

This set of tutorials focuses on giving you practical experience on using Docker Compose when working with containers on Alibaba Cloud.

By Alwyn Botha, Alibaba Cloud Tech Share Author. Tech Share is Alibaba Cloud's incentive program to encourage the sharing of technical knowledge and best practices within the cloud community.

This set of tutorials focuses on giving you practical experience on using Docker Compose when working with containers on Alibaba Cloud Elastic Compute Service (ECS).

Part 2 of this series explored several Docker compose configurations. In Part 3, we will explore depends_on, volumes and the important init docker-compose options.

depends_on

depends_on declares dependencies between services.

depends_on let docker-compose up starts services in dependency order.

The dependent services start first, before the main service starts.

As you will see the dependencies start in random order. Also, docker-compose does not wait for any dependencies to be running before starting the main service.

Let's see that in action so you can understand how it works:

Add the following to your docker-compose.yml using

nano docker-compose.yml
version: "3.7"

services:

alpine:

image: alpine:3.8

command: sleep 600

depends_on:

- service-2

- service-3

- service-4

container_name: main-container

service-2:

image: alpine:3.8

container_name: service-2

command: sleep 600

service-3:

image: alpine:3.8

container_name: service-3

command: sleep 600

service-4:

image: alpine:3.8

container_name: service-4

command: sleep 600

Please note the dependencies are declared in neat number order.

Let's start up this docker-compose file using:

docker-compose up -d -t 0

Expected output :

Creating service-3 ... done

Creating service-4 ... done

Creating service-2 ... done

Recreating compose-tuts_alpine_1 ... done

Note that dependencies started in random order. The main service started last - not clearly shown in above output.

If you make small edits to the docker-compose file ( such as sleep time edits ) and redo the docker-compose up you will see those services created in different sequences each time. Try it.

To more clearly see the sequence of events, let us shut down all services an redo the docker-compose up.

docker-compose down -t 0

Expected output :

Stopping main-container ... done

Stopping service-4  ... done

Stopping service-3  ... done

Stopping service-2  ... done

Removing main-container ... done

Removing service-4  ... done

Removing service-3  ... done

Removing service-2  ... done

Removing network compose-tuts_default

Note the main-container gets stopped first, then the dependencies.

Now run

docker-compose up -d -t 0

Expected output :

Creating network "compose-tuts_default" with the default driver

Creating service-3 ... done

Creating service-2 ... done

Creating service-4 ... done

Creating main-container ... done

Clearly shows main-container created last.

If you need you dependency services to be READY before the main container starts you need to study https://docs.docker.com/compose/startup-order/

volumes

volumes are used to mount named volumes.

To make this interesting we will reuse the 4 services from the previous example.

All 4 services will use the named volume: demo-data-volume. It does not exist yet.

Add the following to your docker-compose.yml using

nano docker-compose.yml
version: "3.7"

services:

alpine:

image: alpine:3.8

command: sleep 600

depends_on:

- service-2

- service-3

- service-4

container_name: main-container

service-2:

image: alpine:3.8

container_name: service-2

command: sleep 600

volumes:

- demo-data-volume:/root/dir1/dir2

service-3:

image: alpine:3.8

container_name: service-3

command: sleep 600

volumes:

- demo-data-volume:/root/dira/dirb

service-4:

image: alpine:3.8

container_name: service-4

command: sleep 600

volumes:

- demo-data-volume:/root/1/2/3/4

volumes:

demo-data-volume:

Right at the bottom: that is the top level volumes key ( as named in the Docker documentation ).

Top level keys start at extreme left side of the docker-compose file.

Our top level key defines demo-data-volume - the named volume we want to use.

When we run

docker-compose up -d -t 0

demo-data-volume will be created ( if it does not exist ).

Expected output :

Creating network "compose-tuts_default" with the default driver

Creating volume "compose-tuts_demo-data-volume" with default driver

Creating service-2 ... done

Creating service-4 ... done

Creating service-3 ... done

Creating main-container ... done

Note second line: Creating volume "compose-tuts_demo-data-volume" with default driver

Run:

docker volume ls

Expected output :

DRIVER  VOLUME NAME

local  compose-tuts_demo-data-volume

There is our named volume. Its name is the concatenation of our current directory and the volume name we specified in our docker-compose file.

In our docker-compose file we defined that we want to use this named volume at different mount points in 3 services.

service-2:

volumes:

- demo-data-volume:/root/dir1/dir2

service-3:

volumes:

- demo-data-volume:/root/dira/dirb

service-4:

volumes:

- demo-data-volume:/root/1/2/3/4

We are now going to see these 3 services all have access to the same named volume.

Let's enter service-2 and work with our named volume.

docker exec -it service-2 /bin/sh

Execute commands as shown: ls to show the path exists and echo to create myoutfile. Then exit.

/ # ls /root/dir1

dir2

/ # echo from service 2 > /root/dir1/dir2/myoutfile

/ # exit

Let's enter service-3 and investigate our named volume.

docker exec -it service-3 /bin/sh

Enter commands shown: ls to confirm that our file exists. cat it to show its content.

/ # ls /root/dira/dirb

myoutfile

/ # cat /root/dira/dirb/myoutfile

from service 2

/ # exit

Note that with service-3 the volume is mounted at /root/dira/dirb

service-3:

volumes:

- demo-data-volume:/root/dira/dirb

Note that with service-4 the volume is mounted at /root/1/2/3/4

service-4:

volumes:

- demo-data-volume:/root/1/2/3/4

Enter service-4 to confirm that our file is available there.

docker exec -it service-4 /bin/sh

Expected output :

/ # cat /root/1/2/3/4/myoutfile

from service 2

/ # exit

Our named volume is available for reading and writing inside these 3 containers. It will continue to exist even if we stop these containers.

Other containers, services and swarms are allowed to used this named volume. It is not specifically linked to these 3 services.

During these few minutes you explored probably less than 5% of Docker volume functionality.

There are more than 25 volume plugins listed at https://docs.docker.com/engine/extend/legacy_plugins/#volume-plugins

You can read the official docs at https://docs.docker.com/storage/volumes/ and https://docs.docker.com/storage/

At least now you have actually used named volumes and have some idea of how it works.

Fortunately named volumes are the easiest to understand, most flexible and most frequently used.

restart

restart is applied when starting a service using docker-compose up

It has 4 easy to understand options:

  1. restart: "no" ... the default value
  2. restart: always
  3. restart: on-failure
  4. restart: unless-stopped

Let's experiment with those:

Demo for restart: "no" ... the default value

Add the following to your docker-compose.yml using

nano docker-compose.yml
version: "3.7"

services:

alpine:

image: alpine:3.8

command: sleep 1

Bring up the service:

docker-compose up -d -t 0

Check its status:

docker ps -a

The sleep 1 causes the container to exit after just one second.

You will only see one exited container - even if you rerun docker ps -a many times. As expected: restart = no does not restart containers.

Demo for restart: unless-stopped

Add the following to your docker-compose.yml using

nano docker-compose.yml
version: "3.7"

services:

alpine:

image: alpine:3.8

command: sleep 1

restart: unless-stopped

Prune previous container - so we have a clean docker ps -a to work from.

docker container prune -f;docker ps -a

Bring up new container.

docker-compose up -d -t 0

Check its status:

docker ps -a

The sleep 1 causes the container to exit after just one second.

If you run this command repeatedly you will see the container continually restarting.

docker container stop your-container-id does not stop it.

docker-compose down removes the container within seconds.

If you use docker-compose up to start up a container, use docker-compose down to take it down.

restart: unless-stopped work as expected.

Demo for restart: always

Add the following to your docker-compose.yml using

nano docker-compose.yml
version: "3.7"

services:

alpine:

image: alpine:3.8

command: sleep 1

restart: always

Bring up new container.

docker-compose up -d -t 0

Check its status:

docker ps -a

The sleep 1 causes the container to exit after just one second.

If you run this command repeatedly you will see the container continually restarting.

docker container stop your-container-id does not stop it.

docker-compose down removes the container within seconds.

If you use docker-compose up to start up a container, use docker-compose down to take it down.

restart: always restarts containers that exit with zero ( success ) exit code.

If you replace command with:

command: sleep 1; exit 1

you can test that restart: always restarts containers that exit with non-zero ( failed ) exit code.

Demo for restart: on-failure

You learn new software by reading the docs and then using it.

Reading means zero experience.

Experiments means experience.

So based on what you experienced above, design and execute tests for restart: on-failure

It must test container restart on failure ( obviously )

It must also test if it will restart on successful exits ( exit return code 0 ).

Based on the above texts this should take 5 minutes maximum.

init: not set to true

From https://docs.docker.com/compose/compose-file/#init

Run an init inside the container that forwards signals and reaps processes. Set init to true to use the default init.

Having this option set to true is important if you care about having processes inside your container shut down cleanly.

If you run docker container stop the processes inside the container must receive the correct signals forwarded to them. Then the shutdown subroutines for your applications are correctly signaled to shutdown cleanly.

If you have several programs or threads running in your container they must receive the STOP signal correctly managed by some overall manager process: INIT is that process.

You will see the explanation below demonstrated in real life example:

Init first sends SIGTERM ( signal 15 ) to let processes catch that signal to execute their clean shutdown routines. SIGTERM means - you must shut down, but do you clean shutdown routines first.

stop_grace_period = 10 seconds default.

stop_grace_period then waits 10 seconds for the container to exit CLEANLY before sending SIGKILL ( signal 9). SIGKILL immediately kills the proces - files not closed properly - no shutdown routines done. SIGKILL means process is running one moment; brains blown out the next.

Only with init: true does this critical important signals propagate properly through your container.

Summary : specify init: true if you care about preventing data corruption.

Important: This option got added in version 3.7 file format. The first line of your docker-compose.yml must read: version: "3.7"

First we will see how no init setting causes STOP to be handled badly.

Then we do it correctly by specifying init: true and observing the difference.

Add the following to your docker-compose.yml using

nano docker-compose.yml
version: "3.7"

services:

alpine:

image: alpine:3.8

Bring up new container ( no init anywhere ).

docker-compose up -d -t 0

Check its status and get container id:

docker ps -a

docker exec -it  your-container-id  /bin/sh

Enter it and run ps

/ # ps

PID  USER  TIME  COMMAND

1 root  0:00 sleep 600

11 root  0:00 /bin/sh

16 root  0:00 ps

/ # exit

Note PID process 1 is not init.

Open a new shell console and run docker events

Back to original shell, if you now run

docker container stop your-container-id

you will see it takes 10 seconds to stop.

IMPORTANT: Look at the console output for docker events

Expected output :

2018-11-12T09:51:46.112995194+02:00 container kill b676e3bf51f9f1f99edc531ffa04b0ab164834cb218fe9b78abb65b396782d6a (com.docker.compose.config-hash=c19f3ca3bb22826fee98596d3cb40c4f403f8a51cede518945f5ef37bb989589, com.docker.compose.container-number=1, com.docker.compose.oneoff=False, com.docker.compose.project=compose-tuts, com.docker.compose.service=alpine, com.docker.compose.version=1.22.0, image=alpine:3.8, name=compose-tuts_alpine_1, signal=15)

2018-11-12T09:51:56.134707392+02:00 container kill b676e3bf51f9f1f99edc531ffa04b0ab164834cb218fe9b78abb65b396782d6a (com.docker.compose.config-hash=c19f3ca3bb22826fee98596d3cb40c4f403f8a51cede518945f5ef37bb989589, com.docker.compose.container-number=1, com.docker.compose.oneoff=False, com.docker.compose.project=compose-tuts, com.docker.compose.service=alpine, com.docker.compose.version=1.22.0, image=alpine:3.8, name=compose-tuts_alpine_1, signal=9)

2018-11-12T09:51:56.307015745+02:00 container die b676e3bf51f9f1f99edc531ffa04b0ab164834cb218fe9b78abb65b396782d6a (com.docker.compose.config-hash=c19f3ca3bb22826fee98596d3cb40c4f403f8a51cede518945f5ef37bb989589, com.docker.compose.container-number=1, com.docker.compose.oneoff=False, com.docker.compose.project=compose-tuts, com.docker.compose.service=alpine, com.docker.compose.version=1.22.0, exitCode=137, image=alpine:3.8, name=compose-tuts_alpine_1)

2018-11-12T09:51:56.387969293+02:00 network disconnect 9802e2506ea80e7597f0b018450f86fa0c7dcc045a920702cc0e463aacfda84f (container=b676e3bf51f9f1f99edc531ffa04b0ab164834cb218fe9b78abb65b396782d6a, name=compose-tuts_default, type=bridge)

2018-11-12T09:51:56.454351535+02:00 container stop b676e3bf51f9f1f99edc531ffa04b0ab164834cb218fe9b78abb65b396782d6a (com.docker.compose.config-hash=c19f3ca3bb22826fee98596d3cb40c4f403f8a51cede518945f5ef37bb989589, com.docker.compose.container-number=1, com.docker.compose.oneoff=False, com.docker.compose.project=compose-tuts, com.docker.compose.service=alpine, com.docker.compose.version=1.22.0, image=alpine:3.8, name=compose-tuts_alpine_1)

First event sends container SIGTERM - signal 15 text right at end of that line.

Docker determines container still does not shut down properly. Container ignores signal 15.

Second line: KILL signal event happens 10 seconds later - signal 9 text right at end of that line.

Docker kills container with signal 9 = KILL.

Summary: no init, no orderly shutdown, KILL had to be used.

init: true

Now observe how init handles the stop properly:

Add the following to your docker-compose.yml using

nano docker-compose.yml
version: "3.7"

services:

alpine:

image: alpine:3.8

command: sleep 600

init: true

Bring up new container with init: true

docker-compose up -d -t 0

Check its status and get container id:

docker ps -a

docker exec -it  your-container-id  /bin/sh

Enter it and run ps

/ # ps

PID  USER  TIME  COMMAND

1 root  0:00 /dev/init -- sleep 600

6 root  0:00 sleep 600

7 root  0:00 /bin/sh

12 root  0:00 ps

/ # exit

Note PID 1 runs /dev/init. Init is running PID 1: the only perfect place to manage sending signals to all processes in the container.

Press CTRL-c in the docker events console so that list of events are interrupted.

In this console run docker events again - now we going to observe INIT-managed shutdown events.

Back to original shell, do docker ps -a to get your container id.

If you now run

docker container stop your-container-id

you will see it stops very quickly.

IMPORTANT: Look at the console output for docker events

Expected output :

2018-11-12T09:50:14.892684432+02:00 container kill 7973ccec4849cf8759821d613e1b4870c68d76f396fefcfb1cd1bf1f7a3bd509 (com.docker.compose.config-hash=8d5184879de73d9f624319983822eafca40959cbe9f91929ff4648ace18acc6a, com.docker.compose.container-number=1, com.docker.compose.oneoff=False, com.docker.compose.project=compose-tuts, com.docker.compose.service=alpine, com.docker.compose.version=1.22.0, image=alpine:3.8, name=compose-tuts_alpine_1, signal=15)

2018-11-12T09:50:15.019039627+02:00 container die 7973ccec4849cf8759821d613e1b4870c68d76f396fefcfb1cd1bf1f7a3bd509 (com.docker.compose.config-hash=8d5184879de73d9f624319983822eafca40959cbe9f91929ff4648ace18acc6a, com.docker.compose.container-number=1, com.docker.compose.oneoff=False, com.docker.compose.project=compose-tuts, com.docker.compose.service=alpine, com.docker.compose.version=1.22.0, exitCode=143, image=alpine:3.8, name=compose-tuts_alpine_1)

2018-11-12T09:50:15.104203647+02:00 network disconnect 9802e2506ea80e7597f0b018450f86fa0c7dcc045a920702cc0e463aacfda84f (container=7973ccec4849cf8759821d613e1b4870c68d76f396fefcfb1cd1bf1f7a3bd509, name=compose-tuts_default, type=bridge)

2018-11-12T09:50:15.149018229+02:00 container stop 7973ccec4849cf8759821d613e1b4870c68d76f396fefcfb1cd1bf1f7a3bd509 (com.docker.compose.config-hash=8d5184879de73d9f624319983822eafca40959cbe9f91929ff4648ace18acc6a, com.docker.compose.container-number=1, com.docker.compose.oneoff=False, com.docker.compose.project=compose-tuts, com.docker.compose.service=alpine, com.docker.compose.version=1.22.0, image=alpine:3.8, name=compose-tuts_alpine_1)

First event sends container SIGTERM - signal 15 text right at end of that line.

Docker determines container does shut down properly. Container properly processes signal 15. ( Container processes caught signal 15 and called their clean shutdown routines. )

The container die events happens 100 milliseconds later.

NO signal 9 = KILL sent.

Summary: init, fast orderly shutdown, no KILL used.

In short, I strongly suggesting using init: true. It's easy, fast and saves corrupted data hassles.

0 0 0
Share on

Alibaba Clouder

1,632 posts | 276 followers

You may also like

Comments