Debugging a containerized application

Computer programs (software) sometimes fail to behave as expected. This is due to faulty code or due to the environmental changes between the development, testing, and deployment systems. Docker container technology eliminates the environmental issues between development, testing, and deployment as much as possible by containerizing all the application dependencies. Nonetheless, there could be still anomalies due to faulty code or variations in the kernel behavior, which needs debugging. Debugging is one of the most complex processes in the software engineering world and it becomes much more complex in the container paradigm because of the isolation techniques. In this section, we are going to learn a few tips and tricks to debug a containerized application using the tools native to Docker, as well as the tools provided by external sources.

Initially, many people in the Docker community individually developed their own debugging tools, but later Docker started supporting native tools, such as exec, top, logs, events, and many more. In this section, we will dive deep into the following Docker tools:

  • exec
  • ps
  • top
  • stats
  • events
  • logs

The Docker exec command

The docker exec command provided the much-needed help to users, who are deploying their own web servers or other applications running in the background. Now, it is not necessary to log in to run the SSH daemon in the container.

First, run the docker ps -a command to get the container ID:

$ sudo docker ps -a
b34019e5b5ee        nsinit:latest             "make local"           
a245253db38b        training/webapp:latest    "python app.py"        

Then, run the docker exec command to log in to the container.

$ sudo docker exec -it a245253db38b bash
root@a245253db38b:/opt/webapp#

It is important to note that the docker exec command can only access the running containers, so if the container stops functioning, then you need to restart the stopped container in order to proceed. The docker exec command spawns a new process in the target container using the Docker API and CLI. So if you run the pe -aef command inside the target container, it looks like this:

# ps -aef
UID        PID  PPID  C STIME TTY          TIME CMD
root         1     0  0 Mar22 ?        00:00:53 python app.py
root        45     0  0 18:11 ?        00:00:00 bash
root        53    45  0 18:11 ?        00:00:00 ps -aef

Here, python app.y is the application that is already running in the target container, and the docker exec command has added the bash process inside the container. If you run kill -9 pid(45), you will be automatically logged out of the container.

If you are an enthusiastic developer, and you want to enhance the exec functionality, you can refer to https://github.com/chris-rock/docker-exec.

It is recommended that you use the docker exec command only for monitoring and diagnostic purposes, and I personally believe in the concept of one process per container, which is one of the best practices widely accentuated.

The Docker ps command

The docker ps command, which is available inside the container, is used to see the status of the process. This is similar to the standard ps command in the Linux environment and is not a docker ps command that we run on the Docker host machine.

This command runs inside the Docker container:

root@5562f2f29417:/# ps –s
  UID   PID   PENDING   BLOCKED   IGNORED    CAUGHT STAT TTY        TIME COMMAND
    0     1  00000000  00010000  00380004  4b817efb Ss   ?          0:00 /bin/bash
    0    33  00000000  00000000  00000000  73d3fef9 R+   ?          0:00 ps -s
root@5562f2f29417:/# ps -l
F S   UID   PID  PPID  C PRI  NI ADDR SZ WCHAN  TTY          TIME CMD
4 S     0     1     0  0  80   0 -  4541 wait   ?        00:00:00 bash
0 R     0    34     1  0  80   0 -  1783 -      ?        00:00:00 ps
root@5562f2f29417:/# ps -t
  PID TTY      STAT   TIME COMMAND
    1 ?        Ss     0:00 /bin/bash
   35 ?        R+     0:00 ps -t
root@5562f2f29417:/# ps -m
  PID TTY          TIME CMD
    1 ?        00:00:00 bash
    - -        00:00:00 -
   36 ?        00:00:00 ps
    - -        00:00:00 -
root@5562f2f29417:/# ps -a
  PID TTY          TIME CMD
   37 ?        00:00:00 ps

Use ps --help <simple|list|output|threads|misc|all> or ps --help <s|l|o|t|m|a> for additional help text.

The Docker ps command

The Docker top command

You can run the top command from the Docker host machine using the following command:

docker top [OPTIONS] CONTAINER [ps OPTIONS]

This gives a list of the running processes of a container without logging into the container, as follows:

$ sudo docker top  a245253db38b
UID                 PID                 PPID                C
STIME               TTY                 TIME                CMD
root                5232                3585                0
Mar22               ?                   00:00:53            python app.py  
$ sudo docker top  a245253db38b  -aef
UID                 PID                 PPID                C
STIME               TTY                 TIME                CMD
root                5232                3585                0
Mar22               ?                   00:00:53            python app.py

The Docker top command provides information about the CPU, memory, and swap usage if you run it inside a Docker container:

root@a245253db38b:/opt/webapp# top
top - 19:35:03 up 25 days, 15:50,  0 users,  load average: 0.00, 0.01, 0.05
Tasks:   3 total,   1 running,   2 sleeping,   0 stopped,   0 zombie
Cpu(s):  0.0%us,  0.0%sy,  0.0%ni, 99.9%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:   1016292k total,   789812k used,   226480k free,    83280k buffers
Swap:        0k total,        0k used,        0k free,   521972k cached
  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
    1 root      20   0 44780  10m 1280 S  0.0  1.1   0:53.69 python
   62 root      20   0 18040 1944 1492 S  0.0  0.2   0:00.01 bash
   77 root      20   0 17208 1164  948 R  0.0  0.1   0:00.00 top

In case you get the error error - TERM environment variable not set while running the top command inside the container, perform the following steps to resolve it:

Run the echo $TERM command. You will get the result as dumb. Then, run the following command:

$ export TERM=dumb 

This will resolve the error.

The Docker stats command

The Docker stats command provides you with the capability to view the memory, CPU, and the network usage of a container from a Docker host machine, as illustrated here:

$ sudo docker stats a245253db38b
CONTAINER           CPU %               MEM USAGE/LIMIT       MEM %
  NET I/O
a245253db38b        0.02%               16.37 MiB/992.5 MiB   1.65%
  3.818 KiB/2.43 KiB

You can run the stats command to also view the usage for multiple containers:

$ sudo docker stats a245253db38b f71b26cee2f1

In the latest release of Docker 1.5, Docker provides you access to container statistics read only parameters. This will streamline the CPU, memory, network IO, and block IO of your containers. This helps you choose the resource limits and also in profiling. The Docker stats utility provides you with these resource usage details only for running containers. You can get detailed information using the end point APIs at https://docs.docker.com/reference/api/docker_remote_api_v1.17/#inspect-a-container.

Docker stats is originally taken from Michael Crosby's code contribution, which can be accessed at https://github.com/crosbymichael.

The Docker events command

Docker containers will report the following real-time events: create, destroy, die, export, kill, omm, pause, restart, start, stop, and unpause. The following are a few examples that illustrate how to use these commands:

$ sudo docker pause  a245253db38b
a245253db38b
$ sudo docker ps -a
a245253db38b        training/webapp:latest    "python app.py"        4 days ago         Up 4 days (Paused)       0.0.0.0:5000->5000/tcp   sad_sammet
$ sudo docker unpause  a245253db38b
a245253db38b
$ sudo docker ps -a
a245253db38b        training/webapp:latest    "python app.py"        4 days ago    Up 4 days        0.0.0.0:5000->5000/tcp   sad_sammet

The Docker image will also report the untag and delete events.

Using multiple filters will be handled as an AND operation; for example, --filter container= a245253db38b --filter event=start will display events for the container a245253db38b and the event type is start.

Currently, the supported filters are container, event, and image.

The Docker logs command

This command fetches the log of a container without logging into the container. It batch-retrieves logs present at the time of execution. These logs are the output of STDOUT and STDERR. The general usage is shown in docker logs [OPTIONS] CONTAINER.

The –follow option will continue to provide the output till the end, -t will provide the timestamp, and --tail= <number of lines> will show the number of lines of the log messages of your container:

$ sudo docker logs a245253db38b
 * Running on http://0.0.0.0:5000/
172.17.42.1 - - [22/Mar/2015 06:04:23] "GET / HTTP/1.1" 200 -
172.17.42.1 - - [24/Mar/2015 13:43:32] "GET / HTTP/1.1" 200 -
$
$ sudo docker logs -t a245253db38b
2015-03-22T05:03:16.866547111Z  * Running on http://0.0.0.0:5000/
2015-03-22T06:04:23.349691099Z 172.17.42.1 - - [22/Mar/2015 06:04:23] "GET / HTTP/1.1" 200 -
2015-03-24T13:43:32.754295010Z 172.17.42.1 - - [24/Mar/2015 13:43:32] "GET / HTTP/1.1" 200 -

We also used the docker logs utility in Chapter 2, Handling Docker Containers and Chapter 7, Running Services in a Container, to view the logs of our containers.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset