13

Dockerizing the Django Project

In the previous chapter, we learned more about software deployment, and we deployed the Django application on an AWS server. However, we came across issues such as poor preparation of the project for deployment, violation of some security issues, and deployment and development configuration.

In this chapter, we will learn how to use Docker on the Django backend and configure environment variables. We will also configure the database on a web server called NGINX using Docker. Here are the big sections of the chapter:

• What is Docker?
• Dockerizing the Django application
• Using Docker Compose for multiple containers
• Configuring environment variables in Django
• Writing NGINX configuration

Technical requirements

For this chapter, you will need to have Docker and Docker Compose installed on your machine. The Docker official documentation has a well-documented process for the installation on any OS platform. You can check it out at https://docs.docker.com/engine/install/.

The code written in this chapter can also be found at https://github.com/PacktPublishing/Full-stack-Django-and-React/tree/chap13.

What is Docker?

Before defining what Docker is, we must understand what a container is and its importance in today’s tech ecosystem. To make it simple, a container is a standard unit of software that packages up the software and all of its required dependencies so that the software or the application can run quickly and reliably from one machine to another, whether the environment or the OS.

An interesting definition from Solomon Hykes at the 2013 PyCon talk is: containers are “self-contained units of software you can deliver from a server over there to a server over there, from your laptop to EC2 to a bare-metal giant server, and it will run in the same way because it is isolated at the process level and has its own file system.”

Important note

Containerization is different from virtualization. Virtualization enables teams to run multiple operating systems on the same hardware, while containerization allows teams to deploy multiple applications using the same operating system on single hardware with their own images and dependencies.

Great, right? Remember at the beginning of this book when we had to make configurations and installations depending on the OS mostly for the Python executable, the Postgres server, and different commands to create and activate a virtual environment? Using Docker, we can have a single configuration for a container, and this configuration can run the same on any machine. Docker ensures that your application can be executed in any environment. Then, we can say that Docker is a software platform for building, developing, and developing applications inside containers. It has the following advantages:

• Minimalistic and portable: Compared to virtual machines (VMs) that require complete copies of an OS, the application, and the dependencies, which can take a lot of space, a Docker container requires less storage because the image used comes with megabytes (MB) in size. This makes them fast to boot and easily portable even on small devices such as Raspberry Pi-embedded computers.
• Docker containers are scalable: Because they are lightweight, developers or DevOps can launch a lot of services based on containers and easily control the scaling using tools such as Kubernetes.
• Docker containers are secure: Applications inside Docker containers are running isolated from each other. Thus, a container can’t check the processes running in another container.

With a better understanding of what Docker is, we can now move on to integrate Docker into the Django application.

Dockerizing the Django application

In the precedent section, we defined Docker and its advantages. In this section, we will configure Docker with the Django application. This will help you understand better how Docker works under the hood.

Adding a Docker image

A characteristic of projects that use Docker is the presence of files called Dockerfiles in the project. A Dockerfile is a text document that contains all the commands necessary to assemble a Docker image. A Docker image is a read-only template with instructions to create a Docker container.

Creating an image with a Dockerfile is the most popular way to go as you only need to enter the instructions you will require to set up an environment, install the package, make migrations, and a lot more. This is what makes Docker very portable. For example, in the case of our Django application, we will write the Dockerfile based on an existing image for Python 3.10 based on the popular Alpine Linux project (https://alpinelinux.org/). This image has been chosen because of its small size, equal to 5 MB. Inside the Dockerfile, we will also add commands to install Python and Postgres dependencies, and we will further add commands to install packages. Let’s get started with the steps:

1. Start by creating a new file at the root of the Django project called Dockerfile and adding the first line:

Dockerfile

FROM python:3.10-alpine
# pull official base image

Most of your Dockerfile will start with this line. Here, we are telling Docker which image to use to build our image. The python:3.10-alpine image is stored in what is called a Docker registry. This is a storage and distribution system for Docker images, and you can find the most popular one online, called Docker Hub, at https://hub.docker.com/.

2. Next, let’s set the working directory. This directory will contain the code of the running Django project:

Dockerfile

WORKDIR /app

3. As the Django application uses Postgres as a database, add the required dependencies for Postgres and Pillow to our Docker image:

Dockerfile

# install psycopg2 dependencies
RUN apk update 
    && apk add postgresql-dev gcc python3-dev musl-dev
    jpeg-dev zlib-dev

4. Then, install the Python dependencies after making a copy of the requirements.txt file in the /app working directory:

Dockerfile

# install python dependencies
COPY requirements.txt /app/requirements.txt
RUN pip install --upgrade pip
RUN pip install --no-cache-dir -r requirements.txt

5. After that, copy over the whole project itself:

Dockerfile

# add app
COPY . .

6. And finally, expose port 8000 of the container for access to the other applications or the machine, run the migrations, and start the Django server:

Dockerfile

EXPOSE 8000
CMD ["python", "manage.py", "migrate"]
CMD ["python", "manage.py", "runserver", "0.0.0.0:8000"]

The Dockerfile file will have the following final code:

Dockerfile

# pull official base image
FROM python:3.10-alpine
# set work directory
WORKDIR /app
# set environment variables
ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONUNBUFFERED 1
# install psycopg2 dependencies
RUN apk update 
   && apk add postgresql-dev gcc python3-dev musl-dev
# install python dependencies
COPY requirements.txt /app/requirements.txt
RUN pip install --upgrade pip
RUN pip install --no-cache-dir -r requirements.txt
# copy project
COPY . .
EXPOSE 8000
CMD ["python", "manage.py", "migrate"]
CMD ["python", "manage.py", "runserver", "0.0.0.0:8000"]

We have just written the steps to build an image for the Django application. Let’s build the image with the following command.

docker build -t django-postagram .

The preceding command uses the Dockerfile to build a new container image—that’s why we have a dot (.) at the end of the command. It tells Docker to look for the Dockerfile in the current directory. The -t flag is used to tag the container image. Then, we are building an image with the django-backend tag using the Dockerfile we have written. Once the image is built, we can now run the application in the container by running the following command:

docker run --name django-postagram -d -p 8000:8000 django-postagram:latest

Let’s describe the preceding command:

• --name will set the name of the Docker container
• -d makes the image run in detached mode, meaning that it can run in the background
• django-postagram specifies the name of the image to use

After typing the preceding command, you can check the running container with the following command:

docker container ps

You will have a similar output:

Figure 13.1 – Listing Docker containers on the machine

The container is created, but it looks like it’s not working well. In your browser, go to http://localhost:8000, and you will notice that the browser returns a page with an error. Let’s check the logs for the django-postagram container:

docker logs --details django-postagram

The command will output in the terminal what is happening inside the container. You will have a similar output to this:

Figure 13.2 – Logs for the django-postagram container

Well, that’s quite normal. The container is running on its own network and doesn’t have direct access to the host machine network.

In the previous chapter, we added services for NGINX and Postgres and made the configurations. We need to also do the same with Docker; I mean, we can have two other Dockerfiles for NGINX and Postgres. And let’s be honest: it starts to become a little bit much. Imagine adding a Flask service, a Celery service, or even another database. Depending on the number n of components of your system, you will need n Dockerfiles. This is not interesting, but thankfully, Docker provides a simple solution for that called Docker Compose. Let’s explore it more.

Using Docker Compose for multiple containers

Docker Compose is a tool developed and created by the Docker team to help define configurations for multi-container applications. Using Docker Compose, we just need to create a YAML file to define the services and the command to start each service. It also supports configurations such as container name, environment setting, volume, and a lot more, and once the YAML file is written, you just need a command to build the images and spin all the services.

Let’s understand the key difference between a Dockerfile and Docker Compose: a Dockerfile describes how to build the image and run the container, while Docker Compose is used to run Docker containers. At the end of the day, Docker Compose still uses Docker under the hood, and you will—most of the time—need at least a Dockerfile. Let’s integrate Docker Compose into our workflow.

Writing the docker-compose.yaml file

Before writing the YAML file, we will have to make some changes to the Dockerfile. As we will be launching the Django server from the docker-compose file, we can remove the lines where we expose the port, run the migrations, and start the server. Inside the Dockerfile, remove the following lines of code:

Dockerfile

EXPOSE 8000
CMD ["python", "manage.py", "migrate"]
CMD ["python", "manage.py", "runserver", "0.0.0.0:8000"]

Once it’s done, create a new file called docker-compose.yaml at the root of the project. Make sure that the docker-compose.yaml file and the Dockerfile are in the same directory. The docker-compose.yaml file will describe the services of the backend application. We will need to write three services:

• NGINX: We are using NGINX as the web server. Thankfully, there is an official image available we can use to write quick configurations.
• Postgres: There is also an official image available for Postgres. We will just need to add environment variables for the database user.
• django-backend: This is the backend application we have created. We will use the Dockerfile so that Docker Compose will build the image for this service.

Let’s start writing the docker-compose.yaml file by adding the NGINX service first:

docker-compose.yaml

version: '3.8'
services:
 nginx:
   container_name: postagram_web
   restart: always
   image: nginx:latest
   volumes:
     - ./nginx.conf:/etc/nginx/conf.d/default.conf
     - uploads_volume:/app/uploads
   ports:
     - "80:80"
   depends_on:
     - api

Let’s see what is going on in the preceding code because the other services will follow a similar configuration. The first line sets the file format we are using, so it is not related to Docker Compose, just to YAML.

After that, we are adding a service called nginx:

• container_name represents, well, the name of the container.
• restart defines the container restart policy. In this case, the container is always restarted if it fails.

Concerning the restart policies for a container, you can also have:

• no: Containers will not restart automatically
• on-failure[:max-retries]: Restart the container if it exits with a nonzero exit code and provides a maximum number of attempts for the Docker daemon to restart the container
• unless-stopped: Always restart the container unless it was stopped arbitrarily or by the Docker daemon

• image: This tells Docker Compose to use the latest NGINX image available on Docker Hub.
• volumes are a way of persisting data generated and used by Docker containers. If a Docker container is deleted or removed, all its content will vanish forever. This is not ideal if you have files such as logs, images, video, or anything you want to persist somewhere because every time you remove a container, this data will vanish. Here is the syntax: /host/path:/container/path.
• ports: Connection requests coming from the host port 80 are redirected to the container port 80. Here is the syntax: host_port:container_port.
• depends_on: This tells Docker Compose to wait for some services to start before starting the service. In our case, we are waiting for the Django API to start before starting the NGINX server.

Great! Next, let’s add the service configuration for the Postgres service:

docker-compose.yaml

db:
 container_name: postagram_db
 image: postgres:14.3-alpine
 env_file: .env
 volumes:
   - postgres_data:/var/lib/postgresql/data/

We have new parameters here called env_file which specifies the path to the environment file that will be used to create the database and the user, and set the password. Let’s finally add the Django API service:

docker-compose.yaml

api:
 container_name: postagram_api
 build: .
 restart: always
 env_file: .env
 ports:
   - "8000:8000"
 command: >
   sh -c "python manage.py migrate --no-input && gunicorn
          CoreRoot.wsgi:application --bind 0.0.0.0:8000"
 volumes:
  - .:/app
  - uploads_volume:/app/uploads
 depends_on:
  - db

The build parameter in the Docker Compose file tells Docker Compose where to look for the Dockerfile. In our case, the Dockerfile is in the current directory. Docker Compose allows you to have a command parameter. Here, we are running migrations and starting the Django server using Gunicorn, which is new. gunicorn is a Python Web Server Gateway Interface (WSGI) HTTP server for Unix systems. Why use gunicorn? Most web applications run with an Apache server, so gunicorn is basically designed to run web applications built with Python.

You can install the package in your current Python environment by running the following command:

pip install gunicorn

But you will need to put the dependency in the requirements.txt file so that it can be preset in the Docker image:

requirements.txt

gunicorn==20.1.0

Finally, we need to declare at the end of the file the volumes used:

docker-compose.yaml

volumes:
 uploads_volume:
 postgres_data:

And we have just written a docker-compose.yaml file. As we are going to use environment variables in the project, let’s update some variables in the settings.py file.

Configuring environment variables in Django

It is a bad habit to have sensitive information about your application available in the code. This is the case for the SECRET_KEY setting and the database settings in the settings.py file of the project. It is quite bad because we have pushed the code to GitHub. Let’s correct this.

An environment variable is a variable whose value is set outside the running code of the program. With Python, you can read files from a .env file. We will use the os library to write the configurations. So, first, create a .env file at the root of the Django project and add the following content:

.env

SECRET_KEY=foo
DATABASE_NAME=coredb
DATABASE_USER=core
DATABASE_PASSWORD=wCh29&HE&T83
DATABASE_HOST=postagram_db
DATABASE_PORT=5432
POSTGRES_USER=core
POSTGRES_PASSWORD=wCh29&HE&T83
POSTGRES_DB=coredb
ENV=DEV
DJANGO_ALLOWED_HOSTS=127.0.0.1,localhost

Important note

SECRET_KEY is an important variable for your Django project, so you need to ensure that you have a long and complicated chain of characters as the value. You can visit https://djecrety.ir/ to generate a new chain of characters.

The next step is to install a package to help you manage environment variables. The package is called python-dotenv, and it helps Python developers read environment variables from .env files and set them as environment variables. If you are going to run the project again on your machine, then add the package to your actual Python environment with the following command:

pip install python-dotenv

And finally, add the package to the requirements.txt file so that it can be installed in the Docker image. Here’s a look at the requirements.txt file:

Django==4.0.1
psycopg2-binary==2.9.3
djangorestframework==3.13.1
django-filter==21.1
pillow==9.0.0
djangorestframework-simplejwt==5.0.0
drf-nested-routers==0.93.4
pytest-django==4.5.2
django-cors-headers==3.11.0
python-dotenv==0.20.0
gunicorn==20.1.0

Once the installation of the python-dotenv package is done, we need to write some code in the CoreRoot/settings.py file. In this file, we will import the python-dotenv package and modify the syntax of some settings so that it can support environment variables’ reading:

CoreRoot/settings.py

from dotenv import load_dotenv
load_dotenv()

Let’s rewrite the values of variables such as SECRET_KEY, DEBUG, ALLOWED_HOSTS, and ENV:

CoreRoot/settings.py

ENV = os.environ.get("ENV")
# SECURITY WARNING: keep the secret key used in production secret!
SECRET_KEY = os.environ.get(
   "SECRET_KEY", default=
     "qkl+xdr8aimpf-&x(mi7)dwt^-q77aji#j*d#02-5usa32r9!y"
)
# SECURITY WARNING: don't run with debug turned on in production!
DEBUG = False if ENV == "PROD" else True
ALLOWED_HOSTS = os.environ.get("DJANGO_ALLOWED_HOSTS", default="*").split(",")

The os package provides an object to retrieve environment variables from the user machine. After python-dotenv has forced the loading of the environment variables, we use os.environ to read the values from the .env file. Let’s finally add the configuration for the DATABASES setting:

CoreRoot/settings.py

DATABASES = {
   "default": {
       "ENGINE": "django.db.backends.postgresql_psycopg2",
       "NAME": os.getenv("DATABASE_NAME", "coredb"),
       "USER": os.getenv("DATABASE_USER", "core"),
       "PASSWORD": os.getenv("DATABASE_PASSWORD",
                             "wCh29&HE&T83"),
       "HOST": os.environ.get("DATABASE_HOST",
                              "localhost"),
       "PORT": os.getenv("DATABASE_PORT", "5432"),
   }
}

Great! We are done configuring the environment variables in the settings.py file. We can now move on to write the configurations for NGINX.

Writing NGINX configuration

NGINX requires some configuration from our side. If there is a request on the HTTP port of the machine (by default 80), it should redirect the requests to port 8000 of the running Django application. Put simply, we will write a reverse proxy. A proxy is an intermediary process that takes an HTTP request from a client, passes the request to one or many other servers, waits for a response from those servers, and sends back a response to the client.

By using this process, we can forward a request on the HTTP port 80 to port 8000 of the Django server.

At the root of the project, create a new file called nginx.conf. Then, let’s define the upstream server where HTTP requests will be redirected to:

nginx.conf

upstream webapp {
   server postagram_api:8000;
}

The preceding code follows the simple syntax shown next:

upstream upstream_name {
   server host:PORT;
}

Important note

Docker allows you to refer to the container’s host with the defined container name. In the NGINX file, we are using postagram_api instead of the IP address of the container, which can change, and for the database, we are using postagram_db.

The next step is to declare the configuration for the HTTP server:

nginx.conf

server {
   listen 80;
   server_name localhost;
   location / {
       proxy_pass http://webapp;
       proxy_set_header X-Forwarded-For
         $proxy_add_x_forwarded_for;
       proxy_set_header Host $host;
       proxy_redirect off;
   }
   location /media/ {
    alias /app/uploads/;
   }
}

In the server configuration, we first set the port of the server. In the preceding code, we are using port 80. Next, we are defining locations. A location in NGINX is a block that tells NGINX how to process the request from a certain URL:

• A request on the / URL is redirected to the web app upstream
• A request on the /media/ URL is redirected to the uploads folder to serve files

With the NGINX configuration ready, we can now launch the containers.

Launching the Docker containers

Let’s launch the Docker containers. As we are now using Docker Compose to orchestrate containers, let’s use the following command to build and start the containers:

docker compose up -d –build

This command will spin up all the containers defined in the docker-compose.yaml file. Let’s describe the command options:

• up: This builds, recreates, and starts containers
• -d: This is used to detach, meaning that we are running the containers in the background
• —build: This flag tells Docker Compose to build the images before starting the containers

After the build is done, open your browser at http://localhost, and you should see the following:

Figure 13.3 – Dockerized Django application

We have successfully containerized the Django application using Docker. It is also possible to execute commands inside containers, and right now, we can start by running a test suite inside the postagram_api container:

docker compose exec -T api pytest

The syntax to execute a command in a Docker container is to first call the exec command followed by the –T parameter to disable pseudo-tty allocation. This means that the command being run inside the container will not be attached to a terminal. Finally, you can add the container service name, followed by the command you want to execute in the container.

We are one step closer to the deployment of AWS using Docker, but we need to automate it. In the next chapter, we will configure the project with GitHub Actions to automate deployment on the AWS server.

Summary

In this chapter, we have learned how to dockerize a Django application. We started by looking into Docker and its use in the development of modern applications. We also learned how to build a Docker image and run a container using this image—this introduced us to some limitations of Dockerization using Dockerfiles. This led us to learn more about Docker Compose and how it can help us manage multiple containers with just one configuration file. This in turn directed us to configure a database and an NGINX web server with Docker to launch the Postagram API.

In the next chapter, we will configure the project for automatic deployment on AWS but also carry out regression checks using the tests we have written.

Questions

1. What is Docker?
2. What is Docker Compose?
3. What is the difference between Docker and Docker Compose?
4. What is the difference between containerization and virtualization?
5. What is an environment variable?