In this section, we will take you through a journey in which we will show you how TDD is done using stubs, and how Docker can come handy in developing software in the deployment equivalent system. For this purpose, we take a web application use case that has a feature to track the visit count of each of its users. For this example, we use Python as the implementation language and redis
as the key-value pair database to store the users hit count. Besides, to showcase the testing capability of Docker, we limit our implementation to just two functions: hit
and getHit
.
As per the TDD practice, we start by adding unit test cases for the hit
and getHit
functionalities, as depicted in the following code snippet. Here, the test file is named test_hitcount.py
:
import unittest import hitcount class HitCountTest (unittest.TestCase): def testOneHit(self): # increase the hit count for user user1 hitcount.hit("user1") # ensure that the hit count for user1 is just 1 self.assertEqual(b'1', hitcount.getHit("user1")) if __name__ == '__main__': unittest.main()
This example is also available at https://github.com/thedocker/testing/tree/master/src.
Here, in the first line, we are importing the unittest
Python module that provides the necessary framework and functionality to run the unit test and generate a detailed report on the test execution. In the second line, we are importing the hitcount
Python module, where we are going to implement the hit count functionality. Then, we will continue to add the test code that would test the hitcount
module's functionality.
Now, run the test suite using the unit test framework of Python, as follows:
$ python3 -m unittest
The following is the output generated by the unit test framework:
E ====================================================================== ERROR: test_hitcount (unittest.loader.ModuleImportFailure) ---------------------------------------------------------------------- Traceback (most recent call last): ...OUTPUT TRUNCATED ... ImportError: No module named 'hitcount' ---------------------------------------------------------------------- Ran 1 test in 0.001s FAILED (errors=1)
As expected, the test failed with the error message ImportError: No module named 'hitcount'
because we had not even created the file and hence, it could not import the hitcount
module.
Now, create a file with the name hitcount.py
in the same directory as test_hitcount.py
:
$ touch hitcount.py
Continue to run the unit test suite:
$ python3 -m unittest
The following is the output generated by the unit test framework:
E ====================================================================== ERROR: testOneHit (test_hitcount.HitCountTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/user/test_hitcount.py", line 10, in testOneHit hitcount.hit("peter") AttributeError: 'module' object has no attribute 'hit' ---------------------------------------------------------------------- Ran 1 test in 0.001s FAILED (errors=1)
Again the test suite failed like the earlier but with a different error message AttributeError: 'module' object has no attribute 'hit'
. We are getting this error because we have not implemented the hit
function yet.
Let's proceed to implement the hit
and getHit
functions in hitcount.py
, as shown here:
import redis # connect to redis server r = redis.StrictRedis(host='0.0.0.0', port=6379, db=0) # increase the hit count for the usr def hit(usr): r.incr(usr) # get the hit count for the usr def getHit(usr): return (r.get(usr))
This example is also available on GitHub at https://github.com/thedocker/testing/tree/master/src.
Note: To continue with this example, you must have the python3
compatible version of package installer (pip3
).
The following command is used to install pip3
:
$ wget -qO- https://bootstrap.pypa.io/get-pip.py | sudo python3 -
In the first line of this program, we are importing the redis
driver, which is the connectivity driver of the redis
database. In the following line, we are connecting to the redis
database, and then we will continue to implement the hit
and getHit
function.
The redis
driver is an optional Python module, so let's proceed to install the redis
driver using the pip
installer, which is illustrated as follows:
$ sudo pip3 install redis
Our unittest
will still fail even after installing the redis
driver because we are not running a redis
database server yet. So, we can either run a redis
database server to successfully complete our unit testing or take the traditional TDD approach of mocking the redis
driver. Mocking is a testing approach wherein complex behavior is substituted by predefined or simulated behavior. In our example, to mock the redis driver, we are going to leverage a third-party Python package called
mockredis
. This mock package is available at https://github.com/locationlabs/mockredis and the pip
installer name is mockredispy
. Let's install this mock using the pip
installer:
$ sudo pip3 install mockredispy
Having installed mockredispy
, the redis
mock, let's refactor our test code test_hitcount.py
(which we had written earlier) to use the simulated redis
functionality provided by the mockredis
module. This is accomplished by the patch method provided by the unittest.mock
mocking framework, as shown in the following code:
import unittest from unittest.mock import patch # Mock for redis import mockredis import hitcount class HitCountTest(unittest.TestCase): @patch('hitcount.r',mockredis.mock_strict_redis_client(host='0.0.0.0', port=6379, db=0)) def testOneHit(self): # increase the hit count for user user1 hitcount.hit("user1") # ensure that the hit count for user1 is just 1 self.assertEqual(b'1', hitcount.getHit("user1")) if __name__ == '__main__': unittest.main()
This example is also available on GitHub at https://github.com/thedocker/testing/tree/master/src.
Now, run the test suite again:
$ python3 -m unittest . ---------------------------------------------------------------------- Ran 1 test in 0.000s OK
Finally, as we can see in the preceding output, we successfully implemented our visitors count functionality through the test, code, and refactor cycle.
In the previous section, we walked you through the complete cycle of TDD, in which we installed additional Python packages to complete our development. However, in the real world, one might work on multiple projects that might have conflicting libraries and hence, there is a need for the isolation of runtime environments. Before the advent of Docker technology, the Python community used to leverage the virtualenv
tool to isolate the Python runtime environment. Docker takes this isolation a step further by packaging the OS, the Python tool chain, and the runtime environment. This type of isolation gives a lot of flexibility to the development community to use appropriate software versions and libraries as per the project needs.
Here is the step-by-step procedure to package the test and visitor count implementation of the previous section to a Docker container and perform the test inside the container:
Dockerfile
to build an image with the python3
runtime, the redis
and mockredispy
packages, both the test_hitcount.py
test file and the visitors count implementation hitcount.py
, and finally, launch the unit test:############################################# # Dockerfile to build the unittest container ############################################# # Base image is python FROM python:latest # Author: Dr. Peter MAINTAINER Dr. Peter <[email protected]> # Install redis driver for python and the redis mock RUN pip install redis && pip install mockredispy # Copy the test and source to the Docker image ADD src/ /src/ # Change the working directory to /src/ WORKDIR /src/ # Make unittest as the default execution ENTRYPOINT python3 -m unittest
This example is also available on GitHub at https://github.com/thedocker/testing/tree/master/src.
src
on the directory, where we crafted our Dockerfile
. Move the test_hitcount.py
and hitcount.py
files to the newly created src
directory.hit_unittest
Docker image using the docker build
subcommand:$ sudo docker build -t hit_unittest . Sending build context to Docker daemon 11.78 kB Sending build context to Docker daemon Step 0 : FROM python:latest ---> 32b9d937b993 Step 1 : MAINTAINER Dr. Peter <[email protected]> ---> Using cache ---> bf40ee5f5563 Step 2 : RUN pip install redis && pip install mockredispy ---> Using cache ---> a55f3bdb62b3 Step 3 : ADD src/ /src/ ---> 526e13dbf4c3 Removing intermediate container a6d89cbce053 Step 4 : WORKDIR /src/ ---> Running in 5c180e180a93 ---> 53d3f4e68f6b Removing intermediate container 5c180e180a93 Step 5 : ENTRYPOINT python3 -m unittest ---> Running in 74d81f4fe817 ---> 063bfe92eae0 Removing intermediate container 74d81f4fe817 Successfully built 063bfe92eae0
docker run
subcommand, as illustrated here:$ sudo docker run --rm -it hit_unittest . ---------------------------------------------------------------------- Ran 1 test in 0.001s OK
Apparently, the unit test ran successfully with no errors because we already packaged the tested code.
In this approach, for every change, the Docker image is built and then, the container is launched to complete the test.
In the previous section, we built a Docker image to perform the testing. Particularly, in the TDD practice, the unit test cases and the code go through multiple changes. Consequently, the Docker image needs to be built over and over again, which is a daunting job. In this section, we will see an alternative approach in which the Docker container is built with a runtime environment, the development directory is mounted as a volume, and the test is performed inside the container.
During this TDD cycle, if an additional library or update to the existing library is required, then the container will be updated with the required libraries and the updated container will be committed as a new image. This approach gives the isolation and flexibility that any developer would dream of because the runtime and its dependency live within the container, and any misconfigured runtime environment can be discarded and a new runtime environment can be built from a previously working image. This also helps to preserve the sanity of the Docker host from the installation and uninstallation of libraries.
The following example is a step-by-step instruction on how to use the Docker container as a nonpolluting yet very powerful runtime environment:
docker run
subcommand:$ sudo docker run -it -v /home/peter/src/hitcount:/src python:latest /bin/bash
Here, in this example, the /home/peter/src/hitcount
Docker host directory is earmarked as the placeholder for the source code and test files. This directory is mounted in the container as /src
.
test_hitcount.py
test file and the visitors count implementation hitcount.py
to /home/peter/src/hitcount directory
./src
, and run the unit test:root@a8219ac7ed8e:~# cd /src root@a8219ac7ed8e:/src# python3 -m unittest E ====================================================================== ERROR: test_hitcount (unittest.loader.ModuleImportFailure) . . . TRUNCATED OUTPUT . . . File "/src/test_hitcount.py", line 4, in <module> import mockredis ImportError: No module named 'mockredis' ----------------------------------------------------------------- Ran 1 test in 0.001s FAILED (errors=1)
Evidently, the test failed because it could not find the mockredis
Python library.
mockredispy pip
package because the previous step failed, as it could not find the mockredis
library in the runtime environment:root@a8219ac7ed8e:/src# pip install mockredispy
root@a8219ac7ed8e:/src# python3 -m unittest E ================================================================= ERROR: test_hitcount (unittest.loader.ModuleImportFailure) . . . TRUNCATED OUTPUT . . . File "/src/hitcount.py", line 1, in <module> import redis ImportError: No module named 'redis' Ran 1 test in 0.001s FAILED (errors=1)
Again, the test failed because the redis
driver is not yet installed.
redis
driver using the pip installer, as shown here:root@a8219ac7ed8e:/src# pip install redis
redis
driver, let's once again run the unit test:root@a8219ac7ed8e:/src# python3 -m unittest . ----------------------------------------------------------------- Ran 1 test in 0.000s OK
Apparently, this time the unit test passed with no warnings or error messages.
docker commit
subcommand:$ sudo docker c ommit a8219ac7ed8e python_rediswithmock fcf27247ff5bb240a935ec4ba1bddbd8c90cd79cba66e52b21e1b48f984c7db2
From now on, we can use the python_rediswithmock
image to launch new containers for our TDD.
In this section, we vividly illustrated the approach on how to use the Docker container as a testing environment, and also at the same time, preserve the sanity and sanctity of the Docker host by isolating and limiting the runtime dependency within the container.