Testing your code inside Docker

In this section, we will take you through a journey in which we will show you how TDD is done using stubs, and how Docker can come handy in developing software in the deployment equivalent system. For this purpose, we take a web application use case that has a feature to track the visit count of each of its users. For this example, we use Python as the implementation language and redis as the key-value pair database to store the users hit count. Besides, to showcase the testing capability of Docker, we limit our implementation to just two functions: hit and getHit.

Note

NOTE: All the examples in this chapter use python3 as the runtime environment. The ubuntu 14.04 installation comes with python3 by default. If you don't have python3 installed on your system, refer to the respective manual to install python3.

As per the TDD practice, we start by adding unit test cases for the hit and getHit functionalities, as depicted in the following code snippet. Here, the test file is named test_hitcount.py:

import unittest
import hitcount

class HitCountTest (unittest.TestCase):
     def testOneHit(self):
         # increase the hit count for user user1
         hitcount.hit("user1")
         # ensure that the hit count for user1 is just 1
         self.assertEqual(b'1', hitcount.getHit("user1"))

if __name__ == '__main__':
    unittest.main()

Note

This example is also available at https://github.com/thedocker/testing/tree/master/src.

Here, in the first line, we are importing the unittest Python module that provides the necessary framework and functionality to run the unit test and generate a detailed report on the test execution. In the second line, we are importing the hitcount Python module, where we are going to implement the hit count functionality. Then, we will continue to add the test code that would test the hitcount module's functionality.

Now, run the test suite using the unit test framework of Python, as follows:

$ python3 -m unittest 

The following is the output generated by the unit test framework:

E
======================================================================
ERROR: test_hitcount (unittest.loader.ModuleImportFailure)
----------------------------------------------------------------------
Traceback (most recent call last):
...OUTPUT TRUNCATED ...
ImportError: No module named 'hitcount'


----------------------------------------------------------------------
Ran 1 test in 0.001s

FAILED (errors=1)

As expected, the test failed with the error message ImportError: No module named 'hitcount' because we had not even created the file and hence, it could not import the hitcount module.

Now, create a file with the name hitcount.py in the same directory as test_hitcount.py:

$ touch hitcount.py

Continue to run the unit test suite:

$ python3 -m unittest

The following is the output generated by the unit test framework:

E
======================================================================
ERROR: testOneHit (test_hitcount.HitCountTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/user/test_hitcount.py", line 10, in testOneHit
    hitcount.hit("peter")
AttributeError: 'module' object has no attribute 'hit'

----------------------------------------------------------------------
Ran 1 test in 0.001s

FAILED (errors=1)

Again the test suite failed like the earlier but with a different error message AttributeError: 'module' object has no attribute 'hit'. We are getting this error because we have not implemented the hit function yet.

Let's proceed to implement the hit and getHit functions in hitcount.py, as shown here:

import redis
# connect to redis server
r = redis.StrictRedis(host='0.0.0.0', port=6379, db=0)

# increase the hit count for the usr
def hit(usr):
    r.incr(usr)

# get the hit count for the usr
   def getHit(usr):
    return (r.get(usr))

Note

This example is also available on GitHub at https://github.com/thedocker/testing/tree/master/src.

Note: To continue with this example, you must have the python3 compatible version of package installer (pip3).

The following command is used to install pip3:

$ wget -qO- https://bootstrap.pypa.io/get-pip.py | sudo python3 -

In the first line of this program, we are importing the redis driver, which is the connectivity driver of the redis database. In the following line, we are connecting to the redis database, and then we will continue to implement the hit and getHit function.

The redis driver is an optional Python module, so let's proceed to install the redis driver using the pip installer, which is illustrated as follows:

$ sudo pip3 install redis

Our unittest will still fail even after installing the redis driver because we are not running a redis database server yet. So, we can either run a redis database server to successfully complete our unit testing or take the traditional TDD approach of mocking the redis driver. Mocking is a testing approach wherein complex behavior is substituted by predefined or simulated behavior. In our example, to mock the redis driver, we are going to leverage a third-party Python package called mockredis. This mock package is available at https://github.com/locationlabs/mockredis and the pip installer name is mockredispy. Let's install this mock using the pip installer:

$ sudo pip3 install mockredispy

Having installed mockredispy, the redis mock, let's refactor our test code test_hitcount.py (which we had written earlier) to use the simulated redis functionality provided by the mockredis module. This is accomplished by the patch method provided by the unittest.mock mocking framework, as shown in the following code:

import unittest
from unittest.mock import patch

# Mock for redis
import mockredis
import hitcount

class HitCountTest(unittest.TestCase):

    @patch('hitcount.r',mockredis.mock_strict_redis_client(host='0.0.0.0', port=6379, db=0))
    def testOneHit(self):
        # increase the hit count for user user1
        hitcount.hit("user1")
        # ensure that the hit count for user1 is just 1
        self.assertEqual(b'1', hitcount.getHit("user1"))

if __name__ == '__main__':
    unittest.main()

Note

This example is also available on GitHub at https://github.com/thedocker/testing/tree/master/src.

Now, run the test suite again:

$ python3 -m unittest
.
----------------------------------------------------------------------
Ran 1 test in 0.000s

OK

Finally, as we can see in the preceding output, we successfully implemented our visitors count functionality through the test, code, and refactor cycle.

Running the test inside a container

In the previous section, we walked you through the complete cycle of TDD, in which we installed additional Python packages to complete our development. However, in the real world, one might work on multiple projects that might have conflicting libraries and hence, there is a need for the isolation of runtime environments. Before the advent of Docker technology, the Python community used to leverage the virtualenv tool to isolate the Python runtime environment. Docker takes this isolation a step further by packaging the OS, the Python tool chain, and the runtime environment. This type of isolation gives a lot of flexibility to the development community to use appropriate software versions and libraries as per the project needs.

Here is the step-by-step procedure to package the test and visitor count implementation of the previous section to a Docker container and perform the test inside the container:

  1. Craft a Dockerfile to build an image with the python3 runtime, the redis and mockredispy packages, both the test_hitcount.py test file and the visitors count implementation hitcount.py, and finally, launch the unit test:
    #############################################
    # Dockerfile to build the unittest container
    #############################################
    
    # Base image is python
    FROM python:latest
    
    # Author: Dr. Peter
    MAINTAINER Dr. Peter <[email protected]>
    
    # Install redis driver for python and the redis mock
    RUN pip install redis && pip install mockredispy
    
    # Copy the test and source to the Docker image
    ADD src/ /src/
    
    # Change the working directory to /src/
    WORKDIR /src/
    
    # Make unittest as the default execution
    ENTRYPOINT python3 -m unittest

    Note

    This example is also available on GitHub at https://github.com/thedocker/testing/tree/master/src.

  2. Now create a directory called src on the directory, where we crafted our Dockerfile. Move the test_hitcount.py and hitcount.py files to the newly created src directory.
  3. Build the hit_unittest Docker image using the docker build subcommand:
    $ sudo docker build -t hit_unittest .
    Sending build context to Docker daemon 11.78 kB
    Sending build context to Docker daemon
    Step 0 : FROM python:latest
     ---> 32b9d937b993
    Step 1 : MAINTAINER Dr. Peter <[email protected]>
     ---> Using cache
     ---> bf40ee5f5563
    Step 2 : RUN pip install redis && pip install mockredispy
     ---> Using cache
     ---> a55f3bdb62b3
    Step 3 : ADD src/ /src/
     ---> 526e13dbf4c3
    Removing intermediate container a6d89cbce053
    Step 4 : WORKDIR /src/
     ---> Running in 5c180e180a93
     ---> 53d3f4e68f6b
    Removing intermediate container 5c180e180a93
    Step 5 : ENTRYPOINT python3 -m unittest
     ---> Running in 74d81f4fe817
     ---> 063bfe92eae0
    Removing intermediate container 74d81f4fe817
    Successfully built 063bfe92eae0
    
  4. Now that we have successfully built the image, let's launch our container with the unit testing bundle using the docker run subcommand, as illustrated here:
    $ sudo docker run --rm -it hit_unittest .
    ----------------------------------------------------------------------
    Ran 1 test in 0.001s
    
    OK
    

Apparently, the unit test ran successfully with no errors because we already packaged the tested code.

In this approach, for every change, the Docker image is built and then, the container is launched to complete the test.

Using a Docker container as a runtime environment

In the previous section, we built a Docker image to perform the testing. Particularly, in the TDD practice, the unit test cases and the code go through multiple changes. Consequently, the Docker image needs to be built over and over again, which is a daunting job. In this section, we will see an alternative approach in which the Docker container is built with a runtime environment, the development directory is mounted as a volume, and the test is performed inside the container.

During this TDD cycle, if an additional library or update to the existing library is required, then the container will be updated with the required libraries and the updated container will be committed as a new image. This approach gives the isolation and flexibility that any developer would dream of because the runtime and its dependency live within the container, and any misconfigured runtime environment can be discarded and a new runtime environment can be built from a previously working image. This also helps to preserve the sanity of the Docker host from the installation and uninstallation of libraries.

The following example is a step-by-step instruction on how to use the Docker container as a nonpolluting yet very powerful runtime environment:

  1. We begin with launching the Python runtime interactive container, using the docker run subcommand:
    $ sudo docker run -it 
              -v /home/peter/src/hitcount:/src 
              python:latest /bin/bash
    

    Here, in this example, the /home/peter/src/hitcount Docker host directory is earmarked as the placeholder for the source code and test files. This directory is mounted in the container as /src.

  2. Now, on another terminal of the Docker host, copy both the test_hitcount.py test file and the visitors count implementation hitcount.py to /home/peter/src/hitcount directory.
  3. Switch to the Python runtime interactive container terminal, change the current working directory to /src, and run the unit test:
    root@a8219ac7ed8e:~# cd /src
    root@a8219ac7ed8e:/src# python3 -m unittest
    E
    ======================================================================
    ERROR: test_hitcount (unittest.loader.ModuleImportFailure)
    . . . TRUNCATED OUTPUT . . . 
      File "/src/test_hitcount.py", line 4, in <module>
        import mockredis
    ImportError: No module named 'mockredis'
    
    -----------------------------------------------------------------
    Ran 1 test in 0.001s
    
    FAILED (errors=1)
    

    Evidently, the test failed because it could not find the mockredis Python library.

  4. Proceed to install the mockredispy pip package because the previous step failed, as it could not find the mockredis library in the runtime environment:
    root@a8219ac7ed8e:/src# pip install mockredispy
    
  5. Rerun the Python unit test:
    root@a8219ac7ed8e:/src# python3 -m unittest
    E
    =================================================================
    ERROR: test_hitcount (unittest.loader.ModuleImportFailure)
    . . . TRUNCATED OUTPUT . . . 
      File "/src/hitcount.py", line 1, in <module>
        import redis
    ImportError: No module named 'redis'
    
    
    Ran 1 test in 0.001s
    
    FAILED (errors=1)
    

    Again, the test failed because the redis driver is not yet installed.

  6. Continue to install the redis driver using the pip installer, as shown here:
    root@a8219ac7ed8e:/src# pip install redis
    
  7. Having successfully installed the redis driver, let's once again run the unit test:
    root@a8219ac7ed8e:/src# python3 -m unittest
    .
    -----------------------------------------------------------------
    Ran 1 test in 0.000s
    
    OK
    

    Apparently, this time the unit test passed with no warnings or error messages.

  8. Now we have a runtime environment that is good enough to run our test cases. It is better to commit these changes to a Docker image for reuse, using the docker commit subcommand:
    $ sudo docker c
    ommit a8219ac7ed8e python_rediswithmock
    fcf27247ff5bb240a935ec4ba1bddbd8c90cd79cba66e52b21e1b48f984c7db2
    

    From now on, we can use the python_rediswithmock image to launch new containers for our TDD.

In this section, we vividly illustrated the approach on how to use the Docker container as a testing environment, and also at the same time, preserve the sanity and sanctity of the Docker host by isolating and limiting the runtime dependency within the container.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset