Docker is a great technology, but we have to be careful not to make our images too big and to remove image files when possible. The docker-clean
script at https://gist.github.com/michaelneale/1366325a7737c4cb80b0 (retrieved January 2016) helps reclaim space.
I found it useful to have an install script, which is just a regular shell script, and I added it to the Dockerfile
as follows:
ADD install.sh /root/install.sh
Python creates __pycache__
directories for the purpose of optimization (we can disable this option in various ways). These are not strictly needed and can be easily removed as follows:
find /opt/conda -name \__pycache__ -depth -exec rm -rf {} ;
Anaconda puts a lot of files in its pkgs
directory, which we can remove as follows:
rm -r /opt/conda/pkgs/*
Some people recommend removing test code; however, in certain rare cases, the non-test code depends on the test code. Also, it is useful to have the test code just in case.
There are some gotchas to be aware of when working with Docker. For instance, we have to set the PATH
environment variable as follows:
export PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:${PATH}
For Python scripts, we also need to set the language settings as follows:
ENV LANG=C.UTF-8
It is generally recommended to specify the package version when you install software with pip
or conda
, such as like this:
$ conda install scipy=0.15.0 $ pip install scipy==0.15.0
When installing with conda
, it is also recommended that you install multiple packages at once in order to avoid installing multiple versions of common dependencies:
$ conda install scipy=0.15.0 curl=7.26.0
My Docker setup for the main Docker repository consists of a Dockerfile
script and an install script (install.sh
). The contents of the Dockerfile
are as follows:
FROM continuumio/miniconda3 ADD install.sh /root/install.sh RUN sh -x /root/install.sh ENV LANG=C.UTF-8
I execute the install script with the –x
switch, which gives more verbose output.
The contents of install.sh
are as follows:
export PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:${PATH} apt-get install -y libgfortran3 conda config --set always_yes True conda install beautiful-soup bokeh=0.9.1 execnet=1.3.0 fastcache=1.0.2 joblib=0.8.4 jsonschema ipython=3.2.1 lxml mpmath=0.19 networkx=1.9.1 nltk=3.0.2 numba=0.22.1 numexpr=2.3.1 pandas=0.16.2 pyzmq scipy=0.16.0 seaborn=0.6.0 sqlalchemy=0.9.9 statsmodels=0.6.1 terminado=0.5 tornado conda install matplotlib=1.5.0 numpy=1.10.1 scikit-learn=0.17 pip install dautil==0.0.1a29 pip install hiveplot==0.1.7.4 pip install landslide==1.1.3 pip install LiveStats==1.0 pip install mpld3==0.2 pip install pep8==1.6.2 pip install requests-cache==0.4.10 pip install tabulate==0.7.5 find /opt/conda -name \__pycache__ -depth -exec rm -rf {} ; rm -r /opt/conda/pkgs/* mkdir -p /.cache/dautil/log mkdir -p /.local/share/dautil