C/C++ extensions

The previous chapter already covered this somewhat, as it's a requirement to compile the C/C++ files. But that chapter didn't explain what and how the setup.py was doing in this case.

For convenience, we will repeat the setup.py file:

import setuptools

spam = setuptools.Extension('spam', sources=['spam.c'])

setuptools.setup(
    name='Spam',
    version='1.0',
    ext_modules=[spam],
)

Before you start with these extensions, you should learn the following commands:

  • build: This is actually not a C/C++ specific build function (try build_clib for that) but a combined build function to build everything within setup.py.
  • clean: This cleans the results from the build command. This is generally not needed but sometimes the detection of files that need to be recompiled to work is incorrect. So if you encounter strange or unexpected issues, try cleaning the project first.

Regular extensions

The setuptools.Extension class tells setuptools that a module named spam uses the source file spam.c. This is just the simplest version of an extension, a name, and a list of sources, but in many cases you are going to need more than the simple case.

One example is the pillow library which detects the libraries available on the system and adds extensions based on that. But because these extensions include libraries, some extra compilation flags are required. The basic PIL module itself doesn't appear too involved but the libs are actually filled with all auto-detected libraries with the matching macro definitions:

exts = [(Extension("PIL._imaging", files, libraries=libs,
                   define_macros=defs))]

The freetype extension has something similar:

if feature.freetype:
    exts.append(Extension(
        "PIL._imagingft", ["_imagingft.c"], libraries=["freetype"]))

Cython extensions

The setuptools library is actually a bit smarter than the regular distutils library when it comes to extensions. It actually adds a little trick to the Extension class. Remember the brief introduction to Cython in Chapter 12, Performance – Tracking and Reducing Your Memory and CPU Usage about performance? The setuptools library makes it a bit more convenient to compile those. The Cython manual recommends you to use something similar to the following code:

from distutils.core import setup
from Cython.Build import cythonize

setup(
    ext_modules = cythonize("eggs.pyx")
)

Here eggs.pyx contains:

def make_eggs(int n):
    print('Making %d eggs: %s' % (n, n * 'eggs '))

The problem with this approach is that setup.py will break unless you have Cython installed:

# python setup.py build
Traceback (most recent call last):
  File "setup.py", line 2, in <module>
    import Cython
ImportError: No module named 'Cython'

To prevent that issue, we are just going to let setuptools handle this:

import setuptools

eggs = setuptools.Extension('eggs', sources=['eggs.pyx'])

setuptools.setup(
    name='Eggs',
    version='1.0',
    ext_modules=[eggs],
    setup_requires=['Cython'],
)

Now Cython will be automatically installed if needed and the code will work just fine:

# python setup.py build
running build
running build_ext
cythoning eggs.pyx to eggs.c
building 'eggs' extension
...
# python setup.py develop
running develop
running egg_info
creating Eggs.egg-info
writing dependency_links to Eggs.egg-info/dependency_links.txt
writing top-level names to Eggs.egg-info/top_level.txt
writing Eggs.egg-info/PKG-INFO
writing manifest file 'Eggs.egg-info/SOURCES.txt'
reading manifest file 'Eggs.egg-info/SOURCES.txt'
writing manifest file 'Eggs.egg-info/SOURCES.txt'
running build_ext
skipping 'eggs.c' Cython extension (up-to-date)
copying build/... ->
Creating Eggs.egg-link (link to .)
Adding Eggs 1.0 to easy-install.pth file

Installed Eggs
Processing dependencies for Eggs==1.0
Finished processing dependencies for Eggs==1.0
# python -c 'import eggs; eggs.make_eggs(3)'
Making 3 eggs: eggs eggs eggs

For development purposes however, Cython also offers a simpler method which doesn't require manual building. First, to make sure we are actually using this method, let's install Cython and uninstall and clean eggs completely:

# pip uninstall eggs -y
Uninstalling Eggs-1.0:
  Successfully uninstalled Eggs-1.0
# pip uninstall eggs -y
Cannot uninstall requirement eggs, not installed
# python setup.py clean
# pip install cython

Now let's try and run our eggs.pyx module:

>>> import pyximport
>>> pyximport.install()
(None, <pyximport.pyximport.PyxImporter object at 0x...>)
>>> import eggs
>>> eggs.make_eggs(3)
Making 3 eggs: eggs eggs eggs

That's how easy it is to run the pyx files without explicit compiling.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset