The templates that we discussed earlier are just a basis that you can use to document your software. With time, you will eventually develop your own templates and style for making documentation. But always keep in mind the light but sufficient approach for project documentation: each document added should have a clearly defined target readership and should fill a real need. Documents that don't add a real value should not be written.
Each project is unique and has different documentation needs. For example, small terminal tools with simple usage can definitely live with only a single README file as its document landscape. Having such a minimal single-document approach is completely fine if the target readers are precisely defined and consistently grouped (system administrators, for instance).
Also, do not apply the provided templates too rigorously. Some additional metadata provided as an example is really useful in either big projects or in strictly formalized teams. Tags, for instance, are intended to improve textual search in big documentations but will not provide any value in a documentation landscape consisting only of a few documents.
Also, including the document author is not always a good idea. Such an approach may be especially questionable in open source projects. In such projects, you will want the community to also contribute to documentation. In most cases, such documents are continuously updated whenever there is such a need by whoever makes the contribution. People tend to treat the document author also as the document owner. This may discourage people to update the documentation if every document has its author always specified. Usually, the version control software provides clearer and more transparent information about real document authors than explicitly provided metadata annotations. The situations where explicit authors are really recommended are various design documents, especially in projects where the design process is strictly formalized. The best example is the series of PEP documents with the Python language enhancement proposals.
The document portfolio built in the previous section provides a structure at document level but does not provide a way to group and organize it to build the documentation the readers will have. This is what Andreas Rüping calls a document landscape, referring to the mental map the readers use when they browse documentation. He came up with the conclusion that the best way to organize documents is to build a logical tree.
In other words, the different kinds of documents composing the portfolio need to find a place to live within a tree of directories. This place must be obvious to the writers when they create the document and to the readers when they are looking for it.
A great help when browsing documentation is the index pages at each level that can drive writers and readers.
Building a document landscape is done in two steps:
- Building a tree for the producers (the writers)
- Building a tree for the consumers (the readers) on top of the producers' tree
This distinction between producers and consumers is important since they access the documents in different places and different formats.
From a producer's point of view, each document is processed exactly like a Python module. It should be stored in the version control system and works like code. Writers do not care about the final appearance of their prose and where it is available, they just want to make sure that they are writing a document, so it is the single source of truth on the topic covered. reStructuredText files stored in a folder tree are available in the version control system together with the software code and are a convenient solution to building the documentation landscape for producers.
By convention, the docs folder is used as a root of documentation tree:
$ cd my-project $ find docs docs docs/source docs/source/design docs/source/operations docs/source/usage docs/source/usage/cookbook docs/source/usage/modules docs/source/usage/tutorial
Notice that the tree is located in a source folder because the docs folder will be used as a root folder to set up a special tool in the next section.
From there, an index.txt file can be added at each level (besides the root), explaining what kind of documents the folder contains or summarizing what each subfolder contains. These index files can define a listing of the documents they contain. For instance, the operations folder can contain a list of operations documents available:
========== Operations ========== This section contains operations documents: − How to install and run the project − How to install and manage a database for the project
It is important to know that people tend to forget to update such lists of documents and tables of content. So it is better to have them updated automatically. In the next subsection, we will discuss one tool that, among many other features, can also handle this use case.
From a consumer's point of view, it is important to work out the index files and to present the whole documentation in a format that is easy to read and looks good. Web pages are the best pick and are easy to generate from reStructuredText files.
Sphinx (http://sphinx.pocoo.org) is a set of scripts and docutils extensions that can be used to generate an HTML structure from our text tree. This tool is used (for instance) to build the Python documentation, and many projects are now using it for their documentation. Among its built-in features, it produces a really nice browsing system, together with a light but sufficient client-side JavaScript search engine. It also uses pygments for rendering code examples, which produces really nice syntax highlights.
Sphinx can be easily configured to stick with the document landscape defined in the earlier section. It can be easily installed with pip as Sphinx package.
The easiest way to start working with Sphinx is to use the sphinx-quickstart script. This utility will generate a script together with Makefile, which can be used to generate the web documentation every time it is needed. It will interactively ask you some questions and then bootstrap the whole initial documentation source tree and configuration file. Once it is done, you can easily tweak it whenever you want. Let's assume we have already bootstrapped the whole Sphinx environment and we want to see its HTML representation. This can be easily done using the make html command:
project/docs$ make html sphinx-build -b html -d _build/doctrees . _build/html Running Sphinx v1.3.6 making output directory... loading pickled environment... not yet created building [mo]: targets for 0 po files that are out of date building [html]: targets for 1 source files that are out of date updating environment: 1 added, 0 changed, 0 removed reading sources... [100%] index looking for now-outdated files... none found pickling environment... done checking consistency... done preparing documents... done writing output... [100%] index generating indices... genindex writing additional pages... search copying static files... done copying extra files... done dumping search index in English (code: en) ... done dumping object inventory... done build succeeded. Build finished. The HTML pages are in _build/html.
Figure 4 An example HTML version of documentation built with Sphinx – http://graceful.readthedocs.org/en/latest/
Besides the HTML versions of the documents, the tool also builds automatic pages, such as a module list and an index. Sphinx provides a few docutils extensions to drive these features. The main ones are:
- A directive that builds a table of contents
- A marker that can be used to register a document as a module helper
- A marker to add an element in the index
Sphinx provides a toctree directive that can be used to inject a table of contents in a document with links to other documents. Each line must be a file with its relative path, starting from the current document. Glob-style names can also be provided to add several files that match the expression.
For example, the index file in the cookbook folder, which we have previously defined in the producer's landscape, can look like this:
======== Cookbook ======== Welcome to the Cookbook. Available recipes: .. toctree:: :glob: *
With this syntax, the HTML page will display a list of all the reStructuredText documents available in the cookbook folder. This directive can be used in all the index files to build a browsable documentation.
For module helpers, a marker can be added so that it is automatically listed and available in the module's index page:
======= session ======= .. module:: db.session The module session...
Notice that the db prefix here can be used to avoid module collision. Sphinx will use it as a module category and will group all modules that start with db. in this category.
Another option can be used to fill the index page by linking the document to an entry:
======= session ======= .. module:: db.session .. index:: Database Access Session The module session...
Two new entries, Database Access and Session, will be added in the index page.
Finally, Sphinx provides an inline markup to set cross-references. For instance, a link to a module can be done like this:
:mod:`db.session`
Here, :mod: is the module marker's prefix and `db.session` is the name of the module to be linked to (as registered previously); keep in mind that :mod: as well as the previous elements are the specific directives introduced in reSTructuredText by Sphinx.
Note
Sphinx provides a lot more features that you can discover on its website. For instance, the autodoc feature is a great option to automatically extract your doctests to build the documentation. Refer to http://sphinx.pocoo.org.
Sphinx really improves the readability and experience of reading documentation from the consumer's point of view. As already said, it is especially helpful when some of its parts are tightly coupled to the code, so in the form of dosctrings or module helpers. While this approach really makes it easier to ensure that the source version of the documentation matches with the code it documents, it does not guarantee that documentation readership will have access to the latest and most up to date compiled version.
Having only minimal source representation is also not enough if the target readers of the documentation are not proficient enough with command-line tools and will not know how to build it into browsable and readable form. This is why it is important to build your documentation into a consumer-friendly form automatically whenever any change to the code repository is committed/pushed.
The best way to host the documentation built with Sphinx is to generate an HTML build and serve it as a static resource with your web server of choice. Sphinx provides proper Makefile to build HTML files with the make html command. Because make is a very common utility, it should be very easy to integrate this process with any continuous integration systems discussed in Chapter 8, Managing Code.
If you are documenting an open source project with Sphinx, then you will make your life a lot easier by using Read the Docs (https://readthedocs.org/). It is a free service for hosting documentation of open source Python projects with Sphinx. The configuration is completely hassle-free and it integrates very easily with two popular code hosting services: GitHub and Bitbucket. In practice, if you have your accounts properly connected and code repository properly set up, enabling documentation hosting on Read the Docs is a matter of just a few clicks.