Using Sphinx for PHP Project Documentation

I recently had the need to write proper prose-like source-code documentation for the Diffbot PHP client. Having looked at several documentation generators, and even having suggested a @prose tag for importing of related MD/reST documents into method and class descriptions, I realized there simply is no perfect solution we can all agree on (yet). So until I extend Sage with a @prose token and reST parsing, I opted for ReadTheDocs and Sphinx.

RTD is widely used in the industry. It hosts powerful docs such as Guzzle’s, PhpSpec’s and many more. It supports reST alongside MD (or, to be more accurate, MD alongside reST), which is a huge plus as RST files are more suitable for highly technical documents. It can be run locally and generate offline-friendly HTML files, but it can also compile from documentation source available online and and be automatically hosted as a subdomain of readthedocs.org.

That said, setting it up for a PHP project has some caveats, so we’ll go through a basic guide in this tutorial.

TL;DR

If you’re just looking for the list of commands to get up and running quickly:

sudo pip install sphinx sphinx-autobuild sphinx_rtd_theme sphinxcontrib-phpdomain
mkdir docs
cd docs
sphinx-quickstart
wget https://gist.githubusercontent.com/Swader/b16b18d50b8224f83d74/raw/b3c1d6912aefc390da905c8b2bb3660f513af713/requirements.txt

After the quickstart setup, to activate the theme and PHP syntax highlights run:

sed -i '/extensions = \[\]/ c\extensions = \["sphinxcontrib.phpdomain"\]' source/conf.py
echo '

import sphinx_rtd_theme
html_theme = "sphinx_rtd_theme"
html_theme_path = [sphinx_rtd_theme.get_html_theme_path()]
    
# Set up PHP syntax highlights
from sphinx.highlighting import lexers
from pygments.lexers.web import PhpLexer
lexers["php"] = PhpLexer(startinline=True, linenos=1)
lexers["php-annotations"] = PhpLexer(startinline=True, linenos=1)
primary_domain = "php"

' >> source/conf.py

To build HTML:

make html

or

sphinx-build -b html source build

The latter command supports several options you can add into the mix.

Sphinx Installation

ReadTheDocs uses Sphinx behind the scenes, and as such is a through-and-through Python implementation. To make use of it, we need to install several prerequisites. We’ll use our trusty Homestead Improved to set up a brand new environment for us to play in.

After the VM is set up, SSH into it with vagrant ssh and execute:

sudo pip install sphinx sphinx-autobuild

If you don’t have the command pip, follow the official instructions to get it installed, or just execute the following if on Ubuntu:

sudo apt-get install python-sphinx python-setuptools
sudo easy_install pip

These tools have now made the command sphinx-quickstart available.

Recommended Folder Layout

Generally, you’ll either have:

1. the documentation in the same folder as the project you’re documenting, or…
2. the documentation in its own repository.

Unless the documentation spans several projects or contexts, it is recommended it be in the same folder as the project. If you’re worried about bloating the size of your project when Composer is downloading it for use, the docs can be easily kept out of the distribution by being placed into the .gitattributes file (see here).

When we run the command sphinx-quickstart, it’ll ask us for the root folder of our docs. This is the folder into which all other subfolders regarding the docs will go. Note that this is not the project’s root folder. If your project is at my-php-project, the root of the docs will likely be in something like my-php-project/docs.

Next, Sphinx offers to either make a separate _build folder for the compiled version of the docs (e.g. HTML), while the sources will be in the root (defined in the previous step), or to make two folders under root: source and build, keeping the root clean. Feel free to choose whichever option you prefer (we went with the latter, for cleanliness and structure).

Follow the rest of the instructions to set some meta data, select .rst as the file extension, and finally answer “no” to all questions about additional plugins – those refer to Python projects and are outside our jurisdiction. Likewise, when asked to create make files, accept.

Custom Theme

Building the documents with the command make html produces the HTML documents in the build folder. Opening the documents in the browser reveals a screen not unlike the following:

That’s not very appealing. This theme is much more stylish and modern. Let’s install it.

pip install sphinx_rtd_theme

Then, in the source folder of the docs root, find the file conf.py and add the following line to the top, among the other import statements:

import sphinx_rtd_theme

In the same file, change the name of the HTML theme:

html_theme = "sphinx_rtd_theme"

Finally, tell Sphinx where the theme is by asking the imported module:

html_theme_path = [sphinx_rtd_theme.get_html_theme_path()]

Building the docs with make html should now produce some significantly prettier HTML:

Most themes follow the same installation procedure. Here is a short list. Hopefully, it’ll expand in the future.

Table of Contents

During the quickstart, a user is asked for the name of the master file (typically index). The master file usually contains little to no content – rather, it only holds directives.

A reST directive is like a function – a powerful construct of the reST syntax which accepts arguments, options, and a body. The toctree directive is one of them. It requires an option of maxdepth, indicating the maximum number of levels in a single menu item (e.g. depth of 2 is Mainmenu -> Submenu1 but not -> Submenu2).

After the option goes a blank line, and then a one-per-line list of include files, without extensions. Folders are supported (subfolder/filetoinclude).

.. Test documentation master file, created by
   sphinx-quickstart on Sat Aug  8 20:15:44 2015.
   You can adapt this file completely to your liking, but it should at least
   contain the root `toctree` directive.

Welcome to Test's documentation!
================================

Contents:

.. toctree::
   :maxdepth: 2

   overview
   quickstart

In the example above, Sphinx prints out Contents:, followed by an expanded version of the table of contents, i.e. a bulleted list of all headings found in the included documents. Additionally, many authors include extra information at the top of the master file to give a birds-eye overview of the library right then and there. See Guzzle’s.

The toctree definition in the master file will be automatically mirrored in the left ToC navigation bar.

Let’s grab the overview and quickstart files from Guzzle so that we don’t have to write our own. Put them into the root of the docs, and rebuild with make html.

The documentation should now appear, and the left ToC should be expandable with the little plus icons:

For more information about the toctree directive and everything it can do to give you truly customized output, see here.

PHP Syntax

If we look at the quickstart document, we’ll notice that the PHP samples aren’t syntax highlighted. Not surprising, considering Sphinx defaults to Python. Let’s fix this.

In the source/conf.py file, add the following:

from sphinx.highlighting import lexers
from pygments.lexers.web import PhpLexer
lexers['php'] = PhpLexer(startinline=True, linenos=1)
lexers['php-annotations'] = PhpLexer(startinline=True, linenos=1)
primary_domain = 'php'

This imports the PHP lexer and defines certain code block language-hints as specifically parseable by the module. It also sets the default mode of the documentation to PHP, so that if you omit a language declaration on a code block, Sphinx will just assume it’s PHP. E.g., instead of:

.. code-block:: php

    use myNamespace/MyClass;
    ...

one can type:

.. code-block::

    use myNamespace/MyClass;
    ...

After adding the above into the configuration file, a rebuild is necessary.

make html

This should produce syntax highlighted PHP sections:

PHP Domain

Additionally, we can install the PHP domain.

Domains are sets of reST roles and directives specific to a programming language, making Sphinx that much more adept at recognizing common concepts and correctly highlighting and interlinking them. The PHP domain, originally developed by Mark Story, can be installed via:

sudo pip install sphinxcontrib-phpdomain

The extension needs to be activated by changing the extensions line to:

extensions = ["sphinxcontrib.phpdomain"]

Let’s try and make another reST file now, with a described PHP class so we can see how well the PHP domain does its magic. We’ll create source/class.rst, and add class to the index.rst file under all the others. Then, we’ll put the following into class.rst:

DateTime Class
==============

.. php:class:: DateTime

  Datetime class

  .. php:method:: setDate($year, $month, $day)

      Set the date.

      :param int $year: The year.
      :param int $month: The month.
      :param int $day: The day.
      :returns: Either false on failure, or the datetime object for method chaining.


  .. php:method:: setTime($hour, $minute[, $second])

      Set the time.

      :param int $hour: The hour
      :param int $minute: The minute
      :param int $second: The second
      :returns: Either false on failure, or the datetime object for method chaining.

  .. php:const:: ATOM

      Y-m-d\TH:i:sP

If we rebuild, we get something like this:

Note that without the PHP Domain installed, this screen would be empty.

This doesn’t look too bad, but it could be better. We’ll leave the styling for another day, though.

View Source

It is common for docs to include a link to their source files at the top, so that users can suggest changes, raise issues and send pull requests for improvements if they spot something being out of place.

In the configuration options, there is a flag to show/hide these source links. By default, they’ll lead to _sources/file where file is the currently viewed file, and _sources is a directory inside the build directory – i.e., all source files are copied to build/_sources during the build procedure.

We don’t want this, though. We’ll be hosting the docs on Github, and we want sources to lead there, no matter where they are hosted. We can do this by adding HTML context variables into the conf.py file:

html_context = {
  "display_github": True,
  "github_user": "user",
  "github_repo": project,
  "github_version": "master",
  "conf_py_path": "/doc/",
  "source_suffix": source_suffix,
}

Be careful to add this block after the project definition, else you’ll get an error about the project variable not being defined. Putting this at the bottom of conf.py is generally a safe bet.

ReST vs MD

For a quick and dirty intro to reST, see this, but also look into the custom markup added by the Sphinx team – these additional features help you get the best out of your documentation.

ReST has many more features than MD does, so for parity’s sake and an easier transition, here’s a great conversion guide and a one-for-one comparison of features neatly laid out in tabular format.

Adding MD

Sometimes, you may not be willing or able to convert existing MD documentation into reST. That’s okay, Sphinx can dual-wield MD/reST support.

First, we need to install the MD processing module:

sudo pip install recommonmark

We also need to import the parser into source/conf.py:

from recommonmark.parser import CommonMarkParser

Finally, we need to tell Sphinx to send .md files to the parser by replacing the previously defined source_suffix line with:

source_suffix = ['.rst', '.md']
source_parsers = {
    '.md': CommonMarkParser,
}

If we try it out by adding a file testmd.md into source with the contents:

# TestMD!

We are testing md!

## Second heading!

Testing MD files.

---

    echo "Here is some PHP code!"

Rebuilding should now show the MD content as fully rendered – with one caveat. The syntax won’t be highlighted (not even if we put the code into a PHP code fence). If someone has an idea about why this happens and how to avoid it, please let us know in the comments.

Hosting on ReadTheDocs

Now that our documentation is ready, we can host it online. For the purpose of this demo, we create a repo of sample content at http://github.com/sitepoint-editors/samplesphinx-php.

To host the docs online, we first add a new project…

The next screen asks for a connection with Github. After importing repositories, we click Create on the one we want to create an RTD project from and confirm some additional values which can all be left at default.

After a build successfully finishes, our docs should be live:

Note: this check used to be required, but RTD seems to have fixed the issue and you can now use the same theme declaration both in the local version, and the live one.

Extensions on RTD

Earlier in this tutorial, we installed the PHP Domain for easier directive-powered PHP class descriptions. This extension is not available on ReadTheDocs, though.

Luckily, ReadTheDocs utilizes virtualenv and can install almost any module you desire. To install custom modules, we need the following:

• a requirements.txt file in the repo somewhere
• the path to this file in the Advanced Settings section of our project on ReadTheDocs

To get a requirements file, we can just save the output of the command pip freeze into a file:

pip freeze > requirements.txt

The freeze command will generate a long list of modules, and some of them might not be installable on ReadTheDocs (Ubuntu specific ones, for example). You’ll have to follow the errors in the build phases and remove them from the file, one by one, until a working requirements file is reached, or until RTD improve their build report to flag errors more accurately.

For all intents and purposes, a file such as this one should be perfectly fine:

Babel==2.0
CommonMark==0.5.4
Jinja2==2.8
MarkupSafe==0.23
PyYAML==3.11
Pygments==2.0.2
Sphinx==1.3.1
sphinxcontrib-phpdomain==0.1.4
alabaster==0.7.6
argh==0.26.1
argparse==1.2.1
docutils==0.12
html5lib==0.999
meld3==0.6.10
pathtools==0.1.2
pytz==2015.4
recommonmark==0.2.0
six==1.5.2
snowballstemmer==1.2.0
sphinx-autobuild==0.5.2
sphinx-rtd-theme==0.1.8
wheel==0.24.0

After re-running the online build (happens automatically) the documented PHP class should be available, just as it was when we tested locally.

Troubleshooting

ValueError: unknon locale: UTF-8

It’s possible you’ll get the error ValueError: unknown locale: UTF-8 on OS X after running either sphinx-quickstart or make html. If this happens, open the file ~/.bashrc (create it if it doesn’t exist), put in the content:

export LC_ALL=en_US.UTF-8
export LANG=en_US.UTF-8

… save and close it, and then load it with the command source ~/.bashrc. The error should now no longer show up.

Table of Contents does not display / outdated

Sometimes when adding new files to include into index.rst, the ToC in the left sidebar will be out of date. For example, clicking on a file that was built before the new file was added will show a ToC with the latest file’s heading missing. The cause of this is unknown but it’s easily fixed by forcing a full rebuild:

rm -rf build/
make html

The first command removes the build folder’s contents completely, forcing Sphinx to regenerate everything on make html.

Build Failed

If your builds fail and you can’t discern the reason, explicitly defining the location of conf.py under Advanced settings in Admin sometimes helps.

Conclusion

In this tutorial, we learned how we can quickly set up a Sphinx documentation workflow for PHP projects in an isolated VM, so that the installations don’t interfere with our host operating system. We installed the PHP highlighter, configured the table of contents, installed a custom theme, went through some basic ReST syntax and hosted our docs on ReadTheDocs. In a followup post, we’ll focus on styling, documentation versions and localization.

Do you use another documentation writing workflow? If so, which and why? Any other Sphinx/RTD tips? Let us know!

Frequently Asked Questions (FAQs) about Using Sphinx for PHP Project Documentation