Sphinx documentation on GitHub

    Quickly publish documentation alongside your code by following this easy recipe

    Ben Welsh Los Angeles Times
    DOX'D: The instructions for the Data Desk's Python wrapper for the DocumentCloud API are an example of Sphinx documentation hosted by GitHub.

    The Data Desk of­ten pub­lishes doc­u­ment­a­tion along­side its code on Git­Hub, a so­cial net­work­ing site where de­velopers share their work. Git­Hub’s main ser­vice — free for open-source pro­jects — is for man­aging source code, but it also provides tick­et track­ing, wi­kis and a simple file-host­ing ser­vice for pub­lish­ing HTML pages.

    It’s that last trick that’s great for doc­u­ment­a­tion. The ser­vice is called Git­Hub Pages and it’s tailored to host sites gen­er­ated by Je­kyll, a frame­work for cre­at­ing stat­ic sites with the Ruby pro­gram­ming lan­guage. But, if you make the right moves, you can get it to pub­lish just about any HTML. Not only is it live on­line, but every change you make will be saved in the same ver­sion-con­trol sys­tem as the code.

    But what to put up there? You could hand­craft your doc­u­ment­a­tion, writ­ing out every HTML tag in lov­ing longhand. Or you could be lazy like the Data Desk and use soft­ware that does the hard work for you.

    The Data Desk uses Sphinx, a frame­work for writ­ing and or­gan­iz­ing doc­u­ment­a­tion writ­ten in the Py­thon pro­gram­ming lan­guage. You provide the text and it provides a com­plete base tem­plate and auto­mat­ic­ally gen­er­ates a table of con­tents, nav­ig­a­tion bar and links between dif­fer­ent pages. It even in­dexes your site and sets up a simple JavaS­cript-based search.

    The text is ex­pec­ted in re­Struc­tured­Text markup and the base tem­plate can be themed to look however you like. The Data Desk’s style is based on the design of The Times’ Sunday news­pa­per and can be seen here, here, here, here and here. You can also see Sphinx in use on doc­u­ment­a­tion for Py­thon’s core, pip, Django and Flask.

    It’s a bit of trick to get Sphinx to play nice with Git­Hub Pages, but it can be done. Here’s how.

    Step by step

    Git­Hub is powered by Git, the ver­sion con­trol sys­tem de­veloped for man­aging the Linux source code. Most everything you do on Git­hub is centered on Git-man­aged code re­pos­it­or­ies.

    So that’s where we start. If you don’t have a Git­Hub ac­count already, set one up. Then cre­ate a new code re­pos­it­ory and fol­low the in­struc­tions to get it set up on your com­puter.

    To start, your code is go­ing to go in the “mas­ter” branch cre­ated when Git first ini­tial­izes your re­pos­it­ory. Nor­mally, you cre­ate oth­er branches when you want to fork away and make changes in­de­pend­ent of what’s in the mas­ter branch.

    With Git­Hub Pages, any­thing you com­mit to a branch titled gh-pages will in­stead be pub­lished at ht­tp://your_user­name.git­hub.com/your_re­pos­it­ory_­name/. So the first step to pub­lish­ing any­thing is to cre­ate that branch.

    $ git branch gh-pages

    Now jump in­to the new branch.

    $ git checkout gh-pages

    Be­fore we can start work, we need to clear out any­thing from the mas­ter branch and start fresh. To make that hap­pen, you have to run a few ob­scure Git com­mands. Here they are.

    $ git symbolic-ref HEAD refs/heads/gh-pages
    $ rm .git/index
    $ git clean -fdx

    Once that’s over, in­stall Sphinx from the PyPI pack­age re­pos­it­ory us­ing pip.

    $ pip install sphinx

    Once Sphinx is in­stalled, in­struct it to cre­ate a new doc­u­ment­a­tion set.

    $ sphinx-quickstart

    After you is­sue the com­mand, Sphinx will ask you a series of ques­tions that con­fig­ure your pro­ject. You can give the de­fault an­swer in every case ex­cept one. The second ques­tion it will ask is:

    Separate source and build directories (y/N) [n]:

    You must an­swer yes. That means you should type Y and then press enter on your key­board.

    Next you have to modi­fy Sphinx’s Makefile so it will build your site in the root dir­ect­ory. Open up the Makefile and change the eighth line to:

    BUILDDIR      = ./

    Then go down to the 39th line and change the html com­mand to look like this:

        @echo "Build finished. The HTML pages are in $(BUILDDIR)/html."

    To veri­fy it works, in­struct Sphinx to build your site for the first time. You will need to is­sue this com­mand every time you wish to up­date the files on your loc­al sys­tem.

    $ make html

    Fi­nally, you need to cre­ate an empty file in the root dir­ect­ory that lets Git­Hub know you aren’t us­ing Je­kyll to struc­ture your site.

    $ touch .nojekyll

    Now all that’s left is to com­mit your HTML and push it up to Git­Hub.

    $ git add .
    $ git commit -am "First commit"
    $ git push origin gh-pages

    Wait a few minutes and vis­it ht­tp://your_user­name.git­hub.com/your_re­pos­it­ory_­name/. There you should find a simple Sphinx site live on the Web. To up­date it, edit the files in the source dir­ect­ory, run make html again and push to Git­Hub.

    Ty­ing it to­geth­er

    The routine above will work, but it is tire­some to run through it each time you start a new re­pos­it­ory. You can auto­mate the pro­cess by chain­ing all of the com­mands to­geth­er and ex­ecut­ing them as a batch. The Data Desk does this us­ing Fab­ric, a tool for stream­lin­ing Py­thon tasks. It was de­signed for de­ploy­ing code to re­mote ma­chines, but it’s just as use­ful for run­ning com­mands on your loc­al com­puter.

    If you don’t have Fab­ric in­stalled already, re­trieve it from PyPI.

    $ pip install fabric

    Once it’s ready, cre­ate a new file called fabfile.py at the root of your mas­ter branch and paste in the fol­low­ing code.

    from fabric.api import *
    from fabric.contrib.console import confirm
    def create_sphinx_pages():
        Create a new branch with Sphinx documentation ready to be published
        using GitHub's Pages system.
        Example usage:
            $ fab make_sphinx_branch
        Before you can publish your docs, you need to commit them to the repo.
            $ git add .
            $ git commit -am "First commit"
        Then publish the files by pushing them up to GitHub.
            $ git push origin gh-pages
        Then the docs will appear on GitHub at:
        # Create the new branch
        local("git branch gh-pages")
        # Move into it
        local("git checkout gh-pages")
        # Clear it out
        local("git symbolic-ref HEAD refs/heads/gh-pages")
        local("rm .git/index")
        local("git clean -fdx")
        # Install sphinx
        local("pip install sphinx")
        # Save the dependencies to the requirements file
        local("pip freeze > requirements.txt")
        # Warn the user of a quirk before configuring with Sphinx
        confirm(""".    ___ ___ _     ___ ___  _       
      /\  |   | |_ |\ | |   |  / \ |\ | 
     /--\ |   | |_ | \| |  _|_ \_/ | \|
    Sphinx is about to start configuring your project.
    You can accept the default settings it offers, EXCEPT ONE.
    The second question it will ask is:
    'Separate source and build directories (y/N) [n]:'
        # Start up a Sphinx project
        # Create the .nojekyll file GitHub requires
        local("touch .nojekyll")
        # Make the patches to Sphinx's Makefile we need
        local("echo '' >> Makefile")
        local("echo 'BUILDDIR      = ./' >> Makefile")
        local("echo '' >> Makefile")
        local("echo 'html:' >> Makefile")
        local("echo '\t$(SPHINXBUILD) -b html $(ALLSPHINXOPTS) $(BUILDDIR)' >> Makefile")
        local("echo '\t@echo' >> Makefile")
        local("echo '\t@echo \"Build finished. The HTML pages are in $(BUILDDIR)\"' >> Makefile")
        # Make the branch for the first time
        local("make html")

    Fab­ric does ex­actly the same thing we walked through above, but it does it with one simple com­mand.

    $ fab create_sphinx_pages

    And all that’s left is the hon­or of pub­lish­ing it for the first time.

    $ git add .
    $ git commit -am "First commit"
    $ git push origin gh-pages

    Much re­spect due

    This post pulls to­geth­er ideas worked out else­where by Luca Sbar­della, Dami­en Lebrun, Mi­chael Jones and Tom Pre­ston-Wern­er, who all de­serve our thanks.

    Readers: What’s your take? Share it here.


    Latest work

      About The Data Desk

      This page was created by the Data Desk, a team of reporters and Web developers in downtown L.A.