The Data Desk of­ten pub­lishes doc­u­ment­a­tion along­side its code on Git­Hub, a so­cial net­work­ing site where de­velopers share their work. Git­Hub’s main ser­vice — free for open-source pro­jects — is for man­aging source code, but it also provides tick­et track­ing, wi­kis and a simple file-host­ing ser­vice for pub­lish­ing HTML pages.

It’s that last trick that’s great for doc­u­ment­a­tion. The ser­vice is called Git­Hub Pages and it’s tailored to host sites gen­er­ated by Je­kyll, a frame­work for cre­at­ing stat­ic sites with the Ruby pro­gram­ming lan­guage. But, if you make the right moves, you can get it to pub­lish just about any HTML. Not only is it live on­line, but every change you make will be saved in the same ver­sion-con­trol sys­tem as the code.

But what to put up there? You could hand­craft your doc­u­ment­a­tion, writ­ing out every HTML tag in lov­ing longhand. Or you could be lazy like the Data Desk and use soft­ware that does the hard work for you.

The Data Desk uses Sphinx, a frame­work for writ­ing and or­gan­iz­ing doc­u­ment­a­tion writ­ten in the Py­thon pro­gram­ming lan­guage. You provide the text and it provides a com­plete base tem­plate and auto­mat­ic­ally gen­er­ates a table of con­tents, nav­ig­a­tion bar and links between dif­fer­ent pages. It even in­dexes your site and sets up a simple JavaS­cript-based search.

The text is ex­pec­ted in re­Struc­tured­Text markup and the base tem­plate can be themed to look however you like. The Data Desk’s style is based on the design of The Times’ Sunday news­pa­per and can be seen here, here, here, here and here. You can also see Sphinx in use on doc­u­ment­a­tion for Py­thon’s core, pip, Django and Flask.

It’s a bit of trick to get Sphinx to play nice with Git­Hub Pages, but it can be done. Here’s how.

Step by step

Git­Hub is powered by Git, the ver­sion con­trol sys­tem de­veloped for man­aging the Linux source code. Most everything you do on Git­hub is centered on Git-man­aged code re­pos­it­or­ies.

So that’s where we start. If you don’t have a Git­Hub ac­count already, set one up. Then cre­ate a new code re­pos­it­ory and fol­low the in­struc­tions to get it set up on your com­puter.

To start, your code is go­ing to go in the “mas­ter” branch cre­ated when Git first ini­tial­izes your re­pos­it­ory. Nor­mally, you cre­ate oth­er branches when you want to fork away and make changes in­de­pend­ent of what’s in the mas­ter branch.

With Git­Hub Pages, any­thing you com­mit to a branch titled gh-pages will in­stead be pub­lished at ht­tp://your_user­name.git­hub.com/your_re­pos­it­ory_­name/. So the first step to pub­lish­ing any­thing is to cre­ate that branch.

$ git branch gh-pages

Now jump in­to the new branch.

$ git checkout gh-pages

Be­fore we can start work, we need to clear out any­thing from the mas­ter branch and start fresh. To make that hap­pen, you have to run a few ob­scure Git com­mands. Here they are.

$ git symbolic-ref HEAD refs/heads/gh-pages
$ rm .git/index
$ git clean -fdx

Once that’s over, in­stall Sphinx from the PyPI pack­age re­pos­it­ory us­ing pip.

$ pip install sphinx

Once Sphinx is in­stalled, in­struct it to cre­ate a new doc­u­ment­a­tion set.

$ sphinx-quickstart

After you is­sue the com­mand, Sphinx will ask you a series of ques­tions that con­fig­ure your pro­ject. You can give the de­fault an­swer in every case ex­cept one. The second ques­tion it will ask is:

Separate source and build directories (y/N) [n]:

You must an­swer yes. That means you should type Y and then press enter on your key­board.

Next you have to modi­fy Sphinx’s Makefile so it will build your site in the root dir­ect­ory. Open up the Makefile and change the eighth line to:

BUILDDIR      = ./

Then go down to the 39th line and change the html com­mand to look like this:

html:
    $(SPHINXBUILD) -b html $(ALLSPHINXOPTS) $(BUILDDIR)
    @echo
    @echo "Build finished. The HTML pages are in $(BUILDDIR)/html."

To veri­fy it works, in­struct Sphinx to build your site for the first time. You will need to is­sue this com­mand every time you wish to up­date the files on your loc­al sys­tem.

$ make html

Fi­nally, you need to cre­ate an empty file in the root dir­ect­ory that lets Git­Hub know you aren’t us­ing Je­kyll to struc­ture your site.

$ touch .nojekyll

Now all that’s left is to com­mit your HTML and push it up to Git­Hub.

$ git add .
$ git commit -am "First commit"
$ git push origin gh-pages

Wait a few minutes and vis­it ht­tp://your_user­name.git­hub.com/your_re­pos­it­ory_­name/. There you should find a simple Sphinx site live on the Web. To up­date it, edit the files in the source dir­ect­ory, run make html again and push to Git­Hub.

Ty­ing it to­geth­er

The routine above will work, but it is tire­some to run through it each time you start a new re­pos­it­ory. You can auto­mate the pro­cess by chain­ing all of the com­mands to­geth­er and ex­ecut­ing them as a batch. The Data Desk does this us­ing Fab­ric, a tool for stream­lin­ing Py­thon tasks. It was de­signed for de­ploy­ing code to re­mote ma­chines, but it’s just as use­ful for run­ning com­mands on your loc­al com­puter.

If you don’t have Fab­ric in­stalled already, re­trieve it from PyPI.

$ pip install fabric

Once it’s ready, cre­ate a new file called fabfile.py at the root of your mas­ter branch and paste in the fol­low­ing code.

from fabric.api import *
from fabric.contrib.console import confirm

def create_sphinx_pages():
    """
    Create a new branch with Sphinx documentation ready to be published
    using GitHub's Pages system.

    Example usage:

        $ fab make_sphinx_branch

    Before you can publish your docs, you need to commit them to the repo.

        $ git add .
        $ git commit -am "First commit"

    Then publish the files by pushing them up to GitHub.

        $ git push origin gh-pages

    Then the docs will appear on GitHub at:

        http://<your_account_name>.github.com/<your_repo_name>/

    """
    # Create the new branch
    local("git branch gh-pages")
    # Move into it
    local("git checkout gh-pages")
    # Clear it out
    local("git symbolic-ref HEAD refs/heads/gh-pages")
    local("rm .git/index")
    local("git clean -fdx")
    # Install sphinx
    local("pip install sphinx")
    # Save the dependencies to the requirements file
    local("pip freeze > requirements.txt")
    # Warn the user of a quirk before configuring with Sphinx
    confirm(""".    ___ ___ _     ___ ___  _       
  /\  |   | |_ |\ | |   |  / \ |\ | 
 /--\ |   | |_ | \| |  _|_ \_/ | \|

Sphinx is about to start configuring your project.

You can accept the default settings it offers, EXCEPT ONE.

The second question it will ask is:

'Separate source and build directories (y/N) [n]:'

YOU MUST ANSWER YES. THAT MEANS YOU TYPE 'Y' AND PRESS ENTER.

DO YOU UNDERSTAND?""")
    # Start up a Sphinx project
    local("sphinx-quickstart")
    # Create the .nojekyll file GitHub requires
    local("touch .nojekyll")
    # Make the patches to Sphinx's Makefile we need
    local("echo '' >> Makefile")
    local("echo 'BUILDDIR      = ./' >> Makefile")
    local("echo '' >> Makefile")
    local("echo 'html:' >> Makefile")
    local("echo '\t$(SPHINXBUILD) -b html $(ALLSPHINXOPTS) $(BUILDDIR)' >> Makefile")
    local("echo '\t@echo' >> Makefile")
    local("echo '\t@echo \"Build finished. The HTML pages are in $(BUILDDIR)\"' >> Makefile")
    # Make the branch for the first time
    local("make html")

Fab­ric does ex­actly the same thing we walked through above, but it does it with one simple com­mand.

$ fab create_sphinx_pages

And all that’s left is the hon­or of pub­lish­ing it for the first time.

$ git add .
$ git commit -am "First commit"
$ git push origin gh-pages

Much re­spect due

This post pulls to­geth­er ideas worked out else­where by Luca Sbar­della, Dami­en Lebrun, Mi­chael Jones and Tom Pre­ston-Wern­er, who all de­serve our thanks.