Hosting your Swift Library Docs on Github Pages

The beta for Xcode 13.3 dropped yesterday. With it came a released version of Swift 5.6 and a bunch of neat additions that the 5.6 release enables. A feature I was watching closely was two-fold: the capability for plugins to extend the commands available within swift’s package manager, and a static hosting option that was added to the documentation compiler tool (DocC) that Apple announced and open sourced this past year.

The two elements came together with the initial release of swift-docc-plugin, a plugin to swift package manager that adds the ability to generate documentation on the command line with the command swift package generate-documentation. Before I go any farther, I should note that when the DocC team put this together, they knew a lot of folks wanted to host their content as if it were static HTML – so they took the time to document HOW to do that, and best of all – they hosted _that_ documentation on github pages (I’m assuming using their own tools to do it). The articles, all of which are sourced from the content in the git repository, are hosted on GitHub pages:

• Getting Started with the Swift-DocC Plugin
• Generating Documentation for a Specific Target
• Previewing Documentation
• Generating Documentation for Hosting Online
• Publishing to Github Pages

This works, and quite nicely – I pushed up documentation for the Lindenmayer library I’ve been working on using the following command:

swift package \
    --allow-writing-to-directory ./docs \
    --target Lindenmayer \
    generate-documentation \
    --output-path ./docs \
    --emit-digest \
    --disable-indexing \
    --transform-for-static-hosting \
    --hosting-base-path 'Lindenmayer'

BUT… there are some quirks you should be aware of.

First, the ability to statically host the content does NOT mean that it is static content. DocC is a different take on managing documentation from almost all of the similar, prior tools. Where Doxygen, JavaDoc, Sphinx, Hugo, AsciiDoc and others read through the source and generate HTML, that is not what DocC is doing. While the content dumped out for static hosting ultimately includes HTML, DocC relies on two other critical components to do its work. It starts with the compiler generating what’s called a Symbol Graph. That’s a file that contains all the symbols – types, properties, type aliases, etc – in your source code, and the relationships between them. DocC then tweaks and adjusts that graph of symbols and more specifically mixes it with additional (authored, not automatic) markdown files from a directory – which is called the documentation catalog. If the markdown files in the documentation catalog, or the original source, don’t provide content or structure for the relationships in the symbol graph, then DocC builds up a default set, and attempts to provide a default structure. For what it’s worth, this default content set is where the dreaded “No Overview Available” comes from. The resulting combination is serialized into a bunch of JSON files, stored in the filesystem. Those JSON files contain the information needed to render the content – but the core of that content rendering happens from another project, A Vue-based JavaScript single page app: swift-docc-render.

The JSON files allows this documentation content to be more easily consumed by more than just browsers. I think it’s a reasonable assumption to presume this is what drives and enables the Xcode quick-help summary information.

Back to DocC and static hosting – what this means is that the content that gets dumped into the “docs” directory isn’t just plain HTML – it’s the Javascript and all the associated content as well. The effect this has is that while it LOOKS like you could just pop open the index.html in that directory and see your content, it’s not going to work. Instead you’ll just get a blank page, and if you happen to look at the JavaScript console, you’ll see errors reporting that the requested URL wasn’t found on this server.

It also means that the content isn’t available at the root of the GitHub pages you pushed. The root for my Lindenmayer project is https://heckj.github.io/Lindenmayer/ – but going there directly doesn’t show anything. Instead you need to go a couple directories down: https://heckj.github.io/Lindenmayer/documentation/lindenmayer/. Also note that the name of the library is lower-cased. The first thing I tried was https://heckj.github.io/Lindenmayer/documentation/Lindenmayer/, which didn’t work either.

The key thing is to be aware that the URL you want to point people to has that documentation/your_library_name_lowercased extended on it. Oh – and that first repetition of the name is the github repo, in case you don’t have the benefit of having them the same. For example, for the swift automerge library, the reposity is automerge-swift, while the package name is automerge. The URL for the hosted pages on github then becomes: https://automerge.github.io/automerge-swift/documentation/automerge/.

SIDE NOTE:

The example generate-documentation command I provided above has extra bits in it that you probably don’t need, in particular the --emit-digest option. This option generates an additional JSON file at the top level of the content (linkable-entities.json) which contains a list of all of the (public) symbols within the library. I’ve intentionally chosen to include this file in the content I’m hosting on Github pages (at http://heckj.github.io/Lindenmayer/linkable-entities.json)
, although it’s not (to my knowledge) used by the single-page app that displays the HTML content. The formal content definition (written as an OpenAPI spec) is defined in the DocC repository at https://github.com/apple/swift-docc/blob/main/Sources/SwiftDocC/SwiftDocC.docc/Resources/LinkableEntities.json. The short form is that it provides a list of all the symbols that I can use to reference symbols within that content, which otherwise proved of hard to get.

If you’re curious what this looks like, try the following command (assuming you have curl and jq installed):

curl -sL http://heckj.github.io/Lindenmayer/linkable-entities.json | jq '.[].referenceURL' -r.

The output looks something like:

doc://Lindenmayer/documentation/Lindenmayer/RewriteRuleLeftDirectRightRNG/description
doc://Lindenmayer/documentation/Lindenmayer/SceneKitRenderer/generateScene(lsystem:)-5b948
doc://Lindenmayer/documentation/Lindenmayer/RenderCommand/draw-swift.type.property
doc://Lindenmayer/documentation/Lindenmayer/RewriteRuleLeftDirectDefines/CustomStringConvertible-Implementations

These are the full reference links used within the DocC, and a portion of these reference links are what can be used as symbols within the markdown files in the documentation catalog.

<doc://Lindenmayer/documentation/Lindenmayer/SceneKitRenderer/generateScene(lsystem:)-5b948>

maps to the symbol:

``Lindenmayer/SceneKitRenderer/generateScene(lsystem:)-5b948``

In a few cases, those little hash extensions (the -5b948 bit in the example) are tricky to find. Xcode does a reasonable job of dropping them in using code completion, but I’ve found a few bugs where they were hard to ascertain. This JSON file with all the symbols is exactly what I needed to get a full list.

I haven’t (yet?) figured out a means to transform these doc:// resource URLs into web URLs, but I’ve got a notion there’s a means to enable that for the ability to cross-link documentation when libraries build, or depend, on other libraries. Maybe that’s ultimately what this digest is for, but if so – it’s not a feature that’s easily usable yet from DocC. I’m still exploring the internals of DocC, but there’s an idea of a resolver – which can be an external process or service – that provides this mapping for DocC to build the links (I think?) it needs to enable that functionality.