Ross Patterson's Blog - Posts tagged reStructuredText

Using Directories for Sphinx Pages

2021-04-05T00:00:00+00:00

Creating Sphinx pages as ./foo/index.rst has a number of benefits over ./foo.rst including path consistency and organizing content.

When I first started playing with ABlog, and thus learning more about Python’s wonderful documentation tool Sphinx, I observed that Sphinx actually creates a directory in the built HTML output for each reStructuredText ./**.rst file in the source. For example, if the source for this post had been ./blog/sphinx-page-dirs.rst, Sphinx would render that to ./_website/blog/sphinx-page-dirs/index.html.

One consequence of this is that a relative link to another area of the site will be broken in the rendered HTML build output if it’s correct in the ./**.rst source file. For example, if ./about.rst contains a link such as:

See `the blog area to see all posts <./blog/>`_.

The Sphinx HTML build output will render that ./about.rst file to ./about/index.html and thus the link will point to /about/blog/ instead of the intended /blog/. I dimly recall encountering such path issues with other types of reST path references as well. Path consistency is also particularly handy for editors or other tools that can infer enough to open files from path references in other files, such as FFAP in Emacs. Writing the source reST for a page as ./about/index.rst in the first place nicely avoids those problems and empowers such tools.

I’m not a big fan of embedding code of one sort into content (or code) of another sort. For example, as big an Org-mode fan as I am, I’ve never been able to get into using Babel. In this case, when writing documentation or other text content, I resist embedding code into source text files and prefer to have such code in separate files next to the text. If the code I want to include in documentation is actual used code, as was the case with the export/import scripts that migrated this blog, then it saves time copying and pasting and prevents out-of-date content when forgetting to do so. Even if it is example or demonstration code of more than a few lines, having it in separate sibling files supports using all our favorite tools (static analysis, IDEs, editors, etc.) which further prevents incorrect or misleading examples. So instead of reStructuredText’s :: literal blocks or Markdown’s indented code blocks I’d much rather include code from separate files. Using a directory for such pages, e.g. ./blog/migrating-from-plone-to-ablog/, also allows us to keep such related files organized neatly together.

Finally, using a directory for a Sphinx page supports putting all related content into a separate VCS repository checkout, such as via $ git submodule .... This can be very useful for sharing code related to one page without having to share the source and VCS history for the whole site. For example, I have an upcoming post with a non-trivial amount of working and tested demo code I want to share. I can use this approach to make that code and VCS history public while still preserving the option of having private content or VCS history for the rest of this site.

Migrating From Plone To ABlog

2021-03-16T00:00:00+00:00

My aspiration to return to blogging thanks to reStructuredText, ABlog and GitLab CI/CD.

I stopped blogging years ago. I was never very good at blogging regularly, I always seem to prefer adding further polish to whatever I’m working on over sharing what I’ve learned. But it didn’t help that my old blogging platform, self-hosted Plone, was too heavy for the job. The maintenance work kept piling up and every time I thought of blogging, I’d remember that TODO list and then do something else instead. :-/

I want to be very clear. Plone, the open-source enterprise CMS, is a simply awesome piece of software and a remarkable open-source project. The list of features provided is both extensive and powerful and the capacity to extend those features and frameworks to build custom CMS’ish applications is even more powerful. Finally and foremost, being involved in the open-source community that develops and maintains that software ecosystem has been the most rewarding part of my career in software. Lovely people.

Plone is, however, laughably overkill for a software developer’s professional blog. My intention was to eat my own dog-food by using the CMS that I spent most of my time working with at the time. That intention is perfectly reasonable but my work stream has mostly moved away from Plone in the intervening years and the maintenance burden is never small for such a large feature set, even if you’re using almost none of those features. I’ll try to learn the lesson here to moderate the impulse to dog-food.

Blogging is writing. Text (as opposed to a WYSIWYG word processor, for example) is the format for writing that least gets in the way when one is proficient with a programmer’s editor. Python’s magnificent reStructuredText is the most expressive text markup language out there and provides a number of features that one inevitably needs when doing much more than writing a heading, 3 paragraphs, 2 lists and 5 links. Sphinx is the lingua franca for writing documentation in the Python world and beyond, and for very good reason.

I’ve known for some time, given the above, that I wanted to blog in reST and I also want to become more familiar with Sphinx, so I went looking for static site generators based on Sphinx. I evaluated ABlog and Tinkerer, both blogging-oriented static site generators based on Sphinx. I like that ABlog seems to be a little less opinionated and calls itself a Sphinx extension since I may well want to use this platform for static site content beyond just blogging. It also seems like ABlog is actively maintained while the last release of Tinkerer was in 2017. Given what I read in a few other reviews and comments I made the call to go with Ablog.

Next up I needed to decide whether to preserve my Plone blogging content or start afresh. Given that I’m bad at getting around to blogging, I thought exporting my old posts would help fill things out a but, though one glance at the dates gives that game away. ;-) It also seemed like a fun problem to solve.

Given that the target is text files and a static site generator, I decided to write the export script oriented towards files on the filesystem and as agnostic about the internals of Plone as possible. I’m pretty happy with it for such a pragmatic bit of “software” and I think it does a surprisingly decent job of capturing as much as possible the data and metadata of Plone content in a format most appropriate for files in a filesystem and oriented to file and text-oriented tools. In particular, it exports text fields the format of the “native” MIME type of that field instance, mostly so that posts I wrote in reST would remain in that format on export. It will also export multiple, separate file-like fields to separate files preserving any per-field filename if available. For example, the lead images for Plone News Item instances, were written next to the corresponding reST file with the original uploaded image basename. Finally, it also uses the FTP/WebDAV support built into Zope to export most, if not all, other fields to a corresponding *.properties file in RFC822 format.

Now that I had files reflecting the Plone representation of the content, I created a new repository, copied the files there and began working on massaging them into ABlog posts. I learned about the reST ‘.. raw:: html’ directive for the Plone content I wrote using the WYSIWYG editor and was thus in HTML format in the exported files. It works nicely and some quick shell commands created an *.rst file corresponding to each *.html file:

.. raw:: html
   :file: ./{id}.html

Thus having *.rst files for each exported post, it was time to convert them into ABlog posts. I wanted to preserve all metadata from the *.properties files that corresponded to ABlog post metadata or could reasonably be added to the reST surrounding the main content of the post. Again, I’m pretty happy with how much the import script was able to preserve. I don’t think I’ll miss any of the excluded metadata:

#######
{title}
#######

    {description}

.. raw:: html
   :file: ./{id}.html


.. meta::
   :description: {description}
   :keywords: {subject}

.. post:: {creation_date}
   :tags: {subject}
   :author: Ross Patterson
   :redirect:  @@redirect-to-uuid/{UID}

.. update:: {modification_date}

   Imported from Plone on {now}.  The date for this update is the last
   modified date in Plone.

If any of the above export/import code would be useful to anyone else, LMK and I’ll throw it up to a separate GitLab repository under an open-source license along with my VCS history.

With all the content showing up nicely in the output from the ABlog watchdog, auto-rebuild development server, I fixed a few formatting issues in the imported content, updated the boilerplate content, and went looking for a theme. I do prefer dark themes but mostly, much to my chagrin, I’ve noticed that many if not most people seem to think less of you if you show them something with the default theme, even when the person showing you is decidedly not a designer. ;-) I found a dark Solarized theme that extends the default theme and contributed some fixes. It’s certainly not perfect, but I like it better than the default and it certainly expresses that this is the blog of a developer.

Finally, I created a private GitLab repository and whipped up a simple GitLab CI/CD configuration that builds and deploys the built static site to GitLab Pages when I push to master. I added some GitLab redirects for the few paths that changed. BTW, this is one of the nice things about ABlog so far, most paths stayed the same. I pushed to master and viola!

Do LMK if you find any bugs. This ended up being more complete than I would have expected so I’d love to know of any issues I missed.