Rob Sutherland’s Musings on Life, Code, and Anything Else

How I Manage Content on This Site

I had been toying with the idea of using something like Hugo. I wanted the site to be a static site. Hugo is fairly simple, very fast, and a fairly small learning curve. However, it was a bit more overhead than I really wanted. I highly recommend Hugo if you're looking for a nearly full-featured static site generator. For me, it was just to much.

I wanted to satisfy this fairly simple criteria:

  1. One template
  2. Text content managed in Markdown files
  3. Scriptable. Write content, generate the static site, push to live.

I wanted to explore a few things in Javascript on the client side. I originally thought about using client-side routing and having only markdown files on the server. Fetch the Markdown files when requested, convert them to HTML, and display them to the reader all on the client. I explored this, but stopped because I would have to have something on the server to map all requests to the same html file. I didn't want anything on the server.

So I decided I'd use a very simple Python script to convert the Markdown content to HTML. That's what will get published.

The process is simple enough. Details of how it works and the first version of the script are below.

How it works

The script exists and runs from the current directory. It will clean and It reads in the _template.html file. The final rendered content will be written to the docs/ folder. This could be any folder really. I'm using GitHub Pages to host the site. My options were to use the root of the repository or the docs/ folder. I opted for the docs/ folder so that all of the content, the script, and the final site can be hosted in the same repository.

simple directory structure

static/

The contents of the static folder in the root of the site will be copied to the docs/static/ folder. This folder houses CSS, JavaScript, and any other static resource. It will be copied as is so files can be in subfolders.

_template.html

This is the HTML template that will be used for every page of the site. It has a <content /> element somewhere in it that will be replaced with the rendered html from processing the Markdown files.

content/

All of the files and folders in the content/ folder will be processed and the rendered html will be output in a corresponding location in the docs/ folder. Any file name that starts with an underscore will be ignored as those are considered drafts. I use Visual Studio Code to manage the content. I create links within the Markdown files to other Markdown files. Those links allow me to navigate the site in the editor efficiently. A simple find and replace for .md to .html will enable the rendered html to navigate correctly as well.

site.py

This script converts the _template.html file and all the Markdown files in the content/ folder into the final rendered site. The script works by the conventions outlined above. The idea is to keep overhead as light as possible and work for the way I want to work.

from os import PathLike
from pathlib import Path
from shutil import copy, rmtree
import marko
import re


def put_content_in_template(template: str, content: str) -> str:
    return template.replace("<content />", content)


def replace_md_links_with_html_links(content: str) -> str:
    return re.sub("href=['\"](.*)\.md['\"]", 'href="\g<1>.html"', content)


def read_file_content(p: PathLike) -> str:
    with open(p, mode="r") as o:
        return o.read()


def write_file_content(p: PathLike, content: str) -> None:
    with open(p, mode="w") as o:
        o.write(content)


def copy_recursive(src: PathLike, dest: PathLike) -> None:
    """
    Recursively copies all files and folders in a source path
    to the destination path.
    """

    dest.mkdir(exist_ok=True, parents=True)

    for f in src.glob("*"):
        target = dest.joinpath(f.name)
        if f.is_file():
            copy(f, target)
        if f.is_dir():
            copy_recursive(f, target)


def process_markdown(
    src: PathLike, dest: PathLike, template: str = "<content />"
) -> None:
    """
    Recursivly process markdown files into html files with the
    same folder structure.
    """

    dest.mkdir(exist_ok=True, parents=True)

    for f in src.glob("*"):
        if f.is_file() and f.suffix == ".md" and f.stem[0] != "_":
            target = dest.joinpath(f.stem + ".html")
            h = marko.convert(read_file_content(f))
            h = put_content_in_template(template, h)
            h = replace_md_links_with_html_links(h)
            write_file_content(target, h)
        if f.is_dir():
            process_markdown(f, dest.joinpath(f.name), template)


if __name__ == "__main__":

    template_path = Path("./_template.html")
    content_root = Path("./content")
    static_root = Path("./static")

    destination = Path("./docs")

    if destination.exists():
        rmtree(destination)

    destination.mkdir(exist_ok=True, parents=True)

    static_dest = destination.joinpath("static")

    # copy static to [www]/static
    copy_recursive(static_root, static_dest)

    template = read_file_content(template_path)

    # convert markdow files in [content]/**/*.md
    # to html in [www]/**/*.html
    process_markdown(content_root, destination, template)