How It Works#

A build is four passes.

1. Collect — walk content/, split frontmatter, parse each file into a block tree. Inline text is left as text.
2. Graph — index every name and alias, resolve every [[link]], invent a seed for every name nothing wrote, collect mention excerpts, compute backlinks and Related Notes.
3. Render — turn block trees into HTML, resolving wikilinks through the graph, expanding embeds, laying out global and local SVG graphs, and writing the static search index.
4. Emit — write each output only if its bytes differ from last time, then delete anything the previous build produced and this one did not.

collect.py   walk and parse            graph.py     the thesis
md_block.py  lines -> blocks           render.py    pages -> files
md_inline.py spans, and the hard part  cache.py     what to actually write
md_html.py   blocks -> html            template.py  the small template engine

Incremental builds have no rules#

The obvious design is a rule: rebuild the note that changed, the notes it links to, and the notes that link to it.

That rule is wrong, in at least seven ways:

• Co-citation is a two-hop relationship, so editing one note changes the related list of notes two steps away.
• Adding one alias re-resolves links across the whole site and deletes a seed page.
• Removing the last mention of a name must delete a page, which is not a rebuild at all.
• Index and archive pages list notes they have no link edge to.
• An embedded note's content is part of the embedding note's output.
• Mention counts change with any new mention anywhere.
• Templates and config affect everything.

Every one of those failures is silent. A stale related panel looks perfectly fine; you only find out when someone follows a link that no longer exists.

So mdssg has no invalidation rules. The graph is recomputed from scratch on every build — it is set arithmetic over small structures, and it was never the expensive part — and each output is written only when its bytes differ from what the manifest recorded. That is correct by construction: there is nothing to enumerate, so there is nothing to get wrong.

The test suite asserts it directly. A clean build and an incremental build, after edits of every shape above, must produce byte-identical output trees.

What was actually slow#

The plan for this project assumed parsing markdown was where the time went, and that a parse cache would be the important optimisation.

Measuring a 2,000-note garden said parsing was 5% of a build. The real costs were somewhere else entirely:

• the template loader re-reading every template once per page — eleven thousand file opens to render two thousand pages
• inline text being parsed three times per block, once for the graph, once for the heading table, once to render
• typo detection running difflib over every note name for every seed

Fixing those three took a cold build from 6.0s to 2.5s and a rebuild-after-one- edit from 4.0s to 1.2s. The parse cache was never built, because it would have bought 5%.

Things that were harder than they look#

pattern.match(s, pos) does not anchor ^ at pos. It still anchors at the start of the string. This silently disabled every footnote and every reference link until a test caught it.

Path("/var/www").is_absolute() is False on Windows — a path needs a drive letter to be absolute there. The guard stopping config paths from escaping the site worked on Linux and did nothing on Windows.

isinstance(ChainMap, dict) is False. ChainMap is a Mapping, not a dict subclass, so every variable inside a {% for %} loop resolved to nothing.

Two writes a moment apart can share an mtime, and same-length edits are common. A template cache keyed on (mtime, size) stopped noticing changes, which in serve looks exactly like the template being ignored. Hashes decide; timestamps are only a fast path.

A BOM belongs to the encoding, not the document. Windows editors add one, and it lands in front of the opening ---, silently demoting frontmatter to a paragraph. Files are read as utf-8-sig.

Hash normalised text, not raw bytes. Otherwise a Windows checkout with core.autocrlf reports every file as modified and incrementality never engages.

Deliberate omissions#

No plugin system. No shortcodes. No taxonomy engine. No asset pipeline. No client framework.

The dependency list is empty and the intent is to keep it that way.

CI also generates 1,500 linked notes, performs a clean build under a fixed time budget, then requires the unchanged rebuild to write zero files. This catches performance regressions without putting a timing assertion in every platform and Python version of the unit-test matrix.