Generalize the conversion engine from markdown-source-only to a (from→to)
dispatcher, convert.Convert, supporting:
md → docx | html | pdf
docx → md | html
html → md | docx
- convertToMarkdown (docx→md, html→md): pandoc -t gfm --wrap=none with an
embedded inline-media.lua filter that base64-inlines mediabag images as data:
URIs, so the output .md is self-contained (markdown has no --embed-resources).
- convertToHTML now takes a source format: docx→html reuses the doctype template
and --embed-resources base64-inlines the docx's images automatically.
- convertToDocx takes a source format: html→docx embeds images natively.
- ToDocx/ToHTML/ToPDF are kept as the md-source entry points, delegating to the
shared internals. writeScratchFiles generalizes the old template-set writer.
Routing (converthandler.go):
- RecognizeVirtualConvert maps any target ext {md,docx,html,pdf} to the first
existing real sibling source by precedence (md←docx,html; docx←md,html;
html←md,docx; pdf←md). Real files still win (dispatcher stats first).
- ServeConverted accepts md; buildAndStore dispatches on (ext(src), format) via
convert.Convert; purgeConverted clears all derived siblings on any write.
Tests: per-direction command-shape assertions (convert) + recognizer matrix and
precedence (handler). Verified end-to-end with real pandoc (docx→md/html,
html→md/docx, base64 images). Full ./... suite green.
PDF stays markdown-only for now (docx/html→pdf would need a two-stage hop).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The convert engine renders markdown→HTML/PDF through named doctype templates
selected by the document's `template:` front matter, with per-project/per-party
overrides.
convert package:
- embed.go now embeds the whole templates/ dir (all: prefix so _-prefixed
partials are included) as an embed.FS; drop the single viewer-template.html +
custom.css embeds. New TemplateSet type + DefaultTemplateSet(name) returning the
chosen doctype + its partials.
- ToHTML/ToPDF take a TemplateSet; writeTemplateSetToScratch materialises the
template + partials flat into the per-call scratch dir (pandoc resolves
$partial()$ from the template's own directory).
handler:
- converttemplate.go: templateNameFromFrontMatter (YAML front-matter scan,
sanitized to a bare basename) + resolveTemplateSet, which overlays
<level>/.zddc.d/templates/<name>.html overrides onto the embedded defaults,
walking docDir→fsRoot so a party dir beats the project-global dir. An override
may replace a doctype, a partial, or add a brand-new doctype.
- buildAndStore threads fsRoot + source into the html/pdf paths.
build: pandoc/templates/ is the single source of truth; shared/build-lib.sh
sync_pandoc_templates mirrors it into the embed dir on every build (cmp-guarded,
stale-pruning). convert.TestEmbeddedTemplatesMatchSource fails on drift.
Tests: drift + DefaultTemplateSet (convert); front-matter parse + cascade
override precedence (handler). Full ./... suite green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
New endpoint GET /<path>/foo.md?convert=docx|html|pdf renders a markdown
source on demand. Surfaced as the Download buttons in browse's markdown
editor (separate commit).
Execution model — two upstream container images, lazy-pulled:
• docker.io/pandoc/latex:latest — MD→DOCX, MD→HTML (entrypoint pandoc)
• docker.io/zenika/alpine-chrome — HTML→PDF (entrypoint chromium-browser)
No custom image build. The runner passes --pull=missing on every podman/
docker invocation so the operator only needs the runtime installed —
first request pulls the image, subsequent requests use the local cache.
Overrides: --convert-pandoc-image / --convert-chromium-image (and the
matching ZDDC_CONVERT_* env vars). Engine: --convert-engine (podman
preferred, docker fallback). Resource caps: --convert-mem-mib (512),
--convert-cpus (2), --convert-pids (100), --convert-timeout (30s).
PDF flow is two-stage: pandoc renders the markdown through the embedded
viewer-template.html to standalone HTML, then chromium prints that HTML
via --print-to-pdf. Preserves the print-media CSS already authored in
viewer-template.html rather than going through pandoc's LaTeX template.
Each conversion runs in a throw-away container with --rm --network=none
--read-only --tmpfs=/tmp --cap-drop=ALL --security-opt=no-new-privileges
--env=HOME=/tmp plus a bind-mounted scratch dir for I/O. Pandoc reads
markdown from stdin / writes to stdout; the viewer template lives at
/tpl (ro). Chromium reads HTML from a read-write bind mount at /pdf
and writes the PDF to the same mount; the host reads it back. No shell
wrappers, no shell quoting — argv flows straight into each image's
entrypoint.
On-disk cache at <dir>/.converted/<base>.<ext> with mtime synced to the
source. Fast path is a stat-and-serve with no exec; slow path
singleflights concurrent requests for the same target. PUT/DELETE/MOVE
on the source .md purges the .converted/ sidecars.
Per-project template variables (client/project/contractor/project_number)
come from a new .zddc `convert:` cascade block, walked leaf→root with
per-key latest-wins. Filename-derived variables (title, tracking_number,
revision, status, is_draft) come from a new zddc.ParseFilename helper.
If neither podman nor docker is on PATH, the endpoint serves 503 with
a clear Retry-After. The rest of the server keeps working.
This is the first os/exec site in the codebase. The hardening in
internal/convert/runner.go — context.CancelFunc → process kill,
cmd.WaitDelay, platform-specific SysProcAttr (Setpgid + Pdeathsig on
Linux), minimal env, stdout cap via limitWriter, stderr ring buffer —
sets the pattern for any future shell-outs.
Public surface:
convert.ToDocx(ctx, source, meta) / .ToHTML / .ToPDF
convert.Probe(ctx, engineOverride) → install Runner if engine present
convert.SetImages(pandoc, chromium)
convert.ConfigureLimits(memMiB, cpus, pids, timeout)
convert.Available()
Container handler at internal/handler/converthandler.go; dispatcher
branch in cmd/zddc-server/main.go inserts the convert lookup after the
existing ACL gate, reusing the source file's read policy verbatim.
