ZDDC/zddc/internal/convert/convert.go
ZDDC b5aab81d31 feat(zddc): MD→{docx,html,pdf} server-side conversion via stock pandoc + chromium containers
New endpoint GET /<path>/foo.md?convert=docx|html|pdf renders a markdown
source on demand. Surfaced as the Download buttons in browse's markdown
editor (separate commit).

Execution model — two upstream container images, lazy-pulled:

  • docker.io/pandoc/latex:latest  — MD→DOCX, MD→HTML (entrypoint pandoc)
  • docker.io/zenika/alpine-chrome — HTML→PDF (entrypoint chromium-browser)

No custom image build. The runner passes --pull=missing on every podman/
docker invocation so the operator only needs the runtime installed —
first request pulls the image, subsequent requests use the local cache.
Overrides: --convert-pandoc-image / --convert-chromium-image (and the
matching ZDDC_CONVERT_* env vars). Engine: --convert-engine (podman
preferred, docker fallback). Resource caps: --convert-mem-mib (512),
--convert-cpus (2), --convert-pids (100), --convert-timeout (30s).

PDF flow is two-stage: pandoc renders the markdown through the embedded
viewer-template.html to standalone HTML, then chromium prints that HTML
via --print-to-pdf. Preserves the print-media CSS already authored in
viewer-template.html rather than going through pandoc's LaTeX template.

Each conversion runs in a throw-away container with --rm --network=none
--read-only --tmpfs=/tmp --cap-drop=ALL --security-opt=no-new-privileges
--env=HOME=/tmp plus a bind-mounted scratch dir for I/O. Pandoc reads
markdown from stdin / writes to stdout; the viewer template lives at
/tpl (ro). Chromium reads HTML from a read-write bind mount at /pdf
and writes the PDF to the same mount; the host reads it back. No shell
wrappers, no shell quoting — argv flows straight into each image's
entrypoint.

On-disk cache at <dir>/.converted/<base>.<ext> with mtime synced to the
source. Fast path is a stat-and-serve with no exec; slow path
singleflights concurrent requests for the same target. PUT/DELETE/MOVE
on the source .md purges the .converted/ sidecars.

Per-project template variables (client/project/contractor/project_number)
come from a new .zddc `convert:` cascade block, walked leaf→root with
per-key latest-wins. Filename-derived variables (title, tracking_number,
revision, status, is_draft) come from a new zddc.ParseFilename helper.

If neither podman nor docker is on PATH, the endpoint serves 503 with
a clear Retry-After. The rest of the server keeps working.

This is the first os/exec site in the codebase. The hardening in
internal/convert/runner.go — context.CancelFunc → process kill,
cmd.WaitDelay, platform-specific SysProcAttr (Setpgid + Pdeathsig on
Linux), minimal env, stdout cap via limitWriter, stderr ring buffer —
sets the pattern for any future shell-outs.

Public surface:
  convert.ToDocx(ctx, source, meta) / .ToHTML / .ToPDF
  convert.Probe(ctx, engineOverride) → install Runner if engine present
  convert.SetImages(pandoc, chromium)
  convert.ConfigureLimits(memMiB, cpus, pids, timeout)
  convert.Available()

Container handler at internal/handler/converthandler.go; dispatcher
branch in cmd/zddc-server/main.go inserts the convert lookup after the
existing ACL gate, reusing the source file's read policy verbatim.
2026-05-13 10:33:56 -05:00

253 lines
7.8 KiB
Go

// Package convert turns a markdown source byte-buffer into DOCX, HTML,
// or PDF via two stock upstream container images: pandoc (default
// `docker.io/pandoc/latex:latest`) handles MD↔DOCX and MD→HTML, and
// a headless-chromium image (default `docker.io/zenika/alpine-chrome:latest`)
// handles HTML→PDF. No custom image build is required — the operator
// just needs `podman` or `docker` on PATH and the runner pulls each
// image on first use via `--pull=missing`.
//
// Public surface:
//
// ToDocx(ctx, source, meta) → []byte (DOCX bytes)
// ToHTML(ctx, source, meta) → []byte (standalone HTML)
// ToPDF (ctx, source, meta) → []byte (PDF, via HTML + chromium)
//
// Probe(ctx, override) → Capabilities (call once at startup)
// Available() → (Capabilities, bool)
// SetImages(pandoc, chromium) — install image refs from config
//
// All three converters are safe for concurrent use; each call gets a
// fresh container. The pandoc image's entrypoint is `pandoc`, so the
// argv we pass after the image flows straight into pandoc. The
// alpine-chrome image's entrypoint is `chromium-browser`, so the argv
// flows into chromium-browser. No `sh -c` wrappers, no shell quoting.
//
// Metadata maps to the placeholders consumed by viewer-template.html.
// title/tracking_number/revision/status/is_draft typically come from
// the source filename (zddc.ParseFilename); client/project/contractor/
// project_number from the .zddc cascade `convert:` block.
package convert
import (
"context"
"fmt"
"os"
"path/filepath"
"strings"
"sync/atomic"
"time"
)
// Metadata is the variable bag passed to pandoc as `--variable k=v`
// pairs. Fields with zero values are omitted. The viewer-template.html
// uses `$if(field)$ … $endif$` blocks so absent fields render cleanly.
type Metadata struct {
Title string
TrackingNumber string
Revision string
Status string
Client string
Project string
Contractor string
ProjectNumber string
GenerationTime time.Time
IsDraft bool
NoTOC bool
}
// Default images. Operator overrides via --convert-pandoc-image /
// --convert-chromium-image (see cmd/zddc-server). pandoc/latex carries
// TeX Live for native PDF too, so it's a superset of pandoc/core;
// operators wanting a slimmer footprint can switch to pandoc/core.
const (
DefaultPandocImage = "docker.io/pandoc/latex:latest"
DefaultChromiumImage = "docker.io/zenika/alpine-chrome:latest"
)
var (
pandocImage atomic.Pointer[string]
chromiumImage atomic.Pointer[string]
)
// SetImages installs the image refs used for subsequent ToDocx/ToHTML/
// ToPDF calls. Empty values keep the previous setting (or the
// DefaultPandocImage / DefaultChromiumImage constants on first call).
// Called from cmd/zddc-server/main.go after flag parsing.
func SetImages(pandoc, chromium string) {
if pandoc != "" {
s := pandoc
pandocImage.Store(&s)
}
if chromium != "" {
s := chromium
chromiumImage.Store(&s)
}
}
func currentPandocImage() string {
if p := pandocImage.Load(); p != nil && *p != "" {
return *p
}
return DefaultPandocImage
}
func currentChromiumImage() string {
if p := chromiumImage.Load(); p != nil && *p != "" {
return *p
}
return DefaultChromiumImage
}
// ToDocx renders source markdown to DOCX bytes. One container run via
// the pandoc image. Caller passes the full file content (envelope +
// body); pandoc handles `markdown+yaml_metadata_block` natively.
func ToDocx(ctx context.Context, source []byte, m Metadata) ([]byte, error) {
r := currentRunner()
if r == nil {
return nil, ErrUnavailable
}
cmd := []string{
"--from=markdown+yaml_metadata_block",
"--to=docx",
"--output=-",
}
cmd = append(cmd, metadataArgs(m)...)
cmd = append(cmd, "-")
return r.Run(ctx, currentPandocImage(), source, nil, cmd)
}
// ToHTML renders source markdown to standalone HTML using
// viewer-template.html. Embeds CSS + images via --embed-resources.
// Template + custom.css are bind-mounted into the container at /tpl
// from a per-call scratch dir.
func ToHTML(ctx context.Context, source []byte, m Metadata) ([]byte, error) {
r := currentRunner()
if r == nil {
return nil, ErrUnavailable
}
scratch, err := writeAssetsToScratch()
if err != nil {
return nil, fmt.Errorf("scratch: %w", err)
}
defer os.RemoveAll(scratch)
cmd := []string{
"--from=markdown+yaml_metadata_block",
"--to=html5",
"--standalone",
"--embed-resources",
"--section-divs",
"--id-prefix=",
"--html-q-tags",
"--template=/tpl/viewer-template.html",
}
if !m.NoTOC {
cmd = append(cmd, "--toc", "--toc-depth=6")
}
cmd = append(cmd, metadataArgs(m)...)
cmd = append(cmd, "--output=-", "-")
mounts := []string{scratch + ":/tpl:ro"}
return r.Run(ctx, currentPandocImage(), source, mounts, cmd)
}
// ToPDF renders source markdown to PDF in two stages: pandoc produces
// HTML using viewer-template.html (stage 1, pandoc image), then headless
// Chromium prints that HTML to PDF (stage 2, chromium image). The
// two-stage choice preserves the print-media CSS already authored in
// viewer-template.html — pandoc's native --pdf-engine path uses LaTeX
// which would bypass it entirely.
//
// Chromium runs from the alpine-chrome image whose entrypoint is
// `chromium-browser`; our cmd is the flag list passed straight to that
// binary. The host scratch dir is bind-mounted read-write at /pdf so
// chromium can write out.pdf and we read it back afterward.
func ToPDF(ctx context.Context, source []byte, m Metadata) ([]byte, error) {
html, err := ToHTML(ctx, source, m)
if err != nil {
return nil, err
}
r := currentRunner()
if r == nil {
return nil, ErrUnavailable
}
scratch, err := os.MkdirTemp("", "zddc-pdf-")
if err != nil {
return nil, fmt.Errorf("scratch: %w", err)
}
defer os.RemoveAll(scratch)
htmlPath := filepath.Join(scratch, "in.html")
pdfPath := filepath.Join(scratch, "out.pdf")
if err := os.WriteFile(htmlPath, html, 0o644); err != nil {
return nil, fmt.Errorf("write html: %w", err)
}
if err := chmodTree(scratch, 0o755, 0o644); err != nil {
return nil, err
}
mounts := []string{scratch + ":/pdf:rw"}
// alpine-chrome's entrypoint is `chromium-browser`. --no-sandbox is
// required because the container drops CAP_SYS_ADMIN; the threat
// model is "malicious markdown drives chromium RCE", contained by
// --network=none + --cap-drop=ALL + --read-only + tmpfs.
cmd := []string{
"--headless",
"--disable-gpu",
"--no-sandbox",
"--user-data-dir=/tmp/chrome",
"--no-pdf-header-footer",
"--virtual-time-budget=10000",
"--print-to-pdf=/pdf/out.pdf",
"file:///pdf/in.html",
}
if _, err := r.Run(ctx, currentChromiumImage(), nil, mounts, cmd); err != nil {
return nil, err
}
out, err := os.ReadFile(pdfPath)
if err != nil {
return nil, fmt.Errorf("read pdf: %w", err)
}
if len(out) < 4 || string(out[:4]) != "%PDF" {
return nil, &ConvertError{
Tool: "chromium",
ExitCode: 0,
Stderr: "chromium did not produce a valid PDF",
Cause: fmt.Errorf("invalid PDF magic in output (got %d bytes)", len(out)),
}
}
return out, nil
}
// metadataArgs renders Metadata into pandoc -V flags. Order is stable
// so test fixtures don't churn. Empty values are omitted (the template
// uses $if(...)$ blocks).
func metadataArgs(m Metadata) []string {
var out []string
add := func(k, v string) {
v = strings.TrimSpace(v)
if v == "" {
return
}
out = append(out, "-V", k+"="+v)
}
add("title", m.Title)
add("tracking_number", m.TrackingNumber)
add("revision", m.Revision)
add("status", m.Status)
add("client", m.Client)
add("project", m.Project)
add("contractor", m.Contractor)
add("project_number", m.ProjectNumber)
if !m.GenerationTime.IsZero() {
add("generation_time", m.GenerationTime.Format("January 02, 2006 at 3:04:05 PM MST"))
}
if m.IsDraft {
add("is_draft", "true")
}
if m.NoTOC {
add("no-toc", "true")
}
return out
}