VARASYS/ZDDC
1
1
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
ZDDC/zddc/internal/convert/convert_test.go
ZDDC da4754b6ef feat(convert): bwrap engine as production default 
Replaces the always-spawn-an-OCI-container model with a per-call
bubblewrap sandbox. Pandoc and chromium binaries are baked into the
zddc-server runtime image; each conversion runs them under bwrap's
Linux-namespace isolation. No daemon, no socket, no privileged outer
container, no OCI image pull at conversion time.

Why: the OCI engine paid ≈ 350 MB image pulls + 400 MB persistent
storage + ~300 ms per-conversion startup, plus required either an
on-host daemon socket (zddc-RCE → host-RCE in one hop) or nested
container privileges. bwrap gets the same sandbox properties
(--unshare-all, ro-bind /usr, tmpfs /tmp, clearenv, no-network) at
~5 ms per call and zero external dependencies. This is the same
primitive Flatpak uses for every app launch — battle-tested at scale
for "untrusted-input, short-lived, isolated."

Runner abstraction:
- `Runner.Run` signature: image string → ToolSpec{Image, Binary}.
  Both fields populated by entry points; whichever engine is
  installed reads the one it needs.
- `bwrapRunner` (new): assembles bwrap argv via `buildBwrapArgs`
  helper (testable in isolation), spawns bwrap with the binary.
- `containerRunner` (renamed conceptually to "legacy fallback"):
  unchanged behavior, still reachable for hosts that prefer OCI
  containers per conversion.

Probe order in health.Probe: bwrap → podman → docker. First hit wins.
Engine kinds in Capabilities: "bwrap" | "podman" | "docker". The
no-engine error message now lists all three.

Config (cmd/zddc-server):
- new --convert-pandoc-binary  / ZDDC_CONVERT_PANDOC_BINARY  (default "pandoc")
- new --convert-chromium-binary / ZDDC_CONVERT_CHROMIUM_BINARY (default "chromium-browser")
- existing --convert-pandoc-image / --convert-chromium-image kept
  for the OCI engine, doc updated to clarify they only apply there.
- --convert-engine helptext lists bwrap first.

Images:
- New `zddc/runtime.Containerfile` — alpine + bubblewrap + pandoc-cli +
  chromium + font-noto. Documents build/publish workflow.
- helm/zddc-server-prod/values.yaml.example: runtimeImage default
  switched to a placeholder for the new bundled runtime image; bare
  alpine NO LONGER works for /.convert (clearly called out in the
  comment).
- bitnest dev: /var/lib/zddc-dev-build/Containerfile mirrors the
  production runtime image. Quadlet at /etc/containers/systemd/
  zddc.container drops the podman-socket mount (no longer needed)
  and sets ZDDC_CONVERT_ENGINE=bwrap explicitly to avoid silent
  downgrades if a stray podman ends up on PATH.

Tests:
- convert_test.go: fakeRunner / recordingRunner now record ToolSpec.
- New TestToolSpecPopulation pins that both Image and Binary are
  filled by every entry point.
- New TestBwrapArgs_SandboxFlagsPresent / MountTranslation /
  RejectsBadMountSpec lock in the bwrap argv shape — a refactor that
  drops a hardening flag or misroutes a mount fails this loud.

Docs:
- AGENTS.md § "Server-side document conversion" rewritten around
  the bwrap-first model with podman/docker as legacy fallbacks.
- ARCHITECTURE.md convert reference updated.
- internal/convert package doc reflects the two-engine probe order.

Verified end-to-end on bitnest: probe reports
  engine=bwrap pandoc_binary=pandoc chromium_binary=chromium-browser
on startup. All 15 Go test packages green.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-18 17:42:28 -05:00

417 lines
13 KiB
Go
Raw Blame History

package convert
import (
"context"
"errors"
"strings"
"sync"
"testing"
"time"
)
// fakeRunner records the args it was invoked with and replays canned
// responses. Lets us assert the command lines + image refs without
// needing podman.
type fakeRunner struct {
mu sync.Mutex
calls [][]string
tools []ToolSpec
stdin [][]byte
mounts [][]string
resp []byte
err error
}
func (f *fakeRunner) Run(_ context.Context, tool ToolSpec, stdin []byte, mounts []string, cmd []string) ([]byte, error) {
f.mu.Lock()
defer f.mu.Unlock()
f.calls = append(f.calls, append([]string(nil), cmd...))
f.tools = append(f.tools, tool)
f.stdin = append(f.stdin, append([]byte(nil), stdin...))
f.mounts = append(f.mounts, append([]string(nil), mounts...))
return f.resp, f.err
}
func (f *fakeRunner) lastCall() (string, []string) {
f.mu.Lock()
defer f.mu.Unlock()
if len(f.calls) == 0 {
return "", nil
}
return f.tools[len(f.tools)-1].Image, f.calls[len(f.calls)-1]
}
func TestToDocx_UsesPandocImage(t *testing.T) {
f := &fakeRunner{resp: []byte("FAKE-DOCX")}
InstallRunner(f)
t.Cleanup(func() { InstallRunner(nil) })
SetImages("docker.io/pandoc/latex:latest", "")
out, err := ToDocx(context.Background(), []byte("# Hello\n"), Metadata{
Title: "Hello",
Client: "Acme",
})
if err != nil {
t.Fatalf("ToDocx: %v", err)
}
if string(out) != "FAKE-DOCX" {
t.Errorf("unexpected output: %q", out)
}
image, call := f.lastCall()
if image != "docker.io/pandoc/latex:latest" {
t.Errorf("expected pandoc image, got %q", image)
}
if !contains(call, "--to=docx") {
t.Errorf("missing --to=docx: %v", call)
}
if !contains(call, "title=Hello") {
t.Errorf("missing title metadata: %v", call)
}
if !contains(call, "client=Acme") {
t.Errorf("missing client metadata: %v", call)
}
// Last arg must be "-" so pandoc reads from stdin.
if call[len(call)-1] != "-" {
t.Errorf("expected stdin marker as last arg, got %q", call[len(call)-1])
}
}
func TestToHTML_UsesTemplateAndMountsScratch(t *testing.T) {
f := &fakeRunner{resp: []byte("<html>fake</html>")}
InstallRunner(f)
t.Cleanup(func() { InstallRunner(nil) })
SetImages("docker.io/pandoc/latex:latest", "")
_, err := ToHTML(context.Background(), []byte("# Hi\n"), Metadata{Title: "Hi"})
if err != nil {
t.Fatalf("ToHTML: %v", err)
}
image, call := f.lastCall()
if image != "docker.io/pandoc/latex:latest" {
t.Errorf("expected pandoc image, got %q", image)
}
if !contains(call, "--template=/tpl/viewer-template.html") {
t.Errorf("template flag missing: %v", call)
}
if !contains(call, "--toc") {
t.Errorf("TOC flag missing (default NoTOC=false): %v", call)
}
if len(f.mounts) == 0 || len(f.mounts[0]) == 0 {
t.Fatalf("expected at least one bind mount for /tpl")
}
mount := f.mounts[0][0]
if !strings.Contains(mount, ":/tpl:") {
t.Errorf("mount missing /tpl: %q", mount)
}
}
func TestToHTML_NoTOCSuppressesTOC(t *testing.T) {
f := &fakeRunner{resp: []byte("<html/>")}
InstallRunner(f)
t.Cleanup(func() { InstallRunner(nil) })
_, _ = ToHTML(context.Background(), []byte("# Hi\n"), Metadata{NoTOC: true})
_, call := f.lastCall()
if contains(call, "--toc") {
t.Errorf("TOC should be suppressed when NoTOC=true: %v", call)
}
if !contains(call, "no-toc=true") {
t.Errorf("no-toc metadata variable missing: %v", call)
}
}
// recordingRunner records every call and returns canned responses
// in sequence. Lets ToPDF tests assert the two-stage pipeline
// (pandoc image then chromium image).
type recordingRunner struct {
mu sync.Mutex
calls []recordedCall
resp [][]byte
err []error
cursor int
}
type recordedCall struct {
image string
cmd []string
mounts []string
}
func (r *recordingRunner) Run(_ context.Context, tool ToolSpec, _ []byte, mounts []string, cmd []string) ([]byte, error) {
r.mu.Lock()
defer r.mu.Unlock()
r.calls = append(r.calls, recordedCall{
image: tool.Image,
cmd: append([]string(nil), cmd...),
mounts: append([]string(nil), mounts...),
})
if r.cursor >= len(r.resp) {
return nil, nil
}
out := r.resp[r.cursor]
var e error
if r.cursor < len(r.err) {
e = r.err[r.cursor]
}
r.cursor++
return out, e
}
func TestScratchDir_UsedByToHTML(t *testing.T) {
f := &fakeRunner{resp: []byte("<html/>")}
InstallRunner(f)
t.Cleanup(func() { InstallRunner(nil); SetScratchDir("") })
scratchRoot := t.TempDir()
SetScratchDir(scratchRoot)
_, err := ToHTML(context.Background(), []byte("# Hi\n"), Metadata{})
if err != nil {
t.Fatalf("ToHTML: %v", err)
}
if len(f.mounts) == 0 || len(f.mounts[0]) == 0 {
t.Fatalf("expected at least one mount")
}
mount := f.mounts[0][0] // "<host>:/tpl:ro"
if !strings.HasPrefix(mount, scratchRoot+"/") {
t.Errorf("scratch dir not under configured root: %q (root=%q)", mount, scratchRoot)
}
}
func TestToPDF_TwoStagePipeline(t *testing.T) {
// Stage 1: pandoc emits HTML. Stage 2: chromium reads HTML from
// the bind mount and writes /pdf/out.pdf. The fake runner can't
// actually write the PDF, so we expect ToPDF to fail at the
// read-back step — but we can still assert the two-stage call
// shape and the right image per stage.
r := &recordingRunner{
resp: [][]byte{
[]byte("<html><body>fake</body></html>"), // stage 1 stdout
nil, // stage 2 stdout (chromium writes PDF to bind mount)
},
}
InstallRunner(r)
t.Cleanup(func() { InstallRunner(nil) })
SetImages("docker.io/pandoc/latex:latest", "docker.io/zenika/alpine-chrome:latest")
_, err := ToPDF(context.Background(), []byte("# Hi\n"), Metadata{})
// PDF read-back will fail (fake runner didn't write the file) —
// that's expected for this test which only inspects the call
// shape.
if err == nil {
t.Fatalf("expected error from PDF read-back; got nil")
}
if len(r.calls) != 2 {
t.Fatalf("expected 2 container calls (pandoc + chromium); got %d", len(r.calls))
}
if r.calls[0].image != "docker.io/pandoc/latex:latest" {
t.Errorf("stage 1 image: got %q want pandoc/latex", r.calls[0].image)
}
if r.calls[1].image != "docker.io/zenika/alpine-chrome:latest" {
t.Errorf("stage 2 image: got %q want alpine-chrome", r.calls[1].image)
}
// Stage 2 must include the --print-to-pdf flag pointing at /pdf.
if !contains(r.calls[1].cmd, "--print-to-pdf=/pdf/out.pdf") {
t.Errorf("chromium call missing --print-to-pdf flag: %v", r.calls[1].cmd)
}
if !contains(r.calls[1].cmd, "--no-sandbox") {
t.Errorf("chromium call missing --no-sandbox: %v", r.calls[1].cmd)
}
// Stage 2's bind mount must be writable (chromium writes the PDF).
if len(r.calls[1].mounts) == 0 || !strings.Contains(r.calls[1].mounts[0], ":rw") {
t.Errorf("chromium mount must be :rw, got %v", r.calls[1].mounts)
}
}
func TestErrUnavailable_WhenNoRunner(t *testing.T) {
InstallRunner(nil)
_, err := ToDocx(context.Background(), []byte("x"), Metadata{})
if !errors.Is(err, ErrUnavailable) {
t.Errorf("expected ErrUnavailable, got %v", err)
}
}
func TestMetadataArgs_OmitsEmptyAndOrdersStably(t *testing.T) {
args := metadataArgs(Metadata{
Title: "T",
Project: "P",
GenerationTime: time.Date(2026, 5, 13, 14, 30, 22, 0, time.UTC),
})
want := []string{
"-V", "title=T",
"-V", "project=P",
}
for i, w := range want {
if i >= len(args) || args[i] != w {
t.Fatalf("args[%d]: got %v want prefix %v", i, args, want)
}
}
joined := strings.Join(args, "|")
if !strings.Contains(joined, "generation_time=") || !strings.Contains(joined, "2026") {
t.Errorf("generation_time missing or malformed: %v", args)
}
if strings.Contains(joined, "client=") {
t.Errorf("empty client should not be passed: %v", args)
}
}
func TestImageTag(t *testing.T) {
cases := map[string]string{
"docker.io/pandoc/latex:latest": "pandoc/latex",
"docker.io/zenika/alpine-chrome:latest": "zenika/alpine-chrome",
"pandoc/core": "pandoc/core",
"quay.io/example/foo:v1": "example/foo",
"alpine": "alpine",
}
for in, want := range cases {
if got := imageTag(in); got != want {
t.Errorf("imageTag(%q) = %q, want %q", in, got, want)
}
}
}
func TestSingleflight_Collapses(t *testing.T) {
var g singleflightGroup
const N = 50
var wg sync.WaitGroup
var hits int32
var mu sync.Mutex
wg.Add(N)
for i := 0; i < N; i++ {
go func() {
defer wg.Done()
_, _ = g.Do("k", func() (any, error) {
mu.Lock()
hits++
mu.Unlock()
time.Sleep(20 * time.Millisecond)
return "v", nil
})
}()
}
wg.Wait()
if hits != 1 {
t.Errorf("singleflight collapse: got %d invocations, want 1", hits)
}
}
// contains reports whether haystack has needle as any of its elements.
func contains(haystack []string, needle string) bool {
for _, s := range haystack {
if s == needle {
return true
}
}
return false
}
// TestToolSpecPopulation: the convert entry points populate BOTH the
// Image and Binary fields of ToolSpec, so the runner-of-the-day can
// pick whichever it needs. bwrapRunner reads Binary; containerRunner
// reads Image; the call site doesn't know which is installed.
func TestToolSpecPopulation(t *testing.T) {
f := &fakeRunner{resp: []byte("ok")}
InstallRunner(f)
t.Cleanup(func() { InstallRunner(nil) })
SetImages("docker.io/pandoc/latex:1.0", "docker.io/zenika/alpine-chrome:2.0")
SetBinaries("/opt/bin/pandoc", "/opt/bin/chromium")
t.Cleanup(func() { SetImages("", ""); SetBinaries("", "") })
if _, err := ToDocx(context.Background(), []byte("# x\n"), Metadata{}); err != nil {
t.Fatalf("ToDocx: %v", err)
}
if len(f.tools) != 1 {
t.Fatalf("want 1 tool call, got %d", len(f.tools))
}
got := f.tools[0]
if got.Image != "docker.io/pandoc/latex:1.0" {
t.Errorf("Image = %q, want docker.io/pandoc/latex:1.0", got.Image)
}
if got.Binary != "/opt/bin/pandoc" {
t.Errorf("Binary = %q, want /opt/bin/pandoc", got.Binary)
}
}
// TestBwrapArgs_SandboxFlagsPresent locks in the bwrap argv shape.
// Every conversion must run with these hardening flags — the whole
// point of bwrap-as-default is that the sandbox is built into every
// invocation. A refactor that drops any of them needs to fail this
// test loudly.
func TestBwrapArgs_SandboxFlagsPresent(t *testing.T) {
args, err := buildBwrapArgs("pandoc", nil, []string{"--from=markdown", "--to=docx", "-"})
if err != nil {
t.Fatalf("buildBwrapArgs: %v", err)
}
mustHave := []string{
"--unshare-all", // net + pid + ipc + uts + cgroup
"--unshare-user-try", // user-namespace when kernel allows
"--die-with-parent", // cleanup when zddc-server exits
"--proc", // minimal /proc
"--dev", // minimal /dev
"--tmpfs", // writable /tmp scratch
"--clearenv", // no host env leaks
}
for _, flag := range mustHave {
if !contains(args, flag) {
t.Errorf("bwrap args missing sandbox flag %q: %v", flag, args)
}
}
// /usr must be bind-mounted read-only — that's how the binary
// + its dynamic libs are visible inside the sandbox. The
// "--ro-bind /usr /usr" triple must appear consecutively.
if i := indexOfTriple(args, "--ro-bind", "/usr", "/usr"); i < 0 {
t.Errorf("bwrap args missing --ro-bind /usr /usr: %v", args)
}
// Binary + caller-cmd come last, in order.
last := args[len(args)-4:]
want := []string{"pandoc", "--from=markdown", "--to=docx", "-"}
for i, w := range want {
if last[i] != w {
t.Errorf("trailing args[%d] = %q, want %q", i, last[i], w)
}
}
}
// TestBwrapArgs_MountTranslation: caller "host:target:ro" → bwrap
// "--ro-bind host target"; "host:target:rw" → "--bind host target";
// no mode segment defaults to ro (mirroring containerRunner).
func TestBwrapArgs_MountTranslation(t *testing.T) {
args, err := buildBwrapArgs("pandoc",
[]string{"/host/tpl:/tpl:ro", "/host/pdf:/pdf:rw", "/host/x:/x"},
nil)
if err != nil {
t.Fatalf("buildBwrapArgs: %v", err)
}
if i := indexOfTriple(args, "--ro-bind", "/host/tpl", "/tpl"); i < 0 {
t.Errorf("missing --ro-bind /host/tpl /tpl: %v", args)
}
if i := indexOfTriple(args, "--bind", "/host/pdf", "/pdf"); i < 0 {
t.Errorf("missing --bind /host/pdf /pdf: %v", args)
}
if i := indexOfTriple(args, "--ro-bind", "/host/x", "/x"); i < 0 {
t.Errorf("missing default-ro --ro-bind /host/x /x: %v", args)
}
}
// TestBwrapArgs_RejectsBadMountSpec: a malformed mount string fails
// fast, never reaches exec. Single-segment specs (no target) and
// unknown modes both qualify.
func TestBwrapArgs_RejectsBadMountSpec(t *testing.T) {
for _, bad := range []string{"only-host", "/h:/t:weird", ""} {
if _, err := buildBwrapArgs("pandoc", []string{bad}, nil); err == nil {
t.Errorf("expected error for malformed mount %q", bad)
}
}
}
// indexOfTriple returns the index of `a` in args such that
// args[i:i+3] == {a, b, c}, or -1.
func indexOfTriple(args []string, a, b, c string) int {
for i := 0; i+2 < len(args); i++ {
if args[i] == a && args[i+1] == b && args[i+2] == c {
return i
}
}
return -1
}
Reference in a new issue View git blame Copy permalink