ZDDC/AGENTS.md

AGENTS.md — ZDDC

Commands

# ── ./build subcommands ────────────────────────────────────────────────────
# `./build` (no arg) is a source-side dev build only — assembles tool/dist/
# + cross-compiles zddc-server. dist/release-output/ and the live site are
# left alone. Channel + release subcommands produce a complete release
# bundle in dist/release-output/ (gitignored). Run `./deploy` to publish.
# Workflow: alpha = active dev  →  beta = ready for testing  →  release = ship.

./build                              # dev build (no release bundle)
./build alpha                        # cut alpha          (cascades nothing)
./build beta                         # cut beta           (cascades alpha → beta)
./build release                      # cut stable, coordinated next version
                                     #   (cascades alpha + beta → new stable; tags all seven)
./build release 1.2.0                # cut stable at explicit version
./build help

# ── ./deploy subcommands ────────────────────────────────────────────────────
# rsync the build output and content repo to /srv/zddc/ (Caddy's bind-mount).
# --delete-after — the live tree exactly mirrors source.

./deploy                             # full sync (content + releases)
./deploy --content                   # only ~/src/zddc-website/ → /srv/zddc/
./deploy --releases                  # only dist/release-output/ → /srv/zddc/releases/

# Single-tool dev build for testing (does NOT touch dist/release-output/):
sh tool/build.sh                     # archive|transmittal|classifier|mdedit|landing|form|tables|browse

# Single-tool release (rare; prefer ./build alpha|beta|release so versions
# don't drift between tools). Same flag form as before.
sh tool/build.sh --release [<version>|alpha|beta]
./freshen-channel <tool> <channel>   # rebuild one tool's alpha/beta from its current stable tag

# Test all tools
npm test

# Test single tool
npx playwright test tool             # archive | transmittal | classifier | mdedit | form-safety | tables

# Dev server (cache-busting HTTP, on port 8000)
./dev-server start
./dev-server stop

No lint, typecheck, or format commands exist — the project is plain sh + vanilla JS.

Channel/release cuts seed dist/release-output/ from the current /srv/zddc/releases/ (preserving symlinks) before running per-tool promote, then mutate the channels being cut on top. The bundle is therefore always a complete intended-live snapshot, not a sparse diff. The build ends with a channel-link verifier that asserts every <tool>_{stable,beta,alpha}.html (and zddc-server's per-platform binary mirrors + stub pages) resolves. Build fails if any link is dangling — because the bundle is complete, dangling-link errors mean a real bug. Nothing is pushed automatically. Run ./deploy to publish; commit

• push source changes to main separately.

Architecture

Eight independent single-file HTML tools (archive, transmittal, classifier, mdedit, landing, form, tables, browse). Each compiles to one self-contained .html in dist/ with all CSS and JS inlined — most name their output dist/tool.html; landing writes dist/index.html (served at / by zddc-server). Tools share a small set of canonical helpers in shared/ (filename parsing, ZDDC filter UI, theme, help) — see "Shared modules" below. form is the schema-driven renderer used by zddc-server's form-data system; tables is its read/aggregate counterpart, rendering a directory of YAML files as a sortable table whose rows click through to the form editor — discovered presence-based via <name>.table.yaml next to a sibling <name>/ rows-dir (see "Form-data system" and "Tables system" below).

tool/
  css/           source stylesheets (concatenated in order)
  js/            vanilla JS IIFEs (concatenated in order)
  template.html  placeholder markers: {{CSS_PLACEHOLDER}}, {{JS_PLACEHOLDER}}, {{BUILD_LABEL}}
  build.sh       assembles dist/tool.html
  dist/tool.html generated output — committed with `git add -f`

shared/
  base.css        CSS tokens and primitives included first by every tool's CSS build
  zddc.js         canonical filename/folder/revision parsers, formatters, status validation
  zddc-filter.js  shared ZDDC project/status filter UI module
  zddc-source.js  HTTP source abstraction — FS Access API polyfill (HttpDirectoryHandle,
                  HttpFileHandle) backed by zddc-server's listing JSON + file API
                  (PUT/DELETE/POST). Tools that auto-load the current dir in HTTP mode
                  call window.zddc.source.detectServerRoot() at init. The probe
                  returns { handle, status }: status 200 → use handle; 403 → user
                  lacks `r` on this directory (show "no permission to list"
                  message); 0 → not http(s) or non-zddc-server. Tools must
                  handle the 403 case so a permission-locked path doesn't
                  silently render as an empty welcome screen.
  hash.js         SHA-256 helpers used by the file API + classifier hashes
  theme.js        light/dark theme switcher
  help.js         shared help dialog module
  build-lib.sh    POSIX sh helpers (ensure_exists, concat_files, build_timestamp)
                  sourced by every tool's build.sh via: . "$root_dir/../shared/build-lib.sh"

# Hand-edited website content lives in a SEPARATE Codeberg repo
# (codeberg.org/VARASYS/ZDDC-website), typically cloned at
# ~/src/zddc-website/. Just content — no releases, no LFS:
#   index.html, reference.html, css/, js/, img/    hand-edited content
#   README.md, LICENSE                             repo housekeeping
#
# This repo's ./build produces a release bundle in dist/release-output/
# (gitignored, local-only). ./deploy rsyncs both into /srv/zddc/ on
# the deploy host (Caddy's bind-mount):
#   /srv/zddc/
#     index.html, reference.html, css/, js/, img/     ← from ~/src/zddc-website
#     releases/
#       index.html                        regenerated by `./build`
#       <tool>_v<X.Y.Z>.html              per-version (immutable)
#       <tool>_v<X.Y>.html  -> ...        symlink chain
#       <tool>_stable.html  -> ...        channel mirror, follows latest stable
#       <tool>_{beta,alpha}.html  -> ...  channels (cascade to stable when idle)
#       zddc-server_v<X.Y.Z>_<platform>   per-platform binary (raw bytes, no LFS)
#       zddc-server_<channel>_<platform>  channel binary mirror (symlink)
#       zddc-server_<X>.html              stub page surfacing 4 platform DLs

helm/
  zddc-server-prod/  production-shaped Helm chart (compiles from source via init container)
  zddc-server-dev/   dev-shaped variant (tracks main HEAD; debug-level logging; faster probes)
  README.md          chart design rationale + quick-start

Critical: dist/ files are gitignored. tool/dist/<tool>.html is the canonical built artifact for testing and the source for --release writes into dist/release-output/. dist/release-output/ is the local-only release bundle. Neither is in git. Never edit them directly.

Two-repo + deploy-host model. Source code lives here (codeberg.org/VARASYS/ZDDC); hand-edited website content lives in a separate repo (codeberg.org/VARASYS/ZDDC-website, typically cloned at ~/src/zddc-website/). The live site at zddc.varasys.io is /srv/zddc/ on the deploy host (Caddy bind-mount), populated by ./deploy. Release artifacts are NOT in git — they're produced by ./build alpha|beta|release into dist/release-output/ and rsync'd to /srv/zddc/releases/ by ./deploy --releases. Per-version files (HTML and zddc-server binaries) are real immutable bytes; partial-version pins (_v<X.Y>, _v<X>) and channel mirrors (_stable, _beta, _alpha) are symlinks. shared/build-lib.sh provides promote_release (HTML tools) and promote_zddc_server (binaries + matching stub pages); the top-level ./build seeds from live state, then calls them in lockstep. Older releases are reproducible from any <tool>-vX.Y.Z tag in this repo (git checkout zddc-server-v0.0.8 && ./build release 0.0.8). No Codeberg release assets, no LFS.

Shared CSS (shared/base.css)

Included as the first positional arg to every tool's concat_files CSS call. Provides:

• :root CSS custom properties — --primary, --bg, --text, --border, --font, etc.
• Brand color: --primary: #2a5a8a (matches zddc.varasys.io)
• Button primitive: .btn, .btn-primary, .btn-secondary, .btn-sm, .btn-lg, .btn-link
• .app-header + .app-header__title chrome rules
• .build-timestamp, .hidden, .truncate, webkit scrollbars

Do not define these in any tool's own CSS — they come from shared.

Toast CSS lives in classifier/css/base.css only (classifier is the only tool that uses toasts).

Transmittal CSS quirks

• transmittal/css/base.css overrides html { font-size: 16px } inside @media screen — this must stay. shared/base.css sets 14px; transmittal's floating labels are rem-based and were designed for 16px.
• The floating label position is defined in transmittal/css/forms.css, not Tailwind classes. If adding new Tailwind classes to template.html, add them to transmittal/css/utilities.css too — there is no Tailwind build step.

Build system rules

• Every build.sh sources shared/build-lib.sh first (provides ensure_exists, concat_files, build_timestamp). Set root_dir before sourcing.
• Build scripts use POSIX sh (#!/bin/sh with set -eu), not bash.
• concat_files accepts positional args only (not array names).
• awk processes template.html, replacing {{PLACEHOLDER}} markers and stripping CDN <script>/<link> tags (pattern: https?://)
• {{BUILD_LABEL}} is substituted in all six tools via gsub in awk (use gsub, not print — the placeholder is inline in an HTML line). Value is Built: <timestamp> BETA for dev builds, v<version> for stable releases, and <channel> · <date> · <sha> for alpha/beta channel builds; computed before the awk step. The shared is_red flag controls whether the label is wrapped in a red+bold <span> (true for dev/alpha/beta, false for stable).
• Cleans up temp files via trap cleanup EXIT

</ escaping is mandatory. Any JS containing </tag> inside string or template literals will break inline <script> embedding. Run:

sed 's#</#<\\/#g' "$input_js" > "$safe_js"

Required for any new tool with vendor JS or JS containing HTML template literals.

JS module pattern

All JS is vanilla, no bundlers. Files are IIFEs, registered on window.app.modules. Load order = declaration order in build.sh. window.app is the only global.

(function() {
    window.app.modules.mymodule = { ... };
})();

Exception: archive uses plain globals (APP_STATE, top-level functions) — not the IIFE/modules pattern.

ZDDC filename parsers

All parsing/formatting goes through shared/zddc.js, exposed as window.zddc. Tools call it directly — no per-tool wrappers.

window.zddc exports:

• parseFilename(name) → { trackingNumber, revision, status, title, extension, valid } | null (extension WITHOUT leading dot)
• parseFolder(name) → { date, trackingNumber, status, title, valid } | null
• parseRevision(rev) → { base, modifier, modifierType, modifierNumber, isDraft, modifierIsDraft, full }
• compareRevisions(a, b) → number (canonical sort order)
• formatFilename(parts) / formatFolder(parts) — round-trips parsed output
• isValidStatus(code) — accepts known status codes plus ---

All file objects across tools use file.trackingNumber (string) and file.extension (string, no leading dot, e.g. 'pdf' not '.pdf'). When concatenating into a filename, write name + '.' + ext.

Coverage lives in tests/zddc.spec.js (47 cases). Add new edge cases there, not in tool tests.

Testing quirks

• Playwright + Chromium only (File System Access API requirement)
• Tests open dist/tool.html via file:// protocol — always build before testing
• File System Access API is mocked via page.addInitScript() using tests/fixtures/mock-fs-api.js
• Use waitUntil: 'load' or 'domcontentloaded' not 'networkidle' — bundled scripts keep the network "active"
• Archive's #noDirectoryMessage empty-state overlay is position: absolute; top: 50px — it must clear the header or it will block button clicks in tests

ZDDC filename convention

Format: trackingNumber_revision (status) - title.extension

• trackingNumber: no spaces or underscores (e.g. 123456-EL-SPC-2623)
• revision: A, B, 0; draft prefix ~; modifiers +C1, +B1, +N1, +Q1
• status: IFA IFB IFC IFD IFI IFP IFR IFU REC RSA RSB RSC RSD RSI or ---
• Folder names prefix with date: 2025-10-31_trackingNumber (status) - title

Git workflow

• Feature-branch workflow; squash-merge feature branches to main
• Conventional commits: feat(archive): ..., fix(transmittal): ...
• Release tags: <tool>-v<X.Y.Z> per tool, all nine sharing the same X.Y.Z on a coordinated cut (e.g. archive-v0.0.8, transmittal-v0.0.8, classifier-v0.0.8, mdedit-v0.0.8, landing-v0.0.8, form-v0.0.8, tables-v0.0.8, browse-v0.0.8, zddc-server-v0.0.8)
• dist/ is gitignored. Build artifacts (per-tool dist/<tool>.html and dist/release-output/) are NOT committed to this repo. Reproduce them from a tag with ./build release X.Y.Z
• Hand-edited website content lives in a separate Codeberg repo (codeberg.org/VARASYS/ZDDC-website, cloned at ~/src/zddc-website/). Source-code commits go to main here; content commits go to that repo
• Release artifacts live on the deploy host (/srv/zddc/), not in any git history. Use ./deploy to publish

Releasing — lockstep, channels, layout

Lockstep convention. Every release cut bumps all nine artifacts (8 HTML tools + zddc-server) to the same version, even if a tool didn't change. Per-tool independent versions are gone. The coordinated next-stable target is max(latest tag across all nine tools) + 1 — _coordinated_next_stable in shared/build-lib.sh. Channel cuts (alpha/beta) follow the same lockstep — every tool's channel mirror is overwritten in step. Three channels, ordered: alpha (dev iteration) → beta (general testing) → stable (ship).

Storage model. All release artifacts live on the deploy host at /srv/zddc/releases/ (Caddy bind-mount, served as https://zddc.varasys.io/releases/). Locally they materialize in this repo's dist/release-output/ (gitignored) when ./build alpha|beta|release runs; ./deploy rsyncs them out. No git history holds release artifacts — older versions are reproducible from any <tool>-vX.Y.Z tag (git checkout zddc-server-v0.0.8 && ./build release 0.0.8). No Codeberg release assets, no LFS, no third-party mirrors.

Artifact	Type	Layout
<tool>_v<X.Y.Z>.html	real, immutable	per-version HTML for each of archive, transmittal, classifier, mdedit, landing, form, tables, browse
<tool>_v<X.Y>.html, <tool>_v<X>.html	symlinks	partial-version pins
<tool>_<channel>.html	symlink (or real bytes during active channel dev)	mutable channel mirror per tool, channel ∈ {stable, beta, alpha}
zddc-server_v<X.Y.Z>_<platform>	real binary	per-version cross-compiled binary, platform ∈ {linux-amd64, darwin-amd64, darwin-arm64, windows-amd64.exe}
zddc-server_v<X.Y>_<platform>, zddc-server_v<X>_<platform>, zddc-server_<channel>_<platform>	symlinks (or real bytes during active channel dev)	partial-pin and channel mirrors per platform — same cascade as the HTML tools
zddc-server_<X>.html	generated stub page	per-version / per-channel; lists the four platform downloads. This is what the matrix-cell link points at — one stub fans out to four binaries
index.html	regenerated by build.sh	matrix table, one column per tool, one row per release

Single point of truth. ./build release is the canonical lockstep cut. It seeds dist/release-output/ from /srv/zddc/releases/ (so cascades and the verifier see a complete world), forwards each HTML tool's build with the agreed version, then promote_zddc_server (in shared/build-lib.sh) copies the freshly cross-compiled binaries into dist/release-output/ with the matching symlink chain, then write_zddc_server_stubs_all regenerates every stub page, then build_releases_index rewrites the index, then verify_channel_links asserts nothing dangles. Then the top-level build folds the regenerated zddc/internal/apps/embedded/* files into a release: vX.Y.Z lockstep commit and tags all seven artifacts at that commit. ./deploy --releases then publishes the bundle.

• Stable (./build release or --release X.Y.Z): Writes per-version HTML for the six HTML tools + per-version binaries for zddc-server (real bytes, immutable). Refreshes 5 symlinks per HTML tool + 5 symlinks per zddc-server platform → the new version. Updates zddc/internal/apps/embedded/* to stable-labeled bytes, makes a release commit, tags all seven (<tool>-v<X.Y.Z>) at that commit so binaries built from the tag embed clean stable bytes. Cascade: stable cut means beta and alpha both reset to stable for every tool.
• Beta (./build beta): Overwrites <tool>_beta.html with dist bytes for each HTML tool, and zddc-server_beta_<platform> with each platform's binary. Updates zddc/internal/apps/embedded/* to beta-labeled bytes (the dev image picks them up via ZDDC_REF=main). Cascade: <tool>_alpha.html → <tool>_beta.html and zddc-server_alpha_<platform> → zddc-server_beta_<platform> (symlinks). No tag.
• Alpha (./build alpha): Overwrites only the alpha mirrors in dist/release-output/, all seven tools. Does NOT update zddc/internal/apps/embedded/ — the project invariant is that alpha is never baked into the binary. No tag, no other side-effects.
• Plain dev builds (./build with no arg): produce tool/dist/<tool>.html for HTML tools and zddc/dist/zddc-server-<platform> binaries; do NOT touch dist/release-output/, the live site, or embedded/. Use it to iterate without affecting deployable state.

Bake-in invariant — what zddc-server's binary embeds via //go:embed from zddc/internal/apps/embedded/:

Image	ZDDC_REF	Embeds
Prod (Dockerfile.prod, BMCD)	stable (latest tag)	Stable-labeled bytes from the tagged release commit
Dev (Dockerfile, devshell)	main	Beta or stable bytes — whatever the last beta/stable cut wrote
Local dev iteration	n/a	Use tool/dist/<tool>.html directly; binary's embedded copy lags

Alpha is never baked in. Active dev work uses the tool's local dist HTML opened directly in a browser; the binary's embedded copy is the "default fallback" served when no .zddc apps: override exists, and only ever holds beta or stable bytes.

On-page {{BUILD_LABEL}} format (HTML tools only — zddc-server's version comes from the binary itself):

• Plain dev: vX.Y.Z-alpha · <full-ts> · <sha>[-dirty] (red), where X.Y.Z is the per-tool next-stable target.
• --release alpha: vX.Y.Z-alpha · <date> · <sha> (red).
• --release beta: vX.Y.Z-beta · <date> · <sha> (red).
• --release [version]: v<X.Y.Z> (black).

After cutting a stable release, git push origin main && git push origin --tags to publish the new version files + symlinks + every per-tool tag in lockstep.

Channel discipline (MUST rules)

The build enforces lockstep mechanically (one command bumps all seven). The rules below are still on you.

1. Stable doesn't regress. No known-broken features that worked in the previous stable. If v0.0.5 ships with a bug, the path forward is v0.0.6 with a fix — never edit a previously-published per-version file in place. Stable per-version files are immutable.
2. Lockstep is the contract. Don't cut a single tool's release without bumping the rest. The HTML tool's standalone --release flag still exists as an escape hatch but emits a tag that immediately drifts out of sync with the others.
3. No backports. Always cut a new stable at a higher version. Users pinned to an old version stay pinned by choice.
4. Alpha and beta are mutable. Document this anywhere you invite users to test them. Pinning a deployment to a channel mirror means it gets rebuilt without notice. For reproducibility, pin to a per-version URL — <tool>_v0.0.5.html or zddc-server_v0.0.5.html.
5. Cascade is automatic. Stable cut → beta + alpha mirrors reset to stable (per-tool HTML AND per-platform zddc-server). Beta cut → alpha → beta. "No active beta" silently shows current stable. No freshen step required after a stable release.
6. Hotfix path. For critical bugs: fix on main, cut a new stable. Tag the commit message fix: or include "hotfix" so intent is visible in git log.
7. Beta soak before promoting (recommended). Give a beta a few days of exposure before cutting the same code as stable. Not enforced; use judgment for trivial changes.

Freshen helper

./freshen-channel <tool> <channel> rebuilds the alpha or beta channel of a tool from its current stable tag — useful when you want a channel to advance to current stable code without doing active dev on it (e.g. after upstream dependency changes). Most of the time you don't need it: the cascade rule (rule 5 above) means a stable cut already resets the downstream channel symlinks. Use this when you specifically want a fresh build with a new on-page label timestamp instead of a symlink.

./freshen-channel archive alpha
./freshen-channel transmittal beta

What it does:

1. Finds the latest <tool>-v* clean stable tag.
2. Creates a temporary git worktree at that tag — does not touch the main worktree's HEAD or working tree.
3. Runs <tool>/build.sh --release <channel> inside the worktree, which overwrites <tool>_<channel>.html with the freshly-built bytes. (Note: this is in the worktree, not on main — you'll need to commit the resulting changes back to main afterward.)
4. Removes the worktree.

The build pipeline used is the one at the tag, not on main. That is intentional (pure reproducibility). If you have made build-system improvements since stable was cut and want the freshen to use them, cut a new stable first.

Install model

No install script. Two paths:

• Local — download a tool .html from https://zddc.varasys.io/releases/ and open it. Done.

• Server (zddc-server) — every tool is //go:embed'd into the binary at compile time (the current-stable build). The server virtually serves them at folder-name-driven paths:

  • archive.html at every directory (multi-project, project, archive, vendor levels)
  • classifier.html in any Incoming/Working/Staging directory and its subtree
  • mdedit.html in any Working directory and its subtree
  • transmittal.html in any Staging directory and its subtree
  • index.html (landing) only at the deployment root

  See internal/apps/availability.go. Outside these locations, requesting <app>.html returns 404 (just like any other missing file).

To override at any level, either:

1. Drop a real <app>.html file at the path → static handler serves it (highest priority).
2. Write an apps: entry in any .zddc along the path. Spec is one of stable/beta/alpha/v0.0.4/v0.0/v0/full URL/local path. Closer-to-leaf entries win.

URL sources fetch once and cache forever in <ZDDC_ROOT>/_app/<host>/<path>. To force a re-fetch, delete the cache file. No background refresh, no SHA-256 verification, no admin UI. If a configured URL fetch fails, the server falls back to the embedded copy and emits a one-time WARN log.

Operators audit by reading the X-ZDDC-Source response header: fetch:URL / cache:URL / path:/abs / embedded:<app>@<build>. Direct URL access to /_app/... is blocked at the dispatch layer.

Runtime mode detection in archive is independent of install: it auto-detects multi-project / project-root / in-archive from ?projects= plus folder shape. The other tools don't care where they live.

Worktrees

Use git worktree to run multiple agents on separate branches simultaneously without filesystem collisions.

• Worktrees live at ~/src/zddc-<branch-name> (sibling of the main clone)
• Before starting work on a feature branch, check git worktree list; if no worktree exists, create one: git worktree add ~/src/zddc-<branch-name> -b <branch-name>
• All edits, builds (./build), and tests (npm test) run from within the worktree directory — build scripts use relative paths so this works correctly
• The dist/ force-commit rule (git add -f) applies per-worktree
• After the branch is merged, clean up: git worktree remove ~/src/zddc-<branch-name> then delete the branch
• Never run git checkout or git switch inside a worktree that another agent may be using

Transmittal-specific

• Two-phase hydration: populateStatic() before publish, hydrate() on load of published file
• Reactive state via Proxy — app.state.mode = 'view' auto-notifies subscribers
• Runtime CDN loads (jszip, docx-preview, xlsx) are allowed only for the optional DOCX/XLSX preview; core features work offline
• Published payload stored in <script id="transmittal-data" type="application/json">

mdedit-specific

• css/tailwind-utils.css is a pre-generated static subset (~80 classes). Add new Tailwind classes here; do not re-run Tailwind.
• Toast UI Editor v3.2.2 is bundled in vendor/; template.html loads it from CDN for dev convenience
• </ escaping is essential: sed 's#</#<\\/#g' runs on both app JS and vendor JS at build time

Form-data system (form/ + zddc-server form handler)

A schema-driven form renderer used to collect structured data into YAML files in the file tree. The form tool (form/) is the renderer; the server-side endpoints live in zddc/internal/handler/formhandler.go; the validator is zddc/internal/jsonschema/.

Form spec: <name>.form.yaml — top-level envelope is {title, description, schema, ui, mode}. schema is JSON Schema 2020-12 (subset; see "Validator subset" below). ui is RJSF-style (ui:widget, ui:order, ui:autofocus, ui:placeholder, ui:help, ui:readonly, ui:options.{addable,removable}). LLMs author this dialect well.

URL conventions (form posts back to its own URL; server strips .html). The spec lives inside the rows-dir alongside the row YAMLs, so the whole form (spec + every submission) is a single self-contained directory:

• GET /<dir>/form.html — render empty form
• POST /<dir>/form.html — create new submission → 201 + Location capability URL pointing at the new <dir>/<id>.yaml
• GET /<dir>/<id>.yaml.html — render form pre-filled from <id>.yaml
• POST /<dir>/<id>.yaml.html — overwrite that submission → 200

Storage: spec at <dir>/form.yaml, submissions at <dir>/<YYYY-MM-DD>-<email-sanitized>.yaml (siblings of the spec). Copying <dir> elsewhere copies the spec plus every submission together. ACL applies via the existing .zddc cascade.

Round-trip: v0 is form-as-truth — submission YAML is regenerated from form state on every save; comments in submissions are not preserved. File-as-truth mode (lossless YAML round-trip via the eemeli/yaml Document API) is a v1 feature, needed for hand-edited files like .zddc itself.

Validator subset (zddc/internal/jsonschema/): type (string/number/integer/boolean/array/object), enum, minimum, maximum, minLength, maxLength, required, additionalProperties: false, properties, items, format (date, email). NOT supported in v0: $ref, $defs, if/then/else, oneOf/anyOf/allOf, conditional visibility. The form-spec meta-schema enforces that authors stay in the supported subset.

Renderer subset (form/js/): types listed above, enum (select / ui:widget: radio), format: date|email, textarea, nested objects, arrays of primitives, arrays of objects with add/remove rows. ui:show-when and reorder are v1.

Adding a new form: create a directory <dir>/ and drop form.yaml into it (per .zddc ACL). No code change required. Visit <dir>/form.html.

Tables system (tables/ + zddc-server table handler)

Read/aggregate counterpart to the form system. Renders a directory of YAML row files as a sortable, filterable table; each row clicks through to its <id>.yaml.html form editor. The tables tool (tables/) is the renderer; the server-side recognizer is zddc/internal/handler/tablehandler.go RecognizeTableRequest.

Discovery is presence-based, the same convention as forms: a <dir>/table.yaml on disk auto-mounts at <dir>/table.html. The directory is the table.

Storage (self-contained directory):

<dir>/
  table.yaml         ← spec
  form.yaml          ← row-edit form (paired with table.yaml)
  <id>.yaml ...      ← rows

table.yaml and form.yaml are excluded from the rows list. Each row is also a form submission — the same files the form system reads — so the table view and the per-row form editor are two views of one folder of YAMLs. Copying <dir>/ elsewhere copies the entire table (spec + form + every row) — that's the whole point of the in-dir layout.

One table per directory by construction (the spec is the singleton table.yaml). No .zddc reference needed; presence-based discovery is the entire rule. To make a directory a table, drop a table.yaml in it — that's it.

Subfolders inside a table dir are allowed and silently ignored as rows. The rows iterator filters non-.yaml entries, so directories don't show up in the table view. Legitimate subfolder use cases:

• Nested sub-tables — <dir>/sub-list/table.yaml is its own self-contained table at <dir>/sub-list/table.html. Composition, not violation.
• Per-row attachments — <dir>/<id>.attachments/file.pdf. Natural sidecar pattern; the row YAML can reference its attachments by relative path.
• Drafts / staging — <dir>/.drafts/<id>.yaml (dot-prefix → hidden from listings as well as from the table).
• Future per-row history — <dir>/.history/<id>/<timestamp>.yaml if/when version sidecars are added.

Default-MDL fallback at archive/<party>/mdl/: when no table.yaml (or form.yaml) exists on disk in this exact location, the server serves embedded default bytes. The mdl/ directory itself doesn't even need to exist — the URL renders the default MDL view fully virtually so a fresh archive surfaces the master document list without operator setup. Outside archive/<party>/mdl/, presence-based discovery is the rule.

Adding a new table: create a directory <dir>/ and drop table.yaml (and optionally form.yaml for row editing) into it. No code change required. Visit <dir>/table.html.

Implementation-vs-dependency policy

Match implementation cost to actual surface used. Reimplement focused subsets when a dep's surface area is much larger than what we consume; adopt for genuinely large specs (YAML parsing, etc.) where reimplementing is foolish. Examples in this codebase:

• zddc/internal/jsonschema/ — focused 2020-12 validator (~300 LoC) covering only the v0 form-spec subset. A full library (e.g. santhosh-tekuri/jsonschema/v6) brings 70%+ surface we don't use.
• gopkg.in/yaml.v3 — adopted as a dep. Reimplementing YAML is foolish.

This is a guideline, not a rule. Revisit per-feature: when v1+ form-spec adds $ref + oneOf + if/then/else, the validator's "savings" evaporate and adopting becomes cheaper.

zddc-server

Go HTTP server sub-project living at zddc/. Replaces caddy file-server --browse for ZDDC archives.

Build

zddc-server ships as a cross-compiled binary, not a container image. There's no Containerfile or compose file in this repo (the chart Dockerfiles compile from source at deploy time at the right tag).

# Compile a local binary for the host platform via the build image.
# Same flag pattern as Test below — see that subsection for why.
podman run --rm --network=host -v "$PWD":/src:Z -v /tmp/gocache:/root/go/pkg/mod:Z \
  -w /src/zddc -e GOPROXY=https://proxy.golang.org -e GOSUMDB=off -e GOPRIVATE= \
  localhost/zddc-go:1.24 go build -o zddc-server ./cmd/zddc-server

# `go run` is normally a one-liner but in-container `go run` of a network
# server is awkward — for dev iteration, build with the line above and
# launch the binary on the host (`./zddc/zddc-server`).

The repo's top-level ./build cross-compiles the four release binaries (linux/amd64, darwin/amd64, darwin/arm64, windows/amd64) into zddc/dist/ via a containerized Go toolchain (podman or docker). On ./build alpha|beta|release it also promotes those binaries to dist/release-output/ with the matching symlink chain and stub pages — same lockstep flow as the HTML tools. ./deploy rsyncs the bundle to /srv/zddc/releases/.

Test

Go is not installed on the dev shell host directly — run go test (and go build) through a golang-alpine image. ./build's containerized cross-compile already pulls golang:1.24-alpine as a build stage; tag it for reuse:

# One-time: locate the golang-alpine image (~810 MB, untagged after a build run)
# and give it a stable name. The size and `golang/.../go test` lines distinguish
# it from the small ~18 MB zddc-server runtime image.
podman images --filter dangling=true --format '{{.Size}}\t{{.ID}}' | sort -h | tail
podman tag <id> localhost/zddc-go:1.24

# If you have no <none> 810 MB image (fresh machine), pull directly:
podman pull docker.io/library/golang:1.24-alpine
podman tag docker.io/library/golang:1.24-alpine localhost/zddc-go:1.24

Canonical invocation:

podman run --rm --network=host \
  -v "$PWD":/src:Z \
  -v /tmp/gocache:/root/go/pkg/mod:Z \
  -w /src/zddc \
  -e GOPROXY=https://proxy.golang.org \
  -e GOSUMDB=off \
  -e GOPRIVATE= \
  localhost/zddc-go:1.24 \
  go test ./...

Why each flag:

• --network=host — the alpine image's TLS chain can't shake hands with gopkg.in directly (sandbox hits "Connection reset by peer"); host networking + the proxy below works around it.
• GOPROXY=https://proxy.golang.org — fetch via the public proxy. Without this, the build-image's baked GOPRIVATE=* forces direct VCS, which fails on gopkg.in/natefinch/lumberjack.v2.
• GOSUMDB=off — sum.golang.org isn't reachable from the sandbox either; we already trust the proxy.
• GOPRIVATE= (empty) — explicit override of the image's GOPRIVATE=*, which is a leftover from how ./build does in-container compilation and would otherwise re-trigger direct fetch.
• /tmp/gocache mount — persistent module cache across runs.

Run-it-once-per-session pattern: alias it. Do not apt install golang on the host — the image is the source of truth for the version pin, so dev and CI compile against the same Go.

Run (development)

ZDDC_ROOT=/path/to/your/archive ZDDC_TLS_CERT=none ZDDC_ADDR=:8080 \
  go run ./cmd/zddc-server

For a release binary downloaded from zddc.varasys.io/releases/:

curl -O https://zddc.varasys.io/releases/zddc-server_stable_linux-amd64
chmod +x zddc-server_stable_linux-amd64
ZDDC_ROOT=/path/to/your/archive ZDDC_TLS_CERT=none ZDDC_ADDR=:8080 \
  ./zddc-server_stable_linux-amd64

Key environment variables

Variable	Default	Purpose
ZDDC_ROOT	(required)	Path to served file tree
ZDDC_ADDR	:8443	Bind address
ZDDC_EMAIL_HEADER	X-Auth-Request-Email	Header set by upstream proxy with user email (oauth2-proxy / nginx auth-request convention)
ZDDC_INDEX_PATH	.archive	Virtual archive index URL segment
ZDDC_LOG_LEVEL	info	Logging verbosity
ZDDC_CORS_ORIGIN	(empty)	Comma-separated CORS allowlist; empty (default) disables CORS — appropriate for embedded-tools deployments where tools and data are same-origin. Set explicitly only for self-hosted tools at a different host (e.g. https://tools.acme.com) or the CDN-bootstrap pattern (https://zddc.varasys.io).
ZDDC_INSECURE	(empty)	Must be 1 to allow startup with no <ZDDC_ROOT>/.zddc. Without it, the server refuses to start because no .zddc files anywhere → public-by-default. Set only for deliberately-public archives.
ZDDC_NO_AUTH	(empty)	1 skips ACL enforcement entirely on this instance. On a master: anyone reads everything (dev / trusted-LAN read-only deployments). On a downstream proxy/cache/mirror: trust upstream's filtering, don't re-evaluate ACLs locally. Distinct from ZDDC_INSECURE (which gates a startup safety check).
ZDDC_UPSTREAM	(empty)	Master URL (https://master.example.com). When set, the binary runs as a client (downstream proxy/cache/mirror) instead of a master — the master-side machinery (archive index, apps server, watcher, OPA, ACL middleware, token store) is replaced by the cache layer in zddc/internal/cache/. --root becomes the cache directory. Setting this also downgrades the --addr default to 127.0.0.1:8443 (loopback) — the cache forwards a bearer to upstream without authenticating the local caller, so non-loopback binds with ZDDC_BEARER_FILE set are refused unless ZDDC_INSECURE_DIRECT=1 is also set.
ZDDC_MODE	cache	Client mode: proxy (forward live, no persistence), cache (default; persist responses on access), mirror (phase 3 — currently behaves like cache). Ignored when ZDDC_UPSTREAM is empty.
ZDDC_BEARER_FILE	(empty)	Path to a 0600 file containing the master-issued token (see /.tokens on the master). Forwarded as Authorization: Bearer … to upstream on every request. Ignored when ZDDC_UPSTREAM is empty.
ZDDC_SKIP_TLS_VERIFY	(empty)	1 accepts self-signed / untrusted upstream certs. Distinct from ZDDC_NO_AUTH. Dev / internal-CA scenarios only.
ZDDC_MIRROR_SUBTREE	(empty)	Comma-separated URL subtrees the access-triggered mirror walker keeps current (e.g. /Vendors/Acme,/Public). Empty + ZDDC_MODE=mirror = full mirror (/). Ignored when ZDDC_MODE != mirror.
ZDDC_MIRROR_MIN_INTERVAL	1h	Minimum gap between walks of the same mirror subtree. Idle subtrees generate zero upstream traffic until next access. Format is Go time.ParseDuration.
ZDDC_OPA_URL	internal	Policy decider endpoint. internal (default) = in-process Go evaluator (same .zddc cascade we always had). http(s)://... or unix:///... = external OPA — every access decision becomes a POST /v1/data/zddc/access/allow to the configured endpoint. Federal customers with their own audited Rego use this; commercial deployments leave it internal.
ZDDC_OPA_FAIL_OPEN	(empty)	External OPA only. 1 = allow on transport error; default = fail closed (deny).
ZDDC_OPA_CACHE_TTL	1s	External OPA only. Per-decision cache TTL — amortizes round-trips on bursty patterns (e.g. .archive listings hit the same (email, dir) tuple many times). 0 disables. Format is Go time.ParseDuration.
ZDDC_APPS_PUBKEY	(empty)	Path to PEM Ed25519 pubkey for verifying signatures on URL-fetched apps: artifacts. Empty = URL apps refused. Download from zddc.varasys.io/pubkey.pem (canonical channels) or supply your own. No baked-in default — same posture as TLS certs. Alternative inline form: apps_pubkey: in root .zddc (root-only, env/flag wins).
ZDDC_ACCESS_LOG	<ZDDC_ROOT>/.zddc.d/logs/access-<host>.log	JSON-line audit log (lumberjack-rotated, 100 MB / 10 backups / 90 days, gzipped). Server auto-mkdirs the parent. Set explicitly to empty (--access-log=) to disable. Per-host filename + host field in every record so multi-replica deployments writing to the same .zddc.d/ dir disambiguate cleanly.

URL handling

URLs are case-insensitive. The dispatcher canonicalizes r.URL.Path against on-disk casing before any handler runs (zddc/internal/fs/resolve.go ResolveCanonical). Per segment: lowercase variant wins if it exists on disk; otherwise exact-case wins; otherwise readdir+CI scan with the lowercase variant winning the tiebreak when multiple case variants are siblings on disk. Walk stops at the first segment that doesn't exist so virtual prefixes (.archive, .profile, .tokens, .api, .auth) and 404 paths flow through with their tail preserved verbatim.

File and folder names preserve case on disk. The canonicalization is purely a URL→filesystem-name mapping; nothing renames anything. Lowercase is the project-wide canonical convention, and auto-created folders in internal/zddc/ensure.go (working/, staging/, archive/<party>/incoming/) and the server's own state dirs (_app/, .zddc.d/tokens/, .zddc.d/outbox/, .zddc.d/logs/) are all lowercase by string literal. Operators can drop a Mixed-Case-Folder/ and it stays mixed-case.

Audit log captures the as-typed path. AccessLogMiddleware snapshots r.URL.Path before dispatch rewrites it; the audit record's path field is what the client sent. When canonicalization changed it, a resolved_path field is added.

Client mode (proxy / cache / mirror)

When --upstream <url> is set, the binary runs as a downstream client of another zddc-server instead of a master. cmd/zddc-server/main.go short-circuits to runClient(cfg), which builds a *cache.Cache from zddc/internal/cache/ and uses it as the entire request handler — no archive index, no apps server, no watcher, no OPA decider, no ACL middleware, no token store.

Three modes via --mode <proxy|cache|mirror> (default cache). Cache directory layout is intentionally a normal ZDDC root: <master>/foo/bar.txt → <root>/foo/bar.txt. Unset --upstream and the same root serves as a plain master, useful for portable offline snapshots.

Pipeline:

• Cache hit → serve immediately + background If-Modified-Since revalidate (304 no-op, 200 overwrite, 403/404 purge).
• Cache miss → forward to upstream; stream response simultaneously to client and a tmp-file atomically renamed into the cache.
• Network error + cached version → serve stale + X-ZDDC-Cache: offline.
• Network error + no cache → 503 + X-ZDDC-Cache: offline.
• Directory listings cached as <dir>/.zddc-listing.<html|json> sidecars (Accept-varied).
• Cache-Control: no-store / private responses pass through but are not persisted.
• Writes (PUT / POST / DELETE) forward to upstream when online; on transport error, queue in <root>/.zddc-outbox/<id>/ (meta + body) and return 202 Accepted + X-ZDDC-Cache: queued. Background loop replays in order — 2xx deletes the entry, 412 → <id>.conflict-<ts>/, 4xx-other drops, 5xx defers. PUT/DELETE include If-Unmodified-Since from the cached mtime so the master can reject conflicting writes.
• Mirror mode (--mode mirror): adds an access-triggered subtree walker (rate-limited via --mirror-min-interval, default 1h) that recursively pre-fetches under --mirror-subtrees; idle mirrors generate zero upstream traffic.

Two-instance smoke test recipe:

# Master.
mkdir -p /tmp/m && echo 'admins: [you@example.com]' > /tmp/m/.zddc
echo "hello" > /tmp/m/hello.txt
zddc-server --root /tmp/m --addr 127.0.0.1:18443 --tls-cert=none --no-auth &

# Client (cache mode).
mkdir -p /tmp/c
zddc-server --root /tmp/c --addr 127.0.0.1:18444 --tls-cert=none \
  --upstream http://127.0.0.1:18443 --mode cache --no-auth &

curl -sI http://127.0.0.1:18444/hello.txt | grep -i x-zddc-cache  # → miss
curl -sI http://127.0.0.1:18444/hello.txt | grep -i x-zddc-cache  # → hit
ls /tmp/c                                                          # → hello.txt + .zddc-upstream marker
kill %1; sleep 1
curl -sI http://127.0.0.1:18444/hello.txt | grep -i x-zddc-cache  # → hit (still served from disk)
curl -si http://127.0.0.1:18444/never.txt | head -1                # → 503

X-ZDDC-Cache response header values: miss, hit, proxy (no-persist or directory), offline (network unreachable). Useful for browser-side freshness UI.

Implementation: zddc/internal/cache/cache.go (a single file). Tests in zddc/internal/cache/cache_test.go use httptest.NewServer as a fake upstream and cover hit/miss/offline/range/bearer-forwarding/no-store paths.

Bearer tokens (CLI auth)

zddc-server self-issues bearer tokens for CLI / non-browser callers. No external IDP, no JWKS rotation. Source of truth: <ZDDC_ROOT>/.zddc.d/tokens/<sha256-hex> — a YAML file per token with email, created, expires, description. Filename is the hash of the token; the plaintext is never persisted.

User flow: sign in to the master in a browser, visit /.tokens, click "Create token," copy the value (shown once). Store in a 0600 file and pass --bearer-file <path> to a CLI that calls back into zddc-server, or send Authorization: Bearer <token> directly from scripts.

Endpoints:

• GET /.tokens — HTML self-service page (gated by browser auth).
• GET/POST /.api/tokens — list / create. Plaintext returned only on POST response.
• DELETE /.api/tokens/<id> — revoke. <id> is the 8-char short ID or full hash.

Validation flow inside the request path: ACLMiddleware checks for Authorization: Bearer … first; on success, sets the request email from the token file. On any failure (unknown / expired / store unavailable), returns 401 — there is no fallback to header-based auth, so a misconfigured client can't silently masquerade as anonymous. If no Bearer is present, the existing cfg.EmailHeader path runs unchanged.

The tokens directory inherits the existing .zddc.d/ exclusion: dot-prefix segments 404 from direct GETs, and fs.ListDirectory filters them from listings. Verify on any new deployment by attempting GET /.zddc.d/tokens/anything and confirming 404.

Implementation: zddc/internal/auth/ (storage), zddc/internal/handler/tokenhandler.go (HTTP layer), middleware extension in zddc/internal/handler/middleware.go.

Release tagging

zddc-server has no separate release script. The top-level ./build alpha|beta|release [version] is the canonical path: it cross-compiles the binaries inside the containerized Go toolchain, copies them into dist/release-output/ with the lockstep symlink chain (one set of symlinks per platform), regenerates the per-version + per-channel stub pages, refreshes the index, and (on stable) tags zddc-server-v<X.Y.Z> alongside the six HTML-tool tags.

./build release            # lockstep stable, coordinated next version
./build release 1.2.0      # lockstep stable, explicit version
./build alpha              # lockstep alpha cut for everything
./build beta               # lockstep beta cut for everything
./deploy --releases        # publish the bundle to /srv/zddc/releases/

The script tags every tool but does NOT push — finish with git push origin main && git push origin --tags (and run ./deploy to put the artifacts on the live site).

Versioning — clean semver. Stable cuts emit one <tool>-vX.Y.Z tag per tool, all seven sharing the same X.Y.Z. No -alpha.N / -beta.N counter tags — channel URLs are stable URLs by design. Historical per-tool independent tags (archive-v0.0.2, zddc-server-v0.0.7, etc.) stay as artifacts; the next coordinated cut jumps every tool to the same number.

Binary distribution — /srv/zddc/releases/zddc-server_<X>_<platform> (on the deploy host) are real static files served from zddc.varasys.io/releases/. No Codeberg release assets, no $CODEBERG_TOKEN, no third-party mirror, no LFS. The matrix-cell link points at zddc-server_<X>.html, a generated stub page that surfaces the four platform downloads in one click.

There is no CI for this — solo workflow benefits from one canonical local path that fails loudly and visibly on the developer's terminal.

Notes

• No external test framework yet — Go unit tests run with go test ./... inside zddc/. The Go toolchain is not on the host; use the localhost/zddc-go:1.24 image as documented in the Test subsection above.
• Portfolio files (*.portfolio) in the served tree appear as virtual group directories
• Every folder under a project exposes a .archive virtual directory backed by that project's index bucket — the project is the first slash-separated segment of the contextPath. Depth within a project doesn't change scope: /ProjectA/sub/sub/.archive/X.html resolves the same as /ProjectA/.archive/X.html, just with a different URL prefix on the listing entries. The flat listing emits two entries per tracking number: <tracking>.html (highest base rev) and <tracking>_<rev>.html (each specific base rev). Both serve in place — the handler streams the first chronologically received copy's bytes back at the .archive/ URL without redirecting. The per-transmittal URL is intentionally hidden so external links of the form .archive/<tracking>.html#section keep tracking the latest revision (a redirect would expose the snapshot URL and people would forward THAT instead). Cache-Control is no-cache so each load revalidates against the on-disk file's Last-Modified/ETag; when a new revision lands the resolver picks it and the browser refetches. Modifier files (<tracking>_<rev>+C1.html etc.) remain reachable via the resolver but are not surfaced in the listing — they're return traffic, not primary documents. /.archive/ at the very root has no project segment and returns 404 — stable references must include the project directory. Within one project, two different files claiming the same (tracking, rev) are an authoring mistake; chronological winner still wins, but a WARN is emitted with both paths. ACL is enforced twice: the listing endpoint is gated by the contextPath's .zddc chain, and each entry is then filtered against the ACL of its resolved file's directory — per-target denials return 404 rather than 403 to avoid leaking that the tracking number exists in another subtree.
• ACL is enforced via cascading .zddc YAML files — first-explicit-match-wins evaluated bottom-up (deepest level first), with deny checked before allow within a single .zddc; default-deny when any .zddc exists in the chain. Authentication is delegated to the upstream proxy via the X-Auth-Request-Email header (configurable with ZDDC_EMAIL_HEADER). Operator-facing detail, anti-patterns, worked layouts, the verify-it-works recipe, and the federal-readiness gap analysis are in zddc/README.md § "Access control: the .zddc cascade." The architectural framing (cooperating layers, commercial-vs-federal trust model, why archive auto-serves at every directory) is in ARCHITECTURE.md § "Server security model."
• .zddc schema also supports a top-level admins: glob list, peer to acl.allow/acl.deny. Honored only at the root .zddc (subdir admins entries are ignored to prevent privilege escalation via subtree write access). Drives the built-in debug dashboard at /.admin/ (sub-routes: /whoami, /config, /logs); non-admin requests get 404 so the page is invisible. See zddc/README.md § "Admin Debug Page".
• GET /.auth/admin is a forward_auth target for upstream proxies — returns 200 if the request's X-Auth-Request-Email is in the root .zddc admins: list, 403 otherwise. No body, no UI. Used by the dev-shell pod's Caddy to gate /devshell/* (code-server) on root-admin status without code-server learning about auth. zddc-server's own routes use the regular .zddc cascade ACL — they do NOT go through this endpoint.
• Reserved entry prefixes under ZDDC_ROOT: .-prefixed entries are excluded from listings AND 404 on direct fetch (only .archive and .admin are exempt) — for invisible side-state like dev-shell home dirs. _-prefixed entries are excluded from listings only — for operator scaffolding like the _template/ directory created by the self-contained install snippet, still reachable by direct URL. Drop side-state under _ if it should be linkable; under . if it should be unreachable.
• Caching on embedded tool HTMLs (landing, browse served at /, plus the five canonical app HTMLs at <dir>/<app>.html): Cache-Control: public, max-age=0, must-revalidate + content-addressed ETag (sha256 hex prefix). Browser revalidates on every load; matching ETag returns 304 Not Modified with empty body. ETag changes only when the binary is redeployed (computed once at startup from EmbeddedBytes + BuildVer, memoized).
• Compression: gzip middleware (github.com/klauspost/compress/gzhttp) wraps the entire mux. Skipped for bodies under 1 KB and for 304 responses. Roughly 75% size reduction on tool HTMLs and JSON listings.
• Public landing page: GET / (HTML or JSON) bypasses the directory-level ACL gate so anonymous callers see the project picker. Per-project filtering inside fs.ListDirectory still hides projects the caller can't reach. Subdirectory ACL gates remain in force.
• Audit log: every request is mirrored to a JSON-line file under <ZDDC_ROOT>/.zddc.d/logs/access-<host>.log (configurable via --access-log / ZDDC_ACCESS_LOG, opt out with empty). Lumberjack rotation (100 MB / 10 backups / 90 days, gzip). Hostname is in both the filename and every record's host field — multi-replica deployments sharing one .zddc.d/ dir disambiguate cleanly.
• HTTP timeouts: ReadHeaderTimeout: 10s, ReadTimeout: 60s, WriteTimeout: 60s, IdleTimeout: 120s. Slowloris-resistant; legit traffic completes in milliseconds even with gzip.