New endpoint GET /<path>/foo.md?convert=docx|html|pdf renders a markdown
source on demand. Surfaced as the Download buttons in browse's markdown
editor (separate commit).
Execution model — two upstream container images, lazy-pulled:
• docker.io/pandoc/latex:latest — MD→DOCX, MD→HTML (entrypoint pandoc)
• docker.io/zenika/alpine-chrome — HTML→PDF (entrypoint chromium-browser)
No custom image build. The runner passes --pull=missing on every podman/
docker invocation so the operator only needs the runtime installed —
first request pulls the image, subsequent requests use the local cache.
Overrides: --convert-pandoc-image / --convert-chromium-image (and the
matching ZDDC_CONVERT_* env vars). Engine: --convert-engine (podman
preferred, docker fallback). Resource caps: --convert-mem-mib (512),
--convert-cpus (2), --convert-pids (100), --convert-timeout (30s).
PDF flow is two-stage: pandoc renders the markdown through the embedded
viewer-template.html to standalone HTML, then chromium prints that HTML
via --print-to-pdf. Preserves the print-media CSS already authored in
viewer-template.html rather than going through pandoc's LaTeX template.
Each conversion runs in a throw-away container with --rm --network=none
--read-only --tmpfs=/tmp --cap-drop=ALL --security-opt=no-new-privileges
--env=HOME=/tmp plus a bind-mounted scratch dir for I/O. Pandoc reads
markdown from stdin / writes to stdout; the viewer template lives at
/tpl (ro). Chromium reads HTML from a read-write bind mount at /pdf
and writes the PDF to the same mount; the host reads it back. No shell
wrappers, no shell quoting — argv flows straight into each image's
entrypoint.
On-disk cache at <dir>/.converted/<base>.<ext> with mtime synced to the
source. Fast path is a stat-and-serve with no exec; slow path
singleflights concurrent requests for the same target. PUT/DELETE/MOVE
on the source .md purges the .converted/ sidecars.
Per-project template variables (client/project/contractor/project_number)
come from a new .zddc `convert:` cascade block, walked leaf→root with
per-key latest-wins. Filename-derived variables (title, tracking_number,
revision, status, is_draft) come from a new zddc.ParseFilename helper.
If neither podman nor docker is on PATH, the endpoint serves 503 with
a clear Retry-After. The rest of the server keeps working.
This is the first os/exec site in the codebase. The hardening in
internal/convert/runner.go — context.CancelFunc → process kill,
cmd.WaitDelay, platform-specific SysProcAttr (Setpgid + Pdeathsig on
Linux), minimal env, stdout cap via limitWriter, stderr ring buffer —
sets the pattern for any future shell-outs.
Public surface:
convert.ToDocx(ctx, source, meta) / .ToHTML / .ToPDF
convert.Probe(ctx, engineOverride) → install Runner if engine present
convert.SetImages(pandoc, chromium)
convert.ConfigureLimits(memMiB, cpus, pids, timeout)
convert.Available()
Container handler at internal/handler/converthandler.go; dispatcher
branch in cmd/zddc-server/main.go inserts the convert lookup after the
existing ACL gate, reusing the source file's read policy verbatim.
A focused security review of phases 1-4 surfaced one MEDIUM finding
(confidence 9/10): in client mode (--upstream set) the cache layer
forwards the configured bearer to upstream on every incoming request
without authenticating the local caller, AND --addr defaulted to
:8443 (all interfaces). Together those mean a CLI user running
`zddc-server --upstream https://master --bearer-file ~/token` on a
laptop on hotel/cafe Wi-Fi exposes an open-proxy confused-deputy:
any attacker on the same L2 connects to https://<laptop-ip>:8443,
accepts the self-signed cert, issues GETs (or PUTs/DELETEs that
queue in the outbox), and the cache laundries each request through
upstream with the engineer's bearer. The full cached subtree leaks.
Two layers of defense in config.Load:
1. Loopback default in client mode. When cfg.Upstream is set and
neither --addr nor ZDDC_ADDR was passed explicitly, --addr
downgrades to "127.0.0.1:8443" (vs ":8443" in master mode). CLI
users on a laptop get safe-by-default. Operators who want a
non-loopback bind opt in explicitly.
2. Refuse non-loopback bind + bearer-file without acknowledgement.
When cfg.Upstream is set, BearerFile is non-empty, the chosen
addr is non-loopback, AND --insecure-direct is not set, the load
fails with an error that names the bind, the threat (open-proxy
confused-deputy laundering bearer credentials), and the
acknowledgement flag. The helm zddc-server-cache/ chart already
sets ZDDC_INSECURE_DIRECT=1 and relies on Kubernetes-namespaced
pod networking for the gating, so the chart path is unaffected.
The guard is bearer-file-conditional because proxy mode without a
bearer doesn't have a credential to launder, and refusing it
would needlessly block proxy-without-auth deployments.
Tests in internal/config/config_test.go lock down all four cases:
- --upstream with no explicit --addr → 127.0.0.1:8443
- --upstream + non-loopback --addr + --bearer-file (no IDirect) → refuse
- --upstream + non-loopback --addr + --bearer-file + --insecure-direct → ok
- --upstream + non-loopback --addr + NO bearer → ok (no credential to leak)
Doc updates: zddc/README.md client-mode "Flags" section gets a
WARNING block describing the loopback default + insecure-direct
escape hatch. AGENTS.md ZDDC_UPSTREAM row mentions the addr
downgrade. ARCHITECTURE.md gains a "Confused-deputy guard at
startup" subsection under "Master + proxy/cache/mirror" with the
two-layer defense rationale. helm/zddc-server-cache/values.yaml.example
adds an inline note next to addr: ":8080" explaining why the chart
sets ZDDC_INSECURE_DIRECT=1 and what the consequence is of removing
either side of the gating.
Master mode is unaffected — the client-mode validation block is
gated by `if cfg.Upstream != ""`. All existing tests pass.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
--mode mirror layers an access-triggered walker on top of the cache
pipeline. When an incoming request's URL falls under one of the
configured --mirror-subtree paths, the scheduler kicks off a recursive
walk of that subtree iff (a) no walk for that subtree is in flight and
(b) now - last_walk_at >= --mirror-min-interval (default 1h). Walks
run in a goroutine; the user's request never blocks on scheduling.
Why access-triggered: a naive "walk on a fixed timer" would produce
thundering-herd polls on a master from many vendor mirrors most of
which are idle most of the time. Demand-triggering means idle mirrors
generate zero upstream traffic until someone hits them; active
mirrors stay current as a side effect of normal use.
The walk:
1. Recursively fetches JSON listings under the subtree, persisting
each at <dir>/.zddc-listing.json so directory browsing works
offline for walked subtrees.
2. For each file, fires a conditional If-Modified-Since GET (bounded
parallelism; default 4 concurrent) — 304 no-op, 200 overwrites,
403/404 purges the local cache.
3. After enumeration, per-directory orphan purge: local files absent
from upstream's filtered listing are removed (handles upstream
deletes + ACL revocations).
State persists at <root>/.zddc-mirror-state.json as
{subtrees: {<path>: {last_walk_at}}}. In-flight tracking is in-memory
only — a crash mid-walk lets the next access retry without manual
cleanup. Subtree path matching is longest-prefix-wins; "/" is a
catch-all (full mirror, the default when --mode=mirror is set without
explicit --mirror-subtree).
The cache layer also gained directory-listing caching (independent of
mirror mode but enabled by it). Directories are now stored at
<dir>/.zddc-listing.<html|json> sidecars, varied by Accept header.
Hit/miss/offline semantics mirror the file pipeline. Phase 2's
limitation that directories always proxied live (no offline browse)
is now resolved for any directory the user has visited or that mirror
mode has walked.
Mirror scope falls out of auth: the walker uses the local instance's
bearer, so it sees exactly what the user can see at upstream. Admin
bearer → full mirror; vendor bearer → vendor's permitted subtree;
no code distinguishes the cases.
New flags (also as ZDDC_* env vars), ignored when --mode != mirror:
- --mirror-subtree <csv> — repeatable subtrees (comma-separated);
empty + --mode=mirror = "/" (full mirror)
- --mirror-min-interval <duration> — default 1h
Tests (15 new in walker_test.go, 3 new in cache_test.go): subtree
normalization, longest-prefix matching, root-as-catch-all, walk
fetches all files in scope, out-of-scope URLs are no-op, rate-
limiting prevents double-walks within min-interval, walks re-fire
after interval elapses, orphan purge removes local-only files,
state file survives restart, concurrent triggers don't double-walk,
end-to-end ServeHTTP-kicks-mirror-on-access, listing format varies
by Accept, listing offline serves stale, persisted state atomic
write + corrupt-input handling. Full suite + go vet clean.
Doc updates: zddc/README.md flags table gains the two new entries
plus a "Mirror mode (access-triggered subtree walker)" subsection
with trigger semantics and properties; the "What client mode is NOT,
yet" list shrinks accordingly. AGENTS.md env-var table gains the
two new entries. ARCHITECTURE.md "Master + proxy/cache/mirror"
section now documents the walker scheduler / walk algorithm / state
file in a "Mirror walker (access-triggered)" subsection.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
zddc-server can now run as a downstream client of another zddc-server.
Set --upstream <url> and the master-side machinery (archive index, apps
server, watcher, OPA decider, ACL middleware, token store) is bypassed
entirely; cmd/zddc-server/main.go short-circuits to runClient(cfg)
which uses zddc/internal/cache/Cache as the entire request handler.
Three modes via --mode <proxy|cache|mirror>:
- proxy: forward upstream live, no disk persistence
- cache (default): persist responses on access; subsequent hits serve
from disk + background If-Modified-Since revalidate
- mirror: accepted but currently behaves like cache; the access-
triggered walker lands in phase 3
Cache directory layout is intentionally a normal ZDDC root: a file
fetched from <master>/foo/bar.txt is stored at <root>/foo/bar.txt with
no sidecar metadata. The local file's mtime is set to the upstream's
Last-Modified header so revalidation reflects the master's notion of
file age, not local fetch time. Running zddc-server --root <cache-dir>
without --upstream serves the cached files as a plain master — useful
for portable offline snapshots. A small .zddc-upstream marker is
written once on first persist for provenance.
Pipeline (GET/HEAD only — writes deferred):
- Hit → http.ServeContent serves directly (range-aware, 304-aware) +
background revalidate (304 no-op, 200 overwrite, 403/404 purge)
- Miss → forward to upstream with the configured bearer; tee response
body to client + tmp-file atomically renamed into the cache
- Network error + cached → serve stale + X-ZDDC-Cache: offline
- Network error + no cache → 503 + X-ZDDC-Cache: offline
- Directories always proxy live (no listing cache yet — phase 3)
- Cache-Control: no-store / private and non-200 responses bypass cache
Range requests work end-to-end (Range/If-Range headers forwarded on
miss; http.ServeContent handles them natively on hit). Hop-by-hop
headers per RFC 7230 §6.1 are dropped from forwarded responses.
New flags (also as ZDDC_* env vars), all ignored when --upstream is
empty (so master deployments are untouched):
- --upstream <url>
- --mode proxy|cache|mirror (default cache)
- --bearer-file <path> (0600 file with the master-issued token)
- --skip-tls-verify (separate from --no-auth; for self-signed dev)
Validation: --upstream must be http(s)://...; trailing / is trimmed.
Mode validated to one of the three known values. The startup
no-root-.zddc check is skipped in client mode (the cache directory
starts empty by design). The plain-HTTP-on-non-loopback check is also
skipped (the local instance never reads the email header to decide
anything; auth is forwarded to upstream as a Bearer).
Tests: zddc/internal/cache/cache_test.go runs httptest.NewServer as
the upstream and covers miss-then-hit, proxy-mode-no-persist,
directory-never-cached, HEAD-no-body, offline-with-cache,
offline-no-cache → 503, bearer forwarding, query-string preservation,
no-store bypass, path-traversal rejection, error-status forwarding,
revalidate-on-403/404/200/304, range-on-hit, concurrent-same-URL,
cache-path boundary cases. 23 new tests, full suite + go vet clean.
Live two-instance smoke verified: master at 127.0.0.1:18443, client
at :18444 with --mode cache, miss→hit→hit transitions work, file
materialises under cache root with parent dirs created, marker file
written once, range-on-hit returns 206, master sees background 304s
on every hit, killing master leaves cached files serving from disk
and never-cached files returning 503 + offline header.
Doc updates: zddc/README.md gains a "Client mode" section with the
modes table, flag reference, pipeline summary, two-instance recipe,
and explicit list of phase-2 limitations; AGENTS.md adds the four
new env vars to the reference table and a "Client mode" subsection
with smoke-test recipe and a pointer to the cache package;
ARCHITECTURE.md adds "Master + proxy/cache/mirror" before "Bearer
token issuance," covering the topology, the persist/warm switches,
the cache-IS-a-ZDDC-root invariant, the request pipeline, and the
v1-out-of-scope multi-tenancy note; CLAUDE.md's zddc/ entry
expanded to mention both deployment shapes so future agents pick it
up by default.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
zddc-server now issues its own bearer tokens for non-browser callers
(CLI tools, scripts, downstream proxy/cache/mirror instances). No
external IDP, no JWKS rotation. Self-service flow: sign in via the
browser, visit /.tokens, click "Create token," paste the resulting
plaintext into a 0600 file, and pass --bearer-file <path> to whatever
calls back into the server.
Storage is <ZDDC_ROOT>/.zddc.d/tokens/<sha256-hex>, YAML per token
with email/created/expires/description. Filename is the *hash* of the
plaintext, never the plaintext itself — a leak of the tokens
directory exposes hashes, not credentials. Mode 0600 / 0700, atomic
writes via temp+rename. Already shielded from public serving by the
existing dot-prefix guards in dispatch and fs.ListDirectory.
ACLMiddleware now recognises Authorization: Bearer <token>. On valid
token, sets the request email from the token file and falls through
to the existing ACL chain. On any failure (unknown / expired / store
unavailable / Bearer with no validator), returns 401 — no silent
fallback to anonymous, so a misconfigured client fails loudly.
JSON API at /.api/tokens (GET list, POST create, DELETE /<id> revoke)
backs a small inline HTML self-service page at /.tokens. Users can
only see and revoke their own tokens; cross-user revoke returns 404
to avoid leaking ownership.
--no-auth (ZDDC_NO_AUTH=1) skips ACL enforcement entirely on this
instance. On master: anyone reads everything (dev / trusted-LAN /
public-read deployments). On a downstream proxy/cache/mirror: trust
upstream's filtering, don't re-evaluate ACLs locally. Implemented as
a swap to policy.AllowAllDecider; all existing handlers keep calling
AllowFromChain unchanged. Distinct from --insecure, which only
relaxes the no-root-.zddc startup check. WARN-level startup log when
--no-auth is active so accidental enablement is visible.
33 new tests covering token storage, validation/expiry/revocation,
the JSON API end-to-end, the HTML page, and the middleware-Bearer
integration including the case-insensitive prefix and expired-token
paths. Full suite + go vet clean.
Doc updates: zddc/README.md "Authentication" rewritten to cover both
auth paths and the token UI/API; AGENTS.md gains ZDDC_NO_AUTH and a
"Bearer tokens" subsection flagging the dot-prefix-shielding pre-
condition; ARCHITECTURE.md adds "Bearer token issuance" and
"--no-auth" subsections under "Server security model" with the
hash-as-filename rationale and dispatch-shielding regression-
sensitivity called out; CLAUDE.md adds a one-line summary of the new
auth topology so future agents pick it up by default.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The fsnotify watcher only sees events the local kernel generates, so on
SMB/CIFS-backed roots (Azure Files) writes from any other client are
invisible — the archive index would silently miss them until pod
restart. Add two backstops:
1. Periodic full re-walk via Index.Rebuild on a configurable interval
(--archive-rescan-interval / ZDDC_ARCHIVE_RESCAN_INTERVAL, default
60s, 0 to disable). Atomically swaps ByProject under the existing
RWMutex; concurrent reads stay safe.
2. Admin-only POST /.profile/reindex that triggers an immediate rebuild
and returns {duration_ms, project_count, tracking_count}, for the
"I just dropped 50 files and don't want to wait" case. Gated by
IsAdmin with the same 404-on-non-admin pattern as the other admin
sub-resources.
Tests: TestRebuild_PicksUpAddsAndDrops covers add+drop semantics and
returned counts; TestServeProfileReindexPOST covers the happy admin
path; matrix entries cover the gate (anonymous/non-admin → 404, admin
GET → 405 method-not-allowed since the route is POST-only).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replaces the binary acl.allow/deny model with five permission verbs
(r/w/c/d/a) and first-class roles, and adds an authenticated file API
(PUT/DELETE/POST move/mkdir) so the HTML tools can edit-in-place over
HTTP. Closes the AC-3(7) and AC-6 federal-readiness gaps.
File API (zddc/internal/handler/fileapi.go)
- PUT <new> → action c
- PUT <existing> → action w
- PUT <.zddc> → action a (CanEditZddc strict-ancestor rule)
- DELETE → action d
- POST mkdir → action c (auto-writes creator-owned .zddc when the
parent is Incoming/Working/Staging)
- POST move → action w on src + c on dst, atomic via os.Rename
- Optional If-Match for optimistic concurrency, --max-write-bytes cap,
audit log emits a structured file_write event per operation.
Permission model (zddc/internal/zddc/{acl,file,roles,cascade_mode}.go)
- acl.permissions: { principal → verb-set } map; principals are email
patterns or role names. Empty verb set is an explicit deny.
- roles: { name → members } definitions, available at the level they
declare and all descendants. Closer-to-leaf shadows ancestor.
- Legacy acl.allow/deny still work; they fold into permissions at
parse time (allow → "rwcd", deny → "").
- Cascade walks leaf→root; first level with any matching entry wins;
the union of matching verb sets at that level decides.
- --cascade-mode=strict adds a root→leaf ancestor-deny pre-pass so an
ancestor explicit-deny is absolute (NIST AC-6). Default delegated
preserves the existing commercial behavior.
Special folders (zddc/internal/zddc/special.go)
- Incoming / Working / Staging: mkdir auto-writes a .zddc into the new
subdir granting created_by + that email rwcda directly. Same form
operators write by hand; creator can edit it later to add others.
- Issued / Received: server-enforced WORM split. Cascade grants
inherited from above the WORM folder are masked to r only; grants
placed at-or-below the WORM folder retain r,c. Operators grant
write-once (cr) to the doc controller via an explicit .zddc at the
Issued/Received folder. Admins exempt — only escape hatch.
Browser polyfill (shared/zddc-source.js)
- HttpDirectoryHandle + HttpFileHandle implement the FS Access API
surface (values, getFileHandle, createWritable, removeEntry,
queryPermission/requestPermission) over zddc-server's listing JSON
and file API. Existing tools written against showDirectoryPicker
work unchanged.
- detectServerRoot() returns { handle, status }: tools auto-load on
HTTP, surface a clear "no permission to list" message on 403, and
fall back to the welcome screen on 0.
- classifier renames take the atomic POST move path on HTTP-backed
handles; mdedit and transmittal route reads/writes through the
polyfill so prior FS-API code paths cover both modes.
Tests
- zddc/internal/zddc/{cascade_mode,roles,special,acl}_test.go cover
delegated vs strict, role membership / shadowing / legacy fallback,
WORM split semantics, verb-set parser round-trip.
- zddc/internal/handler/fileapi_test.go now also covers role-based
vendor scenarios, WORM blocking vendor & doc controller writes,
explicit Issued .zddc unlocking the cr drop-box, admin bypass,
auto-ownership on mkdir, and strict-mode lockouts.
Docs
- ARCHITECTURE.md + zddc/README.md document the verb model, role
syntax, special-folder behaviors, cascade-mode flag, and full file
API surface. Federal-readiness gap analysis strikes AC-3(7) and
AC-6.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two interlocking pieces shipped together:
1. Strict Ed25519 signature verification on URL-fetched apps artifacts.
Every URL the apps cascade resolves must publish a corresponding
<url>.sig (raw 64-byte Ed25519 signature). The fetcher rejects on
any failure (sig 404, transport error, wrong key, tampered body)
and the resolver falls back to the embedded copy.
The trusted public key is OPERATOR-CONFIGURED via --apps-pubkey /
ZDDC_APPS_PUBKEY (PEM file path). No baked-in default — same posture
as TLS certificates. Operators using zddc.varasys.io's canonical
channels download pubkey.pem from there and configure the local
path. Operators with their own signing infrastructure pass their
own public key.
Build pipeline (./build) gains sign_release_artifacts: walks
dist/release-output/ after promote and produces an Ed25519 .sig
alongside every real file. ZDDC_SIGNING_KEY=~/.config/zddc-signing/
key.pem (mode 0600). Symlinks skip — the .sig at the symlink
target is what counts.
Test coverage: parse-PEM round-trip, malformed/wrong-type PEM
rejection, valid-signature accept, tampered-body reject, wrong-key
reject, malformed-signature reject, end-to-end fetch+sign+verify,
fetch-rejects-tampered, fetch-rejects-missing-sig, fetch-rejects-
wrong-key. Existing fetch tests updated to use signed-fixture
helpers.
2. Dev Helm chart mounts production data READ-ONLY and layers an
OverlayFS writable scratch on top. Prod data is the lowerdir;
dev's writes (form submissions, archive index state, .zddc edits)
land in upperdir; main container sees the merged read-write view
at $ZDDC_ROOT. Setup runs in a privileged init container; main
container runs unprivileged. Solves the dev-replica-on-shared-
dataset problem at the filesystem layer with no zddc-server code
change.
Docs: env-var tables in zddc/README.md and AGENTS.md gain a
ZDDC_APPS_PUBKEY row. The Federal-readiness gap analysis "Code-signed
apps: URL fetches" subsection is rewritten as "what's currently in
place" instead of "what would need to be added," with a forward
pointer to per-entry signed_by: (multi-key) and Sigstore as the
federally-acceptable evolution.
The website "Verify your downloads" section + the embedded pubkey
gone — but the website needs separate updates landing in zddc-website
to publish pubkey.pem and add the verify section. Pending in that
repo's commit.
Production binary unchanged at 13.1 MB. All 11 Go test packages green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Phase 2 enhancements to the policy decider, plus listing-level ETags
that benefit every deployment regardless of decider mode.
Reference Rego policy
---------------------
internal/policy/rego/access.rego mirrors InternalDecider's semantics
exactly — bottom-up walk, deny-first within a level, default-deny when
HasAnyFile=true, glob matching with @-boundary semantics (special-cased
bare "*" because OPA's glob.match treats empty delimiters
inconsistently for that pattern).
Embedded into the binary via go:embed; --print-rego dumps it to stdout
so federal customers standing up an external OPA can use it as a
parity-tested baseline:
zddc-server --print-rego > /etc/opa/policies/zddc-access.rego
Parity test runner
------------------
parity_test.go imports the OPA Go module as a TEST-ONLY dependency
(github.com/open-policy-agent/opa@v0.70.0). Every fixture from the
internal Go evaluator's test set runs through both implementations;
any divergence fails CI. The test-only import means production
binaries (built by `go build ./cmd/zddc-server`) stay OPA-free —
release-flag binary size unchanged at ~13 MB.
The parity test caught a real bug on first run: bare "*" patterns
didn't match through OPA's glob.match with empty delimiters. Fixed
in access.rego with a special-case rule. This is exactly the kind of
subtle drift the parity guard exists to catch.
External-mode decision cache
----------------------------
HTTPDecider is now wrapped in a cachingDecider with a default 1s TTL.
Bursty patterns like .archive listings (one OPA round-trip per entry
before, one per (email, decision-input) tuple per TTL window after)
amortize cleanly. Verified: 20 identical /D/ requests produce 1 OPA
hit with cache, 40 hits without (each listing makes 2 ACL queries).
ZDDC_OPA_CACHE_TTL knob (default 1s) lets operators tune. 0 disables.
1s matches the fsnotify watcher debounce window — staleness is
bounded the same way other policy-edit propagation already is.
Internal mode unchanged; the in-process Go evaluator is already
cheaper than a cache lookup would be.
Listing ETags
-------------
GET / (project list) and GET /<dir>/ (directory listing JSON) now
carry content-hash ETag + Cache-Control: private, max-age=0,
must-revalidate. SHA-256 of the rendered JSON, truncated to 16 hex
chars (64 bits — collision risk on a listing of any realistic size
is vanishingly small).
Server-side caching deliberately not added: it would require
mtime-based invalidation, and Azure Files SMB mounts (a common
deployment substrate) don't support fsnotify reliably. The
content-hash ETag delivers the bandwidth savings (304 on identical
fetches) without depending on watcher correctness — the hash is the
actual response, so it can't lie about staleness regardless of
underlying watcher behavior.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add an internal access-decision boundary that all handlers go through
instead of calling zddc.AllowedWithChain directly. Two implementations
ship:
* InternalDecider — wraps the existing zddc.AllowedWithChain. The
default. No new dependencies, identical semantics to the legacy
code path. ZDDC_OPA_URL=internal (or unset).
* HTTPDecider — POSTs the canonical OPA wire format
(POST /v1/data/zddc/access/allow with {"input": {...}}, response
{"result": true|false}) over HTTP, HTTPS, or a Unix-domain socket.
For federal customers running their own audited Rego policies
alongside zddc-server. ZDDC_OPA_URL=http(s)://… or unix:///….
External-mode failure semantics: unreachable / non-2xx / malformed
response → fail closed (deny) by default with a WARN log. Operators
who prefer availability over correctness flip with ZDDC_OPA_FAIL_OPEN=1.
The decider is constructed once at startup, plumbed through ACLMiddleware
into the request context. Handlers retrieve it via DeciderFromContext;
non-request callers (fs.ListDirectory, EnumerateProjects, enumerateAccess)
take it as an explicit parameter.
zddc.ZddcFile and zddc.ACLRules gain JSON tags so external Rego authors
get idiomatic input shape (acl.allow, admins, …) instead of Go field
names (ACL.Allow, Admins, …).
Test coverage:
* InternalDecider parity tests against zddc.AllowedWithChain (every
documented cascade scenario: empty chain, leaf-allow-wins, leaf-
deny-beats-parent, leaf-allows-what-parent-denies, deepest-match-
wins, etc.)
* HTTPDecider happy-path test (canonical wire format)
* Fail-closed / fail-open / malformed-response tests
Production binary size unchanged (no new deps; HTTP transport is
stdlib net/http). 11 ACL call sites migrated. End-to-end verified
against the worked-example layout in zddc/README.md.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two safe-by-default flips, both opt-out via explicit acknowledgement.
1. --insecure / ZDDC_INSECURE=1: zddc-server now refuses to start when
no <ZDDC_ROOT>/.zddc exists. With no .zddc anywhere in the chain,
AllowedWithChain falls through to "HasAnyFile=false → allow" and
the tree is publicly accessible to anonymous callers — almost never
what an operator wants on a fresh deployment, and previously a
silent footgun. The flag is the escape hatch for deliberately-
public archives (no .zddc anywhere by design).
2. ZDDC_CORS_ORIGIN now defaults to empty (CORS disabled) instead of
the canonical "https://zddc.varasys.io". The embedded-tools install
path serves tools and data same-origin, so the default never needed
to permit cross-origin XHRs from a third-party host. Every deployment
was implicitly trusting zddc.varasys.io to make authenticated XHRs
on behalf of every logged-in user; if that origin were ever
compromised, the blast radius extended to every customer server.
Operators who deliberately use the CDN-bootstrap pattern or self-
hosted tools at a different host now set the value explicitly.
Helm chart values updated accordingly: prod default is empty; dev
keeps localhost:8000 for tool-iteration workflows. Existing deployments
that depended on the old defaults will need to either set the value
explicitly or pass --insecure.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two related operational improvements:
1. HTTP timeouts on http.Server (ReadHeaderTimeout 10s, ReadTimeout +
WriteTimeout 60s, IdleTimeout 120s). Caps slow-client connection
hold time; closes the slowloris vector. Listing + tool-HTML
responses complete in milliseconds even with gzip, so 60s is
generous for legit traffic.
2. --access-log defaults to <ZDDC_ROOT>/.zddc.d/logs/access-<host>.log
instead of stderr-only. The server auto-creates the parent tree
(mode 0750), so a fresh deployment gets an audit trail without
operator setup. Every JSON record carries a `host` field (from
os.Hostname) — multi-replica deployments share the .zddc.d/logs/
directory but write to per-host filenames, and downstream
aggregators can disambiguate via the host field.
Opt-out: --access-log= (explicit empty). Distinguishing "unset"
from "set to empty" follows the same pattern config.go already
uses for --cors-origin.
Live verification:
$ zddc-server -root /tmp/r -addr 127.0.0.1:8765 -tls-cert none -insecure-direct
$ curl http://127.0.0.1:8765/
$ ls /tmp/r/.zddc.d/logs/
access-bizon.log
$ tail -1 /tmp/r/.zddc.d/logs/access-bizon.log
{"time":...,"level":"INFO","msg":"access","host":"bizon",...,"email":"anonymous","method":"GET","path":"/","status":200,...}
$ zddc-server -root /tmp/r ... -access-log= # opt-out
$ ls /tmp/r/.zddc.d/ # empty: no logs/ created
Add --access-log <path> (env ZDDC_ACCESS_LOG). When set, every access-
log record is written as a JSON line to the configured file in
addition to the existing slog.Default() stderr output. Empty (default)
keeps the prior behavior — stderr only.
Rotation via gopkg.in/natefinch/lumberjack.v2:
100 MB per file, 10 backups, 90-day max age, gzip rotated files.
Operator usage (e.g. behind a Caddy/quadlet stack):
zddc-server --access-log /srv/.zddc.d/logs/access.log ...
Architecture:
AccessLogMiddleware now takes an optional *slog.Logger. main.go wires
it via setupAccessAuditLog() which builds a slog.JSONHandler over a
lumberjack rotator. Stderr emission stays via slog.Default(); the
audit logger gets the same fields in line-delimited JSON, the format
every standard log shipper (Vector, Loki, fluentbit, journalbeat)
parses natively.
Tests cover the audit logger receiving the same email/path/status
fields as the stderr stream.
Adds command-line flags to zddc-server alongside the existing env vars.
Each setting can be set via --<flag-name> or ZDDC_<NAME>; the flag wins
on conflict, the env var wins over the hard-coded default.
--root / ZDDC_ROOT (now defaults to CWD if both unset)
--addr / ZDDC_ADDR (:8443)
--tls-cert / ZDDC_TLS_CERT ("none" / empty / path)
--tls-key / ZDDC_TLS_KEY
--log-level / ZDDC_LOG_LEVEL (info)
--index-path / ZDDC_INDEX_PATH (.archive)
--email-header / ZDDC_EMAIL_HEADER (X-Auth-Request-Email)
--cors-origin / ZDDC_CORS_ORIGIN (https://zddc.varasys.io; "" disables)
--insecure-direct / ZDDC_INSECURE_DIRECT (false)
--help (prints flag list to stderr, exits 0)
--version (prints binary + embedded tool versions, exits 0)
So an operator can `cd /srv/zddc && zddc-server` with zero config — the
served root defaults to the current directory, and TLS defaults to a
self-signed cert. config.Load now takes []string (test-friendly: nil
skips flag parsing entirely; tests pass an empty slice for env-only
loads).
Adds a `version` package-level var in main.go injected at link time via
`-ldflags="-X main.version=..."`. The build.sh runs git describe against
zddc-server-v* tags; for in-flight commits between releases it produces
e.g. zddc-server-v0.0.7-19-gadb6904-dirty.
Adds an embedded versions manifest:
- Each tool's compute_build_label (in shared/build-lib.sh) writes a
sidecar <tool>.label to $BUILD_LABELS_DIR if that env var is set.
- Top-level build.sh sets BUILD_LABELS_DIR before running each tool's
build, then assembles zddc/internal/apps/embedded/versions.txt as
one `<app>=<build label>` line per app.
- apps.EmbeddedVersions() loads the manifest at runtime.
- main.go logs a compact summary on every startup; --version dumps
the full per-app label.
Removes the old cfg.BuildVersion field — the X-ZDDC-Source: embedded
header now uses the package-level main.version directly.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds internal/apps/ package serving the five tool HTMLs at virtual paths
based on the surrounding folder name convention:
archive every directory (multi-project, project, archive, vendor)
classifier any Incoming/Working/Staging directory and subtree
mdedit any Working directory and subtree
transmittal any Staging directory and subtree
landing only at deployment root
The current-stable build of every tool is //go:embed'd into the binary
at compile time — that's the default with zero config. Operators
override per-directory via .zddc apps: entries; closer-to-leaf wins.
Spec syntax (in any apps: value):
stable / beta / alpha / :stable channel
v0.0.4 / v0.0 / v0 / :v0.0.4 version
https://my-mirror/releases URL prefix only
https://my-mirror/releases:beta URL prefix + channel
https://my-fork/archive.html terminal full URL
./local.html / /abs/path.html terminal local path
The special apps.default key provides a baseline URL prefix and channel
inherited by any app not overridden per-name. Per-axis cascade: a deeper
.zddc can override the URL, the channel, or both.
Cascade walks root→leaf; default applies first at each level, then the
per-app entry. Terminal sources (paths and full .html URLs) short-circuit
composition; deeper non-terminal entries override parent terminals.
URL sources fetch once on first request and cache forever in
<ZDDC_ROOT>/_app/<host>/<path> — different upstreams with the same
filename stay distinct. No background refresh, no SHA-256 verification:
operators delete the cache file to force a refetch. Concurrent misses
for the same source dedupe via a 30-line hand-rolled singleflight.
Per-request override: any user can append ?v=<spec> to a tool URL
(e.g. ?v=beta, ?v=v0.0.4, ?v=:alpha, ?v=https://mirror/releases:beta)
to ask for a different build for one request. Security: ?v= serves
ONLY versions already in the cache (cache miss returns 404; path
sources are rejected outright with 400). Users cannot trigger
arbitrary upstream fetches via crafted URLs.
Failed URL fetches (network down, 5xx) fall back to embedded with a
one-time WARN log. The X-ZDDC-Source response header reports what
served: fetch:URL / cache:URL / path:/abs / embedded:<app>@<build>.
Wire-in (cmd/zddc-server/main.go): dispatch routes <dir>/<app>.html
through apps.MatchAppHTML + AppAvailableAt + apps.Server.Serve when
no real file exists. Direct URL access to /_app/... is blocked at
the dispatch layer — cached files must go through the apps resolver
so they get correct Content-Type and ACL gating.
Schema (internal/zddc/file.go): ZddcFile gains Apps map[string]string
for cascade overrides. Validator (internal/zddc/validate.go) accepts
the special "default" key alongside the five canonical app names and
all spec forms.
Removes ZDDC_APPS_* env vars (no admin UI, no refresh interval, no
upstream allow-list — the simpler model has fewer knobs).
40+ unit tests across the new package: parser shapes, cascade
resolution with default+per-app interactions, terminal short-circuit
semantics, ?v= cache-only enforcement, embedded fallback, atomic
cache writes, singleflight dedup. Plus end-to-end dispatch tests in
cmd/zddc-server/main_test.go.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three improvements bundled because they all ship as zddc-server v0.0.2:
* /.admin/ debug dashboard with /whoami, /config, /logs sub-routes.
Authorization via a top-level `admins:` glob list in <ZDDC_ROOT>/.zddc
(root-only — subdir entries deliberately ignored to prevent privilege
escalation via subtree write access). Non-admin requests get 404 so the
page is invisible. Recent logs surface via a 500-entry slog ring buffer
teed off the existing TextHandler. Lets operators debug without
kubectl exec.
* Default ZDDC_EMAIL_HEADER changes from `X-Email` to
`X-Auth-Request-Email` — the oauth2-proxy / nginx auth-request
convention that the TND helm chart already sets explicitly.
Operators who set the env var explicitly are unaffected; deployments
relying on the previous default need to set ZDDC_EMAIL_HEADER=X-Email
or update their proxy.
* dispatch() rejects any URL whose segments contain a dot prefix other
than the recognized virtual prefixes (.admin, cfg.IndexPath /
.archive). Matches the existing listing-pipeline filter so hidden
subtrees on the served PVC (e.g. /srv/.devshell — used by the
in-cluster dev-shell for persistent home-dir state) become
unreachable via direct HTTP fetch, not just hidden in listings.
Refreshes the X-Email reference in website/index.html accordingly.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
ZDDC — Zero Day Document Control. A file-naming convention plus five
single-file HTML tools (archive, transmittal, classifier, mdedit,
landing) and an optional Go HTTP server (zddc-server) with ACL and a
virtual archive index. Self-contained, offline-capable, dependency-free.
See README.md for an overview, AGENTS.md and ARCHITECTURE.md for the
build/release/architecture detail, bootstrap/README.md for the
two-level deployment install pattern, and zddc/README.md for the
HTTP server.
