Three docs aligned with the preceding three feature commits.
zddc/README.md
--------------
Major overhaul of the access-control narrative. The previous "three-
tier" example table was misleading: it claimed a project-level
allow-list "restricts" access under a parent wildcard, when actually
the cascade is additive (a non-team employee falls up to root and
matches *@company.com). Operators reading the old docs would build
deployments that looked locked-down but leaked across the company.
New sections under "Access control: the .zddc cascade":
* Step 1: starter .zddc — leads with the public-by-default warning
and the --insecure escape hatch
* How a request is evaluated — bottom-up walk with code citations
* Glob patterns — @-boundary rule
* When the cascade helps and when it fights you — the asymmetry
between adding strangers (easy) and excluding insiders (hard)
* Pick your layout — decision matrix for common shapes
* Worked example: paired open/closed projects + third-party archive
— full layout with trace table for two representative users
* Patterns that look secure but aren't — anti-patterns including
same-level allow+deny shadow, leaf-allow-doesn't-restrict,
apps:-as-UI-mount
* Trust model and invariants — auth boundary, subtree authority,
root-only escalation gate
* Trust boundary — network isolation requirement, anonymous
information disclosure on /, audit-log integrity
* Debugging permissions — manual cascade trace
* Directory visibility / Reserved hidden segments
* How to verify in 5 minutes — recipe with negative anti-pattern test
* Federal-readiness gap analysis — bulleted with NIST control refs
* External policy decider — OPA wire format, deployment shapes,
failure modes
* OPA decision cache — TTL semantics, knobs
* Reference Rego policy — --print-rego, parity test rationale
* Caching and ETags — content-hash story, why not server-side
* Future work
Plus env-var table updates for ZDDC_INSECURE, ZDDC_OPA_URL,
ZDDC_OPA_FAIL_OPEN, ZDDC_OPA_CACHE_TTL; CORS narrative reflects
default-empty.
ARCHITECTURE.md
---------------
New "Server security model" section between Form Renderer and CSS:
cooperating layers (auth / policy decider / cascade / tool-rooted
view / reserved prefixes / audit log), commercial-vs-federal trust
model side-by-side, why the tool-rooted view matters for third-party
containment.
AGENTS.md
---------
Two new env-var rows (ZDDC_OPA_URL, ZDDC_OPA_CACHE_TTL); ACL line
sharpened with cascade rules + cross-reference; ZDDC_CORS_ORIGIN
description updated for default-empty.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Phase 2 enhancements to the policy decider, plus listing-level ETags
that benefit every deployment regardless of decider mode.
Reference Rego policy
---------------------
internal/policy/rego/access.rego mirrors InternalDecider's semantics
exactly — bottom-up walk, deny-first within a level, default-deny when
HasAnyFile=true, glob matching with @-boundary semantics (special-cased
bare "*" because OPA's glob.match treats empty delimiters
inconsistently for that pattern).
Embedded into the binary via go:embed; --print-rego dumps it to stdout
so federal customers standing up an external OPA can use it as a
parity-tested baseline:
zddc-server --print-rego > /etc/opa/policies/zddc-access.rego
Parity test runner
------------------
parity_test.go imports the OPA Go module as a TEST-ONLY dependency
(github.com/open-policy-agent/opa@v0.70.0). Every fixture from the
internal Go evaluator's test set runs through both implementations;
any divergence fails CI. The test-only import means production
binaries (built by `go build ./cmd/zddc-server`) stay OPA-free —
release-flag binary size unchanged at ~13 MB.
The parity test caught a real bug on first run: bare "*" patterns
didn't match through OPA's glob.match with empty delimiters. Fixed
in access.rego with a special-case rule. This is exactly the kind of
subtle drift the parity guard exists to catch.
External-mode decision cache
----------------------------
HTTPDecider is now wrapped in a cachingDecider with a default 1s TTL.
Bursty patterns like .archive listings (one OPA round-trip per entry
before, one per (email, decision-input) tuple per TTL window after)
amortize cleanly. Verified: 20 identical /D/ requests produce 1 OPA
hit with cache, 40 hits without (each listing makes 2 ACL queries).
ZDDC_OPA_CACHE_TTL knob (default 1s) lets operators tune. 0 disables.
1s matches the fsnotify watcher debounce window — staleness is
bounded the same way other policy-edit propagation already is.
Internal mode unchanged; the in-process Go evaluator is already
cheaper than a cache lookup would be.
Listing ETags
-------------
GET / (project list) and GET /<dir>/ (directory listing JSON) now
carry content-hash ETag + Cache-Control: private, max-age=0,
must-revalidate. SHA-256 of the rendered JSON, truncated to 16 hex
chars (64 bits — collision risk on a listing of any realistic size
is vanishingly small).
Server-side caching deliberately not added: it would require
mtime-based invalidation, and Azure Files SMB mounts (a common
deployment substrate) don't support fsnotify reliably. The
content-hash ETag delivers the bandwidth savings (304 on identical
fetches) without depending on watcher correctness — the hash is the
actual response, so it can't lie about staleness regardless of
underlying watcher behavior.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add an internal access-decision boundary that all handlers go through
instead of calling zddc.AllowedWithChain directly. Two implementations
ship:
* InternalDecider — wraps the existing zddc.AllowedWithChain. The
default. No new dependencies, identical semantics to the legacy
code path. ZDDC_OPA_URL=internal (or unset).
* HTTPDecider — POSTs the canonical OPA wire format
(POST /v1/data/zddc/access/allow with {"input": {...}}, response
{"result": true|false}) over HTTP, HTTPS, or a Unix-domain socket.
For federal customers running their own audited Rego policies
alongside zddc-server. ZDDC_OPA_URL=http(s)://… or unix:///….
External-mode failure semantics: unreachable / non-2xx / malformed
response → fail closed (deny) by default with a WARN log. Operators
who prefer availability over correctness flip with ZDDC_OPA_FAIL_OPEN=1.
The decider is constructed once at startup, plumbed through ACLMiddleware
into the request context. Handlers retrieve it via DeciderFromContext;
non-request callers (fs.ListDirectory, EnumerateProjects, enumerateAccess)
take it as an explicit parameter.
zddc.ZddcFile and zddc.ACLRules gain JSON tags so external Rego authors
get idiomatic input shape (acl.allow, admins, …) instead of Go field
names (ACL.Allow, Admins, …).
Test coverage:
* InternalDecider parity tests against zddc.AllowedWithChain (every
documented cascade scenario: empty chain, leaf-allow-wins, leaf-
deny-beats-parent, leaf-allows-what-parent-denies, deepest-match-
wins, etc.)
* HTTPDecider happy-path test (canonical wire format)
* Fail-closed / fail-open / malformed-response tests
Production binary size unchanged (no new deps; HTTP transport is
stdlib net/http). 11 ACL call sites migrated. End-to-end verified
against the worked-example layout in zddc/README.md.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two safe-by-default flips, both opt-out via explicit acknowledgement.
1. --insecure / ZDDC_INSECURE=1: zddc-server now refuses to start when
no <ZDDC_ROOT>/.zddc exists. With no .zddc anywhere in the chain,
AllowedWithChain falls through to "HasAnyFile=false → allow" and
the tree is publicly accessible to anonymous callers — almost never
what an operator wants on a fresh deployment, and previously a
silent footgun. The flag is the escape hatch for deliberately-
public archives (no .zddc anywhere by design).
2. ZDDC_CORS_ORIGIN now defaults to empty (CORS disabled) instead of
the canonical "https://zddc.varasys.io". The embedded-tools install
path serves tools and data same-origin, so the default never needed
to permit cross-origin XHRs from a third-party host. Every deployment
was implicitly trusting zddc.varasys.io to make authenticated XHRs
on behalf of every logged-in user; if that origin were ever
compromised, the blast radius extended to every customer server.
Operators who deliberately use the CDN-bootstrap pattern or self-
hosted tools at a different host now set the value explicitly.
Helm chart values updated accordingly: prod default is empty; dev
keeps localhost:8000 for tool-iteration workflows. Existing deployments
that depended on the old defaults will need to either set the value
explicitly or pass --insecure.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Forgejo's uploadpack.allowAnySHA1InWant only matches FULL SHAs —
the 7-char short SHA from build-label / versions.txt produces
"couldn't find remote ref ae75855" on `git fetch --depth=1 origin
<short-sha>` (and `git clone --branch <short-sha>` fails the same
way). The chart's downstream Dockerfile uses fetch-by-ref to handle
SHAs as well as named refs, but only full SHAs go through.
Resolve the short SHA to its full form via `git rev-parse` in
notify-chart-bump.sh before writing the chart's appVersion. The
runner has the full git history (actions/checkout@v4 fetch-depth: 0),
so rev-parse works locally; the chart's appVersion becomes the
canonical 40-char SHA, and the BMCD pipeline-dev / pipeline-prod
fetches succeed cleanly.
Manually re-bumped chart develop after this commit:
appVersion: 0.0.16-beta-ae75855 → 0.0.16-beta-ae758550a855f6a9507df08075475cb87cb67086
The recent history rewrite (squash of 4 thrash CI commits) made the
chart's previous appVersion (0.0.16-beta-8df0def) reference a now-
dangling commit. The dev pipeline failed clone "remote ref not
found" until we re-bumped to a SHA in the new history.
Re-cut beta with the new HEAD parent (ae75855) so notify-chart-dev
rewrites the chart's appVersion to a SHA the BMCD dev pipeline can
actually fetch. Combined with the Dockerfile clone-via-fetch fix in
tnd-zddc-chart 86c5758 (handles bare SHAs), the dev pipeline should
build cleanly.
Chart version was monotonically incrementing PATCH on every chart-bump
— after ~50 betas + a few stable cuts, version was already at 0.2.12.
Triple digits would land within a year of active dev.
Switch the stable-cut branch of notify-chart-bump.sh to bump MINOR
and reset PATCH to 0:
stable: 0.2.X → 0.3.0
beta: 0.3.0 → 0.3.1, 0.3.1 → 0.3.2, ...
next stable: 0.3.X → 0.4.0
Patches stay bounded (≈ betas-per-stable, not all-time). MINOR keeps
incrementing so JFrog chart-repo uploads stay accepted (it rejects
duplicate chart-version numbers — a literal "reset to 0.2.0" cycle
would break uploads on the second stable cut).
Chart version is purely JFrog packaging metadata; the zddc-server
version users actually care about lives in appVersion.
mdedit was the only tool whose dist/<tool>.html was force-tracked
(via `git add -f` in the build's stable-cut path). Inconsistent with
every other tool in the repo, where dist/ is fully gitignored. The
build regenerates mdedit/dist/mdedit.html the same way it regenerates
the others, so there's no reason to track it.
Drop the `git add -f` line in build:735 and `git rm --cached` the
file. The on-disk artifact stays put for the dev iteration loop;
only the index entry goes away.
Two related operational improvements:
1. HTTP timeouts on http.Server (ReadHeaderTimeout 10s, ReadTimeout +
WriteTimeout 60s, IdleTimeout 120s). Caps slow-client connection
hold time; closes the slowloris vector. Listing + tool-HTML
responses complete in milliseconds even with gzip, so 60s is
generous for legit traffic.
2. --access-log defaults to <ZDDC_ROOT>/.zddc.d/logs/access-<host>.log
instead of stderr-only. The server auto-creates the parent tree
(mode 0750), so a fresh deployment gets an audit trail without
operator setup. Every JSON record carries a `host` field (from
os.Hostname) — multi-replica deployments share the .zddc.d/logs/
directory but write to per-host filenames, and downstream
aggregators can disambiguate via the host field.
Opt-out: --access-log= (explicit empty). Distinguishing "unset"
from "set to empty" follows the same pattern config.go already
uses for --cors-origin.
Live verification:
$ zddc-server -root /tmp/r -addr 127.0.0.1:8765 -tls-cert none -insecure-direct
$ curl http://127.0.0.1:8765/
$ ls /tmp/r/.zddc.d/logs/
access-bizon.log
$ tail -1 /tmp/r/.zddc.d/logs/access-bizon.log
{"time":...,"level":"INFO","msg":"access","host":"bizon",...,"email":"anonymous","method":"GET","path":"/","status":200,...}
$ zddc-server -root /tmp/r ... -access-log= # opt-out
$ ls /tmp/r/.zddc.d/ # empty: no logs/ created
./build beta runs locally at HEAD=N, generates embed/*.html with
build-label SHA=N, then the operator commits the generated bytes
as commit N+1. After push, git HEAD on the runner = N+1 but the
served website's BUILD_LABEL still encodes N (it's baked into the
HTML at build time, before the commit).
Previously the bump script computed BETA_VERSION using
`git rev-parse --short HEAD` = N+1, so the chart's appVersion
(and the dev image's tag, and the kubelet's pull) said N+1 while
the served label said N. Two SHAs for the same bytes — confusing
when triaging "is this image actually the latest?".
Read the SHA from zddc/internal/apps/embedded/versions.txt instead
(third pipe-delimited field of any line). That's the single source
of truth for what bytes were baked, and it lines up with what users
see in every tool's header.
Manually re-bumped chart develop after committing this script:
appVersion: 0.0.16-beta-9a3e4d8 → 0.0.16-beta-8df0def
Bake into the dev binary:
- ETag + max-age=0 on embedded HTML (304s on repeat loads)
- gzip compression middleware (~75% wire-size reduction)
- vendored jszip + docx-preview in archive/transmittal/classifier
- tee'd file-based access log via --access-log
Add --access-log <path> (env ZDDC_ACCESS_LOG). When set, every access-
log record is written as a JSON line to the configured file in
addition to the existing slog.Default() stderr output. Empty (default)
keeps the prior behavior — stderr only.
Rotation via gopkg.in/natefinch/lumberjack.v2:
100 MB per file, 10 backups, 90-day max age, gzip rotated files.
Operator usage (e.g. behind a Caddy/quadlet stack):
zddc-server --access-log /srv/.zddc.d/logs/access.log ...
Architecture:
AccessLogMiddleware now takes an optional *slog.Logger. main.go wires
it via setupAccessAuditLog() which builds a slog.JSONHandler over a
lumberjack rotator. Stderr emission stays via slog.Default(); the
audit logger gets the same fields in line-delimited JSON, the format
every standard log shipper (Vector, Loki, fluentbit, journalbeat)
parses natively.
Tests cover the audit logger receiving the same email/path/status
fields as the stderr stream.
Same pattern as the browse fix. archive, transmittal, classifier
previously CDN-loaded jszip + docx-preview on first preview of a
.zip / .docx file via shared/preview-lib.js's loadLibrary helper.
That meant each first-preview blocked on a CDN round-trip + parse,
and broke entirely under restrictive networks or CSPs.
Vendor both libs under shared/vendor/ and concat them at the top of
each tool's build, ahead of init.js. window.JSZip + window.docx are
now defined immediately on page load. Drop the redundant loadLibrary
calls (and classifier's stray <script src="cdn..."> tag in the
template, plus archive's bespoke loadJSZip helper in export.js).
xlsx (SheetJS) intentionally stays CDN-loaded — at ~900 KB it's too
large to inline, and only fires on .xlsx preview which is a rarer
path.
Bundle size impact (uncompressed):
archive: 304 KB → 476 KB (+172 KB)
transmittal: 449 KB → 621 KB (+172 KB)
classifier: 252 KB → 424 KB (+172 KB)
With the gzip middleware (~75% reduction on HTML) and ETag-cached
revalidation now in place, the wire-size delta is ~40 KB per tool
on the first load and 0 on every subsequent load until redeploy.
Add github.com/klauspost/compress/gzhttp wrapper around the request
handler. With MinSize(1024), responses ≥ 1 KB get gzip-encoded when
the client advertises Accept-Encoding: gzip; smaller bodies + 304
Not Modified pass through unchanged.
The wrapper auto-appends Vary: Accept-Encoding (compatible with the
existing Vary: Accept on directory.go's content-negotiated path).
Live-tested against zddc-server -root /tmp/empty:
GET / w/ Accept-Encoding: gzip → 20.9 KB compressed (was 80.9 KB
uncompressed). 74% reduction.
Decompresses cleanly back to the original bytes.
Helps every code path that bypasses Caddy: devshell pods, local dev
binaries, tests, anywhere zddc-server is hit directly. Production
behind Caddy already had compression at the proxy layer; this just
makes the Go server self-sufficient.
Tests in cmd/zddc-server/main_test.go cover:
- large body + Accept-Encoding → compressed + Vary header
- small body → not compressed (under MinSize)
- no Accept-Encoding header → plain bytes
The apps subsystem previously sent Cache-Control: public, max-age=300|3600,
must-revalidate but no ETag. With must-revalidate and no validator, the
browser cannot return 304 — it has to refetch the full body once max-age
expires. For mdedit that's 920 KB on every reload after an hour.
Add a content-addressed ETag (sha256 hex prefix, 32 chars) to:
- apps/handler.go's serveBody + serveEmbedded (both paths now emit ETag
+ handle If-None-Match short-circuit to 304)
- handler/directory.go's embedded:browse fallback (mirror behavior so
the bare-directory landing serves the same way)
Drop max-age to 0 with must-revalidate: every page load revalidates,
but a matching ETag returns 304 with empty body. Steady-state cost of
a reload drops from N KB to a few hundred bytes. When the binary is
redeployed, the ETag changes (content hash) and the next request
returns 200 with the new bytes.
Tests in apps/handler_test.go cover both paths:
- TestServer_Embedded_ConditionalGET: full GET, matching INM, stale INM
- TestEmbeddedETag_Stable: same bytes → same ETag, different → different
Live smoke (curl against zddc-server -root /tmp/empty):
GET / → 200, ETag set, body = 80919 bytes (landing.html)
GET / + INM:tag → 304 Not Modified, empty body
GET / and GET /index.html previously enforced the root .zddc's
top-level acl: gate before serving the landing page. On a deployment
where only specific emails are allowed at root, anonymous (and
unauthorized) callers got 403 — they couldn't even see the project
picker that would tell them which projects were available to them.
Make the landing page public:
- cmd/zddc-server: drop the AllowedWithChain gate from the
apps.Serve("landing") branch; drop it from the IsDir branch when
urlPath == "/".
- handler/directory.go: matching bypass for ServeDirectory at the
root path (covers Accept: application/json and the case where a
real /index.html exists on disk).
Per-project ACL is preserved end-to-end:
- fs.ListDirectory continues to filter sub-entries per email, so
anonymous callers see only projects whose .zddc allows them.
- Subdirectory requests still hit the ACL gate.
Regression test in handler/directory_test.go covers all four cases
(anonymous public, anonymous filters out private, admin sees both,
anonymous still 403 on private subdir). Full go test ./... passes.
Bake the standardized headers + archive bugfix + browse refactor
into the dev binary. Triggers notify-chart-dev → bumps tnd-zddc-chart
develop with appVersion=0.0.16-beta-<sha>.
drag-drop.js and the unsupported-browser handler in app.js both
referenced getElementById('app'), but the template's root has
id="appContainer". The mismatch was masked in production because
sourceMode='http' skips dragDrop.init() — only file:// (sourceMode=
'local') tripped over it, throwing "Cannot read properties of null
(reading 'addEventListener')" at app load.
Surfaced while header-standardizing the other tools; fixed by
pointing both callers at #appContainer.
Bring every tool's header in line with archive's pattern:
[logo] [title] [version] [Add Local Directory] [⟳] ............... [◐] [?]
------------- header-left --------------- ----- header-right -
Changes per tool:
* browse: rename "Select Directory" → "Add Local Directory"; add the
red-non-stable wrap to the build label (was missing); add a help
panel + bundle shared/help.js.
* classifier: rename selectDirectoryBtn → addDirectoryBtn,
refreshBtn → refreshHeaderBtn for consistency. Update all JS
callers and welcome-screen copy to the new label.
* mdedit: same id rename. Move the previously-in-pane refresh
button into the header. Stop renaming the dir button to
"Directory: <name>" once a folder is loaded — instead use the
shared btn--subtle variant to de-emphasize while keeping the
standard label.
* transmittal: convert non-standard <div class="app-header"> with
spacer/icons containers to <header class="app-header"> with the
canonical header-left/header-right pair. Move the publish split-
button into header-left (Transmittal-specific primary action).
Remove dead .app-header__spacer/__icons/header-icon-btn CSS now
that nothing references those classes.
* landing, form: add help-btn + help-panel + bundle shared/help.js.
Each panel is tool-specific (project picker docs for landing,
schema-driven form docs for form).
Cross-cutting:
* shared/base.css: promote .btn--subtle from browse/css/tree.css
so any tool with an online mode can de-emphasize Add Local
Directory consistently.
Verified all 7 tools in headless Chromium: header structure correct,
build label red on non-stable cuts, help panel opens + closes via
button + Esc.
Both notify-chart-dev.yml and the notify-chart-prod job in
deploy-release.yml were carrying ~80 lines of inline shell each,
duplicating the clone-bump-push flow. Extract to a single script:
.forgejo/scripts/notify-chart-bump.sh <beta|stable> [VERSION]
Three benefits:
1. **Locally testable**. The script is invocable directly:
CHART_FORGEJO_TOKEN=$FORGEJO_TOKEN \
.forgejo/scripts/notify-chart-bump.sh beta
No more "push to main and watch what the runner does" debug loop.
2. **Manual escape hatch**. When CI is broken, the same script is
how we recover. The 0.0.16-beta-1ddd331 chart bump preceding
this commit was performed via this very script.
3. **Runner-quirk-immune**. The previous three commits chased a
Forgejo runner v12.9.0 phantom-SIGPIPE bug that would only
surface under the runner's `bash -e -o pipefail {0}` wrapper.
A real script with its own `#!/usr/bin/env bash` and explicit
error handling sidesteps the wrapper entirely.
The workflow YAMLs shrink to checkout + run-script. No GITHUB_OUTPUT
plumbing, no inline if/else gates, no shell flag overrides. The
behavior is identical to the prior inline versions.
Bake the latest dev cut of all six tools into zddc/internal/apps/embedded/
so the dev image (built from main) ships the new browse filter UI +
vendored JSZip. Triggers notify-chart-dev which bumps the chart's
develop branch with appVersion=v0.0.16-beta-<sha>.
- Vendor JSZip locally (shared/vendor/jszip.min.js) and bundle into
the browse build instead of CDN-loading. Eliminates the failure
mode where ZIP rows can't expand because the CDN script doesn't
load (CSP, network, etc.). Tool now works fully offline.
- Replace the toolbar filter input + ext multi-select with two
spreadsheet-style auto-filter rows in <thead>:
- 📄 row: file-name filter + extension filter
- 📁 row: folder-name filter
Each input uses shared/zddc-filter syntax (substring/!negate/
^startsWith/$endsWith/regex/| or/space and).
- New visibility model with ancestor-of-match awareness:
- file matches keep their ancestor folders visible (path-to-hit)
- folder match keeps its descendants visible
- filters compose (file ∧ folder ∧ ext) so combinations narrow
Computed model-side; render walks only visible nodes.
- Replace 🏠 emoji breadcrumb-root with an inline outline-stroke SVG
that tints with currentColor.
Cross-tool header inconsistencies cleaned up after the audit
prompted by the browse Phase 2 work:
- landing/template.html: title was 'ZDDC Archive' (a holdover from
when landing WAS the archive). The page is now the project
picker — title shortened to plain 'ZDDC'. Browser tab title
follows: 'ZDDC Archive — Projects' → 'ZDDC — Projects'. Title +
build label wrapped in title-group div for layout consistency
with archive/classifier/mdedit/browse.
- form/template.html: title was bare; same title-group wrapping.
The id='form-title' stays — its content is overwritten at
runtime by form.js based on the form schema's name.
- classifier/template.html: refresh button text 'Refresh' →
'⟳' icon to match archive + browse. Same title attribute, just
smaller visual weight.
Untouched (intentionally):
- archive's button stays 'Add Local Directory' + addDirectoryBtn
id — semantically different from the others (archive
accumulates multiple directories; everyone else operates on
one). The naming reflects that.
- transmittal — different layout entirely (page-header with
sender/receiver logo cells); not a candidate for app-header
standardization.
Bundles Phase 2 polish + the user-requested header/breadcrumb work:
- Breadcrumbs replacing the plain currentPath span. Server mode
renders linkified ancestor segments (each <a> navigates to that
directory; the browser fetches browse.html, the new instance
auto-loads the listing). FS-API mode renders the rootHandle name
as a non-link (no ancestor handles to navigate). Both prefix the
path with a 🏠 root icon. Trailing slash + bold-current segment
match common file-explorer conventions.
- Subdued 'Select Directory' button in server mode. Once browse is
serving a real directory listing, the local-folder switcher is
available but visually quiet (btn--subtle: transparent, muted
color). FS-API mode keeps the primary styling (it's how the user
got there). New btn--subtle CSS class added to browse's tree.css.
A refresh button (⟳) appears next to it in both modes; clicking
it re-fetches the current root listing.
- Header consistency: browse now matches archive's header layout
(refresh + help buttons in addition to theme on the right). Help
is a placeholder for future help dialog wiring.
- File preview popup. Click a file row → opens a popup window with
the file rendered. Plain types (PDF, HTML, image) load in
iframes; TIFF + ZIP listings via shared/preview-lib.js's
renderTiff / renderZipListing helpers; text via <pre>; unknown
types → 'click Download' placeholder. Modifier-click (ctrl/cmd/
shift) and middle-click still open the file in a new tab via the
underlying <a target=_blank>. Single popup window is reused
across multiple file clicks (matches archive's UX).
- ZIP inline expansion. .zip files have a chevron and act like
folders in the tree. First expand fetches the zip bytes
(server URL or FS handle or parent-zip read), parses with JSZip
(auto-loaded from CDN), and synthesizes the entry tree. Nested
directories within the zip lazy-expand on demand by re-walking
the cached entry list at the right path prefix. Click on a
zip-entry file opens the preview popup with bytes read from
JSZip. Recursive expand-all skips zip archives by design — they
can be very large, and explicit click-to-expand is safer.
- Extension multi-select filter. Toolbar now has a <select
multiple> populated with extensions present in the current
view. Filter is OR-of-selected; combined with the name filter
it's AND-of-both. Folders pass through (so expanding a folder
whose name doesn't match the ext filter still shows its file
children that do match).
Three issues from initial v0.0.12 dev/prod testing:
1. Online listings empty.
directory.go was missing Vary: Accept on its responses, so
browser/CDN cached the HTML response (the embedded browse.html)
and served it again when browse's JS later fetched the same URL
with Accept: application/json. JSON parse failed, autoDetect
returned null, empty state showed. Adds Vary: Accept on both
branches and changes browse.html cache-control to no-cache so
deployed updates land immediately.
2. Top-level folder rows tall, shrink as subtree expands.
The .browse-table had flex:1 in a flex column. <table> in flex
doesn't reliably distribute height across rows — with few rows,
each row stretched. Wrap the table in a div with overflow:auto
and drop flex:1 from the table itself.
3. Recursive expand/collapse.
Shift-click (or alt-click) on a folder now expand-all or
collapse-all its subtree. Plain click still toggles just that
folder. Implementation: tree.expandSubtree() walks BFS, loading
each level's children in parallel, re-rendering between levels
so the user sees progress. tree.collapseSubtree() recursively
marks the subtree collapsed (children stay loaded for instant
re-expand).
A new HTML tool — browse — that lists the contents of any directory.
Designed for ZDDC archives but no ZDDC-specific filtering; just a
straight folder browser with expand/collapse, sort, and name filter.
Modes (auto-detected at page load):
- Online: when served by zddc-server at a folder URL, queries
the same URL with Accept: application/json to load the listing
and renders it. Auto-served as the default at any directory
under ZDDC_ROOT without an index.html (replacing the previous
minimal-HTML stub from directory.go).
- Local: 'Select Directory' button uses FileSystemAccessAPI to
pick any folder on disk; works in Chromium-based browsers.
Features (Phase 1 — what's in this commit):
- Tree view with lazy-loaded folders (children fetched on first
expand).
- Sort by name / size / extension / date (column header click).
- Filter by name substring (toolbar input).
- File click opens in a new tab — for server-backed pages,
routes through zddc-server's normal handler so .archive
redirects + apps cascade overrides + ACL all apply.
Phase 2 deferred:
- ZIP files inline expansion (treat archive entries as virtual
children).
- File preview popup (reuse shared/preview-lib.js).
- Extension multi-select filter.
Wiring:
- browse/ added to top-level ./build's per-tool list, embed
block, versions.txt, and the lockstep release commit + tag set.
All seven tools (archive, transmittal, classifier, mdedit,
landing, form, browse) advance together on stable cuts.
- shared/build-lib.sh: browse added to ZDDC_RELEASE_TOOLS and
verify_channel_links's per-tool loop.
- zddc/internal/apps/embed.go: //go:embed browse.html +
EmbeddedBytes("browse") case.
- zddc/internal/apps/availability.go: browse available at every
directory (same as archive).
- zddc/internal/apps/handler.go: MatchAppHTML routes
/<dir>/browse.html → 'browse'.
- zddc/internal/handler/directory.go: when a directory request
arrives with Accept: text/html and no index.html exists,
serve the embedded browse.html bytes (with a JSON-fallback
if the embedded slot is empty during bootstrap).
The chart repo (BMCD/tnd-zddc-chart) is mirrored Forgejo→GitHub
one-way (we set this up so the chart matches the same canonical-
on-Forgejo pattern as the public repos). When notify-chart-prod
and notify-chart-dev pushed directly to GitHub, the bump landed
on GitHub but Forgejo never got it — and the next time Forgejo's
push-mirror ran, it force-overwrote GitHub's bump with Forgejo's
older state. Symptom: prod stuck at v0.0.9 even after auto-bump
appeared to succeed; manual investigation showed Chart.yaml
appVersion was actually still 0.0.10 (the previous manual bump
that DID land on Forgejo).
Fix: clone+push to Forgejo (git.varasys.io/BMCD/tnd-zddc-chart)
instead of GitHub. Forgejo's mirror replicates to GitHub on the
next sync — going through the canonical-Forgejo path keeps both
sides in sync. Uses a new CHART_FORGEJO_TOKEN secret (separate
from CHART_GITHUB_TOKEN, which is no longer needed for these
workflows but kept for any future direct-GitHub use case).
Per the bake-in invariant: when no active beta exists, dev tracks
stable. Previously notify-chart-prod only bumped chart's main, so
a stable cut updated BMCD prod but left dev one cut behind until
the next beta or manual dispatch.
Now the job loops through {main, develop} and pushes the same
appVersion bump to both. Each branch push triggers its own BMCD
pipeline (pipeline-prod for main, pipeline-dev for develop), so
prod + dev both rebuild against the new ZDDC stable in parallel.
notify-chart-dev.yml continues to handle the beta-cut path
(advances develop ahead of main between stable cuts).
User report: opening an .html file with a '../.archive/' hyperlink in
a new tab works (zddc-server intercepts and serves the right file),
but clicking the same link inside the file previewer does nothing.
Two combined causes:
1. The previewer's iframe was loaded from a blob: URL (built from
the file's bytes). Relative URLs in the iframe resolve relative
to the blob URL — '../.archive/X.html' becomes 'blob:.../.archive/
X.html', which is gibberish. The browser never sends a request to
the server, so the .archive interception never fires.
2. sandbox="" disables every iframe capability including popups,
so even <a target=_blank> is silently swallowed.
Fix per tool:
- archive (table.js): for HTML preview, use file.url (the real
server URL) directly when available; fall back to blob only for
File-System-Access-API mode where there's no server to intercept
anyway. Now relative links in archived HTMLs resolve against the
actual server origin and the .archive interception fires as
designed. Sandbox loosens to allow-same-origin + allow-popups +
allow-popups-to-escape-sandbox so resources within the iframe
load and link clicks (default target / target=_blank / middle-
click) work normally. allow-scripts is intentionally NOT set —
archived HTML still cannot run JS in the popup's origin.
- transmittal (files-preview.js) + classifier (preview.js): same
sandbox loosening for consistency. These tools' files are
typically local (FileSystemAccessAPI), so the file.url branch
doesn't apply — relative URLs that depend on a server still
won't resolve in local mode (intrinsic limitation, no server).
Tested behavior preserved:
- PDFs: unchanged (no sandbox, browser's PDF viewer handles).
- Images / docx / xlsx / tiff / zip / text: unchanged.
- HTML in zddc-server-backed archive: relative '../.archive/' links
now navigate the iframe to the correct target file.
Multi-line git commit message bodies broke YAML parsing — pipe blocks
end on unindented lines, so the body lines starting at column 0 were
being interpreted by Forgejo's YAML parser as keys, yielding:
yaml: line 158: could not find expected ':'
Switch to repeated `-m` flags (one per paragraph). Same end result
in git log; valid YAML.
The skip-if-no-source-changes-since-latest-tag check in
promote_release was a relic of per-tool independent versioning.
In the lockstep era it actively breaks CI re-cuts at a tag commit:
- HEAD is at the v0.0.10 release commit
- latest archive-v* tag is archive-v0.0.10 (== HEAD)
- git diff archive-v0.0.10 HEAD = empty
- SKIP archive promote → no archive_v0.0.10.html written
- dist/release-output/ stays at whatever was seeded from
/srv/zddc/releases/ (i.e. v0.0.9 from the previous deploy)
- ./deploy --releases rsyncs that → live site STAYS at v0.0.9
Symptom: tag-triggered Forgejo deploy-release.yml workflow runs
(run 16) reports success but /srv/zddc/releases/archive_stable.html
still points at archive_v0.0.9.html.
Fix: always run _promote_stable for every tool on a stable cut.
The bytes written are deterministic at the same source, so
overwriting an existing per-version file is a no-op on disk —
the actual work the cut performs is advancing the symlink chain
(_v<X.Y>, _v<X>, _stable, _beta, _alpha) to the new version.
Closes the loop on the user-described workflow:
1. Iterate on tools / cut alpha → no chart involvement.
2. `./build beta` → embedded/ commits to ZDDC main →
notify-chart-dev.yml pushes a chart appVersion bump to
burnsmcd/tnd-zddc-chart's develop branch → BMCD pipeline-dev
fires automatically → dev image rebuilt with new beta bytes
baked in.
3. `./build release` → tag pushed → existing deploy-release.yml's
new notify-chart-prod job pushes a chart appVersion bump to
burnsmcd/tnd-zddc-chart's main branch → BMCD pipeline-prod
fires automatically → prod image rebuilt with new stable bytes.
The chart repo IS still committed to (one Chart.yaml line, auto-
generated by either workflow), but no human ever touches it for
routine ZDDC releases. The chart commits are idempotent (skip if
appVersion already at target) and clearly marked as bot-generated.
The truly chart-commit-free version would require either (a)
BMCD's private helm-deploy-latest reusable to accept --set overrides
we'd compute, or (b) bypassing it entirely with our own helm step.
Both are deeper changes than this PR; this is the simplest reliable
solution within the existing reusable.
Auth: a new repo-scoped Forgejo Actions secret CHART_GITHUB_TOKEN
holds the classic GitHub PAT (already provisioned for the
Forgejo→GitHub mirror; same token, repo+workflow scopes,
SAML-SSO authorized for burnsmcd). The bot identity is
'ZDDC Release Bot <noreply@zddc.varasys.io>'.
Tested behavior:
- Workflow files are added by THIS commit. Pushing this commit
does not fire either workflow (notify-chart-prod requires a
tag; notify-chart-dev requires changes under
zddc/internal/apps/embedded/). Safe to land before testing.
- First real test fires on the next ZDDC stable cut or beta cut.
HTML files in the file previewer (archive, transmittal, classifier
popups) were dispatched to the text renderer because 'html'/'htm'
are in shared/preview-lib.js's TEXT_EXTENSIONS (which is shared with
the syntax-highlighting code path). Result: opening an .html file in
preview showed its source as a <pre> block, not the rendered page.
Fix in each tool's popup builder + dispatcher:
- Add 'html' / 'htm' to the iframe branch (alongside pdf), so the
popup ships an <iframe src=blob:...> instead of an empty
#previewContent div. The blob's MIME type from getMimeType()
is already 'text/html', so the browser renders natively.
- Skip the text-render dispatch for html/htm (the iframe is enough).
- Add to the HTML iframe so an arbitrary archived
HTML file cannot run scripts, navigate top, submit forms, or
open popups in the popup-window's origin. PDFs don't need this
since the browser's PDF viewer is sandboxed natively.
classifier/js/preview.js uses a getPreviewType() switch instead of
chained ifs; adds 'html' as its own preview type (checked BEFORE
'text' since html is in TEXT_EXTENSIONS).
mdedit already handled HTML specially (file-tree.js has an isHtml
check); no change there.
TIFF was already rendered via the shared zddc.preview.renderTiff
canvas viewer in all four tools — no change needed for that path.
If TIFF preview appears broken on the live prod server, that's the
v0.0.9-alpha-baked-in image; the fresh stable redeploy fixes it.
Two related fixes to the lockstep release flow + the project invariant
that prod must always run stable bytes (and dev only ever beta-or-stable).
1) tag-after-commit ordering. `./build release X.Y.Z` previously
regenerated zddc/internal/apps/embedded/* with stable labels but
tagged BEFORE folding those changes in. The tag landed on the
source-side commit (alpha-dirty embedded), and the operator was
expected to commit the embedded changes as a follow-up — which got
dropped in practice, leaving prod binaries with alpha-dirty bytes
baked in. (See the v0.0.9 re-anchor in the immediately preceding
commit for the manifestation.)
Refactor:
- _promote_stable / promote_zddc_server in shared/build-lib.sh
no longer call `git tag`. They keep their pre-flight check
(now: tag must be in HEAD's history rather than == HEAD, since
HEAD will advance after the release commit).
- Top-level ./build adds a new "Release commit + tag" block at
the end of stable cuts: stages the regenerated embedded files,
makes a `release: vX.Y.Z lockstep` commit, and tags all seven
artifacts at the new commit. Idempotent — no commit if there
are no changes.
2) bake-in invariant. Plain `./build` and `./build alpha` now
leave zddc/internal/apps/embedded/ untouched — the binary keeps
shipping whatever the last beta or stable cut wrote. `./build
beta` and `./build release` are the only paths that update
embedded bytes. Active dev iteration uses tool/dist/<tool>.html
directly; the binary's embedded copy is the default fallback,
not a workbench.
Verification on this commit:
./build → embedded mtime unchanged, no "M" lines for embedded/
./build alpha → embedded mtime unchanged, no "M" lines for embedded/
Docs updated to match in CLAUDE.md "Things that bite" + AGENTS.md
"Releasing — lockstep" + the leading help text in ./build itself.
The zddc-server-v0.0.9 (and sibling) tags previously pointed at a
commit whose embedded versions.txt + tool HTMLs still carried
alpha-dirty labels — the cut process regenerated these in the
working tree but never folded them into the tagged commit. The
binary built from that tag (used by tnd-zddc-chart's prod
Dockerfile) embedded the alpha labels.
This commit folds the stable-labeled artifacts in. The seven
v0.0.9 tags are force-moved to point here so future binary builds
from `ZDDC_REF=stable` get clean stable bytes baked in. The
old commit (a02a26d) remains in history; just no tag references
it anymore.
Sustainable fix to ./build's release flow (commit before tag,
skip embedded mutation on plain dev/alpha cuts) is a separate
follow-up — this commit only fixes the in-flight state.
Schema-driven form renderer plus zddc-server endpoints that turn any
<name>.form.yaml into a working data-collection form at <path>/<name>.form.html.
Submissions land in <path>/<name>/<YYYY-MM-DD>-<email-sanitized>.yaml,
ACL-gated by the existing .zddc cascade. The form posts back to its own URL;
the server strips ".html" and routes by what's underneath, so create and
update use the same client-side code path.
Form spec dialect: JSON Schema 2020-12 + RJSF-style ui:* hints, written in
YAML. Chosen for LLM authorability — it's the canonical structured-output
target for OpenAI/Anthropic, and the ui:* convention is the most-trained UI
hint vocabulary. Supported subset for v0: type (string/number/integer/boolean/
array/object), enum, min/max, minLength/maxLength, required, additionalProperties:
false, properties, items, format (date, email). Round-trip mode is form-as-truth:
submission YAML is regenerated each save, comments are not preserved (the v1
file-as-truth mode for hand-edited files like .zddc itself is deferred).
New components:
* form/ — sixth single-file HTML tool, vanilla JS renderer (~760 LoC)
* zddc/internal/jsonschema/ — focused JSON Schema validator covering only
the v0 keyword subset. Match-implementation-cost-to-surface-used: a full
library brings 70%+ surface we don't use; revisit when v1 adds $ref +
oneOf + if/then/else.
* zddc/internal/handler/formhandler.go — RecognizeFormRequest / ServeForm,
capability-URL re-edit, atomic submission writes via the new
zddc.WriteAtomic helper extracted from writer.go.
* dispatch() in zddc-server/main.go now intercepts *.form.html and
*.yaml.html before the static-file path; spec existence is the trigger.
Build pipeline: form joins ZDDC_RELEASE_TOOLS in lockstep, gets its own
embedded copy in handler/form.html (separate from the apps cascade —
the form renderer is fixed, not subject to per-folder version overrides).
Tests: 5 new Playwright specs (form-safety) + 14 new Go tests across the
validator and handler. All 172 Playwright tests + 10 Go packages green.
End-to-end manual verification: GET empty → POST 201 + capability URL →
GET re-edit (pre-filled) → POST update → 200, raw YAML browsable, ACL
deny → 403.
Docs: form/ section added to AGENTS.md and ARCHITECTURE.md. AGENTS.md
also documents the implementation-vs-dependency policy. CLAUDE.md repo-shape
list extended.
Deferred (v1+): .zddc editor migration onto this system, file-as-truth
lossless YAML round-trip, ui:show-when conditional visibility, oneOf/anyOf,
apps-cascade preview hook, cascade-fetched form definitions.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Runner now runs in a quadlet container on caddy-net, so 127.0.0.1
is the runner's own loopback. Reach the Caddy container by name
('caddy') with --connect-to keeping SNI/Host as the public hostname
so the right vhost matches.
Also adds the tag trigger: push of zddc-server-v[0-9]+.[0-9]+.[0-9]+
auto-cuts a stable release. The lockstep set pushes six tags at once;
filtering on zddc-server-v* gives exactly one workflow run per cut.
Re-cutting at the tagged commit is safe — _promote_stable in
shared/build-lib.sh is idempotent re: tag creation.
Forgejo Actions workflow that runs ./build alpha|beta|release [version]
followed by ./deploy --releases. Uses the host-mode runner so the
behavior is identical to manual cuts. Tag-trigger added later once
the dispatch path is exercised.
Migrates from in-repo orphan `website` branch + LFS to a two-repo +
deploy-host model so source editing is fully decoupled from live state.
- Source code stays here (codeberg.org/VARASYS/ZDDC).
- Hand-edited website content moves to a separate Codeberg repo
(codeberg.org/VARASYS/ZDDC-website, cloned at ~/src/zddc-website/).
- Live site is /srv/zddc/ on the deploy host (Caddy bind-mount),
populated by ./deploy from this repo's dist/release-output/ plus
~/src/zddc-website/.
- Releases are no longer in any git history — reproducible from
<tool>-vX.Y.Z tags via `./build release X.Y.Z`. No LFS, no
Codeberg release assets.
Build/deploy split:
- ./build (no arg) is source-only; nothing in dist/release-output/
or /srv/zddc/ is touched.
- ./build alpha|beta|release seeds dist/release-output/ from
/srv/zddc/releases/ (preserving symlinks), then mutates the
channel(s) being cut on top. The bundle is always a complete
intended-live snapshot, so the verifier sees a complete world
and ./deploy --releases (rsync --delete-after) replaces live
state cleanly.
- New ./deploy wraps the rsync flow with --content / --releases
subcommands.
Docs updated to reflect the new model: CLAUDE.md, AGENTS.md,
ARCHITECTURE.md, zddc/README.md, README.md, .gitignore, shared/
build-lib.sh comments, deprecated zddc/release.sh message.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Reverts the prior CLI simplification. ./build (no arg) now does source
work only — tool dist/ + cross-compiled zddc-server binaries — and
leaves the website worktree alone. Channel/release cuts are explicit:
./build dev build (source only, no deploy)
./build alpha cut alpha (cascades nothing)
./build beta cut beta (cascades alpha → beta)
./build release [X.Y.Z] cut stable (cascades all)
Rationale: editing source shouldn't have a side-effect on the live
site. The website worktree at ~/src/zddc-website/ is what Caddy serves
in real time, so any write to it is a deploy. Treating dev iteration
as alpha-publish was confusing — the user wanted source builds and
deploys to be distinct verbs.
Mechanically: a `dev` (default) branch is added to the case statement;
the post-build matrix-index regen + channel-link verifier are
conditional on RELEASE_CHANNEL being set; dev builds skip them.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Renames build.sh → build and replaces the --release flag form with
subcommands:
./build cut alpha (default; active dev iteration)
./build beta cut beta (cascades alpha → beta)
./build release cut stable (coordinated next version)
./build release X.Y.Z cut stable at explicit version
./build help
The contract shift: there's no longer a "plain dev build that doesn't
touch channels" at the top level. Every full-stack build is a publish
action — running ./build IS active dev iteration, which is what alpha
already meant. To iterate on one tool without writing to the website
worktree, use the per-tool sh tool/build.sh (unchanged).
Output continues to land in ${ZDDC_DEPLOY_RELEASES_DIR:-$HOME/src/zddc-website/releases}
and nothing is pushed automatically. Commit + push the website branch
yourself when you want to publish. Stable cuts still tag locally on
main; tags push separately too.
Behind the scenes: the export of ZDDC_DEPLOY_RELEASES_DIR is moved
above the per-tool build.sh invocations so children inherit it. The
prior "if RELEASE_CHANNEL else write_zddc_server_stubs_all" branch is
collapsed since RELEASE_CHANNEL is always set under the new CLI.
Docs (CLAUDE.md, AGENTS.md, ARCHITECTURE.md, zddc/README.md) updated
to reference ./build everywhere; the per-tool sh tool/build.sh refs
stay (they're a separate, narrower entry point).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Moves website source + release artifacts off `main` and into a new
orphan branch named `website` in this same Codeberg repo. A `git worktree`
of that branch — typically at ~/src/zddc-website/ — is what the system
Caddy now bind-mounts and serves at zddc.varasys.io. Decoupling source
from the live site means editing source can no longer accidentally
affect what's published.
Layout going forward:
- ~/src/zddc/ — main worktree (this branch, source only).
- ~/src/zddc-website/ — git worktree of the `website` branch:
hand-edited content + LFS-tracked release
artifacts (server binaries) + regular-git
HTML tool releases + symlinks.
- Caddy bind-mount swapped: ~/src/zddc/website → ~/src/zddc-website
(quadlet at /etc/containers/systemd/caddy.container, restarted).
Build pipeline now writes releases to
${ZDDC_DEPLOY_RELEASES_DIR:-$HOME/src/zddc-website/releases}.
- build.sh: RELEASES_DIR points at the env var
- shared/build-lib.sh: promote_release honors the env var, falls
back to the legacy in-repo path so any
standalone single-tool release on a checkout
that still has website/ keeps working
- freshen-channel: passes ZDDC_DEPLOY_RELEASES_DIR through to
the worktree-based build
Docs (CLAUDE.md, AGENTS.md, ARCHITECTURE.md, .gitignore) updated for
the new layout. The 51 MB of website/ blobs stays in main's history
(no force-push); over time Codeberg's GC will pack them down.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
