serveSpecializedNoSlash now consults zddc.ViewAt(dir, "dir"): an explicit
`views.dir` in the cascade overrides the default_tool-derived app for the
no-slash directory URL. default_tool stays the sugar fallback (ViewAt returns
it when no views.dir is declared), so existing deployments are unaffected —
purely additive.
Also fixes the mergeOverlay trap (per the .zddc-policy-key checklist): added
Views to walker.go's per-level merge so views: survives cascade resolution at
default-driven paths (without it the key silently no-ops). Verified by a
defaults-path unit test (TestViewAt): default_tool/dir_tool surface via ViewAt;
an explicit views: entry overrides default_tool and declares a file shape.
go build + go test ./... all green. (Next: ServeView config injection from
.zddc.d/, the file→form shape, recognizer retirement, client + ./build.)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Replaces the URL/channel/version-fetching tool-HTML system with a
local-only override model. No network fetch, no Ed25519 signatures, no
channels/versions, no `apps:` .zddc key.
Tool HTML resolves, in precedence:
1. a real file on disk at the path (operator drops browse.html / archive.html
/ a new mytool.html) — served by the existing static handler;
2. an `<app>.html` member of the site-root <ZDDC_ROOT>/.zddc.zip bundle, read
server-side via internal/zipfs (local file, no fetch, no signature;
re-stat'd each request for free hot-reload);
3. the embedded //go:embed default.
Remove (complete unwire):
- internal/apps/{fetch,verify,cache,singleflight}.go and their tests; the
spec-parsing/cascade machinery in apps.go (ParseSpec/Resolve/PreviewLine/
SpecComponents/appsState, DefaultUpstream*/DefaultChannel/CacheDirName).
- --apps-pubkey / ZDDC_APPS_PUBKEY flag+env+Config field; the setupApps
cache/fetcher/pubkey wiring (now just apps.NewServer(root, version)).
- the `apps:` / `apps_pubkey:` .zddc keys: ZddcFile.Apps/AppsPubKey, the
walker merges, cascade-summary adds, validate.go apps validation
(ValidateAppSourceSpec/validateURLSpec/validateChannelOrVersion/
AppsDefaultKey/IsValidAppsKey), and the isZero/is-empty refs. A stale
apps:/apps_pubkey: in an existing .zddc is now silently ignored
(back-compat), not a parse error. Client .zddc validator (preview-yaml.js)
drops the apps/apps_pubkey keys + appsmap case.
Add:
- internal/apps/bundle.go — nil-safe Bundle over <root>/.zddc.zip with
stat-based hot-reload, size caps, corrupt-zip tolerance.
- handler.go: Server{Bundle}, resolveBytes (bundle→embedded), simplified
Serve; X-ZDDC-Source = bundle:<m> / embedded:<app>@<ver>.
- dispatch: GET /.zddc.zip is 404 for everyone (config, not content); the
server reads members from the filesystem internally.
Tests: new bundle_test.go (member hit/absent/no-file/hot-reload/corrupt);
handler_test.go rewritten for bundle-overrides-embedded, absent-member→
embedded, unknown-tool 503, conditional-GET for both sources; dispatch test
covers bundle override + /.zddc.zip 404 + availability rules. go build/vet/
test ./... all green; gofmt clean. Docs (AGENTS.md, ARCHITECTURE.md) updated.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Replace the blanket "block every dot/underscore segment" dispatch guard
with a single reserved namespace, .zddc.d/, which is admin-only at every
depth. Everything else dot-prefixed is now ordinary ACL-governed content;
a leading dot only hides an entry from listings (UI), not from the ACL.
.zddc.d/ holds the bearer-token store, so it must stay closed even under a
broad operator grant (e.g. `*: rwcd`). The path-tree cascade has no
match-this-name-at-any-depth rule, so .zddc.d/ is gated by segment name via
a hard rule that overrides operator ACLs — on reads in dispatch (404,
existence-hidden) and on writes in authorizeAction (403 defense-in-depth
for direct callers). Token validation is unaffected: it reads
.zddc.d/tokens directly from the filesystem in ACLMiddleware, before the
HTTP-layer gate.
The segment match is case-insensitive (strings.EqualFold): ZDDC_ROOT may
sit on a case-insensitive filesystem (SMB/CIFS/Azure Files) where .ZDDC.D
resolves to the same dir, so a write to a case-varied path — e.g. a MOVE
destination header that skips dispatch's canonical case-folding — must not
slip past the gate and plant a forged token. The dispatch gate also runs
BEFORE the raw .zddc view so the reserve's own cascade
(/<dir>/.zddc.d/.zddc) is existence-hidden rather than leaked by
ServeZddcFile. Regression tests cover both.
To keep all bookkeeping inside the one reserve, relocate the last two
caches under it (both regenerable, no data migration): the apps cache
_app/ -> .zddc.d/apps/ and the per-directory MD-conversion cache
<dir>/.converted/ -> <dir>/.zddc.d/converted/.
New internal/handler/sidecar.go defines ReservedSidecar + the
HasReservedSidecar / ActiveAdminForSidecar predicates used by both the
dispatch read-gate and the write-path gate.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Repoint handler + dispatch tests to the top-level peer layout: register
parties via ssr/<party>.yaml where party_source gates writes; move
workspace paths out from under archive (incoming/working/staging/reviewing
+ mdl/rsk are top-level, archive/<party>/{received,issued} stay WORM);
rewrite SSR create (writes ssr/<party>.yaml, no archive folder) + SSR
rename (registry-only); accept-transmittal source incoming/<party>/<txn>;
plan-review scaffolds top-level reviewing/staging; tablehandler
classifyVirtualTableDir recognizes <project>/<peer>/<party> (depth-3) for
per-party mdl/rsk tables. Full Go suite green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Reshape the project layout from "archive/ is the only physical dir + six
virtual aggregators" to a flat set of physical, party-partitioned peers:
archive/<party>/{received,issued} pure WORM (one rule, no exceptions)
incoming|reviewing|working|staging/<party>/ workspaces
mdl|rsk/<party>/*.yaml registers (cross-party aggregate at the
peer root, $party from the real subdir)
ssr/<party>.yaml submittal status register AND the
authoritative party registry
A party exists iff ssr/<party>.yaml exists; the new `party_source: ssr`
cascade key gates party-folder creation under every other peer (archive
included) — create <peer>/<party> only when the registry row exists, else
409. Registration is a plain create of ssr/<party>.yaml (no WORM gymnastics),
so archive/ stays purely WORM.
Server core:
- defaults.zddc.yaml rewritten to the flat-peer + WORM-archive + party_source
shape; every virtual: removed; mdl/rsk get document_controller rwcd.
- slots.go: projectPeers/IsProjectPeer; perPartySlots={received,issued}.
- party_source key end-to-end (file.go/walker/lookups/cascade) + PartyRegistered.
- ensure.go canonical-ancestors generalized to peers; virtual reject removed.
- virtualviews.go: deleted the virtual-URL resolver/types/regex; kept
ListParties (reads ssr/*) + repointed ListRollupRows (physical <peer>/*/*).
- fs/tree.go: mdl/rsk peer-root listing aggregates physical party subdirs
(replaces the subdir folder-nav); ssr flat; spec entries advertised.
- fileapi.go: deleted virtual PUT/DELETE rewrites; mkdir allowlist → peers;
partySourceGate on mkdir/PUT/move.
- virtualviewhandler.go → ServeInjectedRow ($party/name injected on read so
the tables client renders the column unchanged).
- ssr/form/table handlers repointed to real paths (SSR create writes
ssr/<party>.yaml; rollup create writes mdl|rsk/<party>/<file>.yaml; SSR
rename is registry-only); IsDefaultSpec recognizes the new spec locations.
- accept-transmittal source incoming/<party>/<txn> (+ PartyRegistered guard);
plan-review scaffolds top-level reviewing/<party> + staging/<party>.
- main.go dispatch: removed virtual-row GET + folder-nav 302; injects the
source column on register-row reads.
Non-test build is green. Test suites still assert the OLD layout (verified:
all current failures are stale expectations, not bugs) — the test rewrite,
browse/tables client updates, and the data-migration script follow.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Consolidate edit-history bookkeeping under the single reserved .zddc.d/
sidecar (where tokens + access logs already live), instead of its own
top-level .history/ dot-name:
- history.go: record + text history now write/read <dir>/.zddc.d/history/<stem>/
(was <dir>/.history/<stem>/). Const renamed .history → .zddc.d/history and
unexported (the only external user was the dispatch carve-out). The history
VIEWER endpoints (<record>.yaml?history=1, <file>?history=…) read it
server-side, so they keep working for anyone with read on the live file;
the raw store is bookkeeping, blocked by the existing dot-prefix guard.
- main.go: drop the .history GET carve-out (b9ebee7) — superseded; history is
reached via the viewer, not raw browsing. Reword the guard comment to
"reserve .zddc.d/ bookkeeping" (Part B will replace the blanket block with a
.zddc.d/ admin-fence).
- Delete dead .devshell references (the dev-shell was dropped from the chart):
guard comment, paths.go comment, test fixtures/cases (→ .zddc.d), and docs.
This is Part A of the approved plan: ship history in its permanent home so we
never migrate it twice. Tests updated to the new paths; the obsolete
TestDispatchHistoryReadCarveOut is removed (raw-block covered by
TestDispatchHidesDotPrefixedSegments, viewer by mdhistory_test).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Two cleanups from the hard-coded-vs-cascade audit:
#2 Centralize the canonical slot names. The lists {ssr,mdl,rsk,working,
staging,reviewing} and the per-party {incoming,received,issued,mdl,rsk,
working,staging,reviewing} were hand-written across ensure.go (×2),
fileapi.go (×2), virtualviews.go, lookups.go. New internal/zddc/slots.go is
the single registry with IsRowSlot/IsFolderNavSlot/IsVirtualAggregatorSlot/
IsPerPartySlot; virtualViewRE is built from it. Slot NAMES stay hard-coded
(they carry bespoke behavior) but now live in one place — adding/adjusting a
slot is one edit, not a hunt. Pure refactor; behavior unchanged.
#1 Make the history file-type selection cascade-driven. IsTextHistoryCandidate
hard-coded ".md"; now it matches the effective history_globs from the .zddc
cascade (default ["*.md"], widen per-deployment e.g. ["*.md","*.txt"]). New
ZddcFile.HistoryGlobs + mergeOverlay + PolicyChain.EffectiveHistoryGlobs +
HistoryGlobsAt, threaded through serveFilePut/serveFileMove/dispatch and
ServeTextHistory (now takes fsRoot). The history: bool still gates whether
snapshots are recorded; history_globs only says which file types qualify.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Clicking a history snapshot in the tree 404'd: the dispatcher's dot-prefix
guard blocks every .-segment URL, and the preview fetch hit the raw
.history/<stem>/<snap>.md path. But .history is ACL-modeled content (it
inherits the shadowed file's .zddc chain), not infra like .devshell — so
the guard was redundant with permissions there.
Carve GET/HEAD of .history out of the dot-prefix guard: snapshots are now
fetchable as ordinary ACL-gated files (read the live file → read its
history). Writes into .history stay blocked, and the listing dot-filter
still hides it from default views unless ?hidden is set. Export
handler.HistoryDirName for the dispatcher.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Redesign the markdown edit-history store from content-hashed blobs +
log.jsonl to one self-describing file per save:
.history/<stem>/<ts>-<email>.<ext>
The filename IS the audit (colon-free UTC timestamp valid on SMB/Azure
Files + the authoring email); listing the directory is the history. No
sidecar log, no hashing. A byte-identical save is a no-op; a pre-existing
file lazy-seeds its current bytes (author "unknown", stamped at mtime).
Reverting copies an old snapshot back (records as a fresh save). Snapshots
are kept forever.
Fixes the 404 reading history: reads no longer require history to be
*currently* enabled — ServeTextHistory serves whatever .history/<stem>/
exists (empty list when none); the dispatch drops the EffectiveHistory
gate for reads. WRITES stay gated by the history: flag. (The 404 came from
the aggregator refactor turning history off on project-level working/,
which made already-recorded snapshots unreadable.)
Renames: an in-place rename carries .history/<stem>/ to the new name
(serveFileMove); a cross-dir move leaves it behind.
Defaults: history: true now ships on the three live-editing slots —
working, mdl, rsk — at both the project-level nodes and the per-party
folders. It's a .zddc cascade key, so operators override per project.
Records (.yaml in mdl/rsk) keep their separate record-history path.
Browse history viewer updated to the filename-based version id (id ←
sha). Tests rewritten for the per-file scheme + rename behavior + SMB-safe
names; HistoryAt defaults test updated.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Creating a folder/file at a project-level folder-nav aggregator root
(working/staging/reviewing) used to error or silently shadow — the slots
are virtual and content is party-scoped. Now browse opens a party picker
that targets archive/<party>/<slot>/<name>, with a "+ New party…" option
(server-gated to the document_controller via the existing archive/ ACL).
- events.js: aggregatorRoot detection + openPartyPicker modal (mirrors the
stage.js modal), createInAggregator routes the create to the canonical
archive path; rewriteAggregatorPath handles right-clicking a party row
shown in an aggregator listing so it never re-prompts.
- server: serveFileMkdir now 409s a mkdir inside an aggregator
(rejectProjectAggregatorMkdir) with a pointer at archive/<party>/<slot>/,
instead of letting the write fall through to an unreachable shadow dir.
Reverts the prior session's project-level creator-owned working/ folders
(per the design decision to make all three folder-nav slots uniformly
party-scoped): working/ is a pure virtual aggregator again like
staging/reviewing — drops the working/ history+auto_own+acl defaults, the
EnsureCanonicalAncestors working exception, the working-root document-
controller file gate (serveFilePut/Move) and zddc.IsRoleMemberAt. Per-party
archive/<party>/working/ keeps its own history + auto-own.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Replace the project-level working/<email> "personal workspace" idea (too much
complexity for too little) with a simpler model on the virtual <project>/working/:
- EnsureCanonicalAncestors now materialises the working/ slot dir on disk the
first time real content is created beneath it (it stays a plain dir, never
auto-owned). ssr/mdl/rsk/staging/reviewing keep rejecting physical writes.
- Each <project>/working/<folder>/ a user creates gets an unfenced auto-own
.zddc (creator rwcda; the team inherits read+create-new, not w/d). history:
true still inherits in, so markdown drafts there are versioned.
- defaults grant project_team rc + document_controller rwc at working/ so users
can create their folders and the DC has authority throughout.
- A bare file DIRECTLY at the working/ root is reserved for the
document_controller: serveFilePut and serveFileMove reject non-DC writes/moves
there (isProjectWorkingRootFile + zddc.IsRoleMemberAt), independent of the ACL
verb since mkdir and file-PUT both authorise as ActionCreate. Users work inside
a folder; the DC creates files at the root or promotes one up with a MOVE.
Tests: ensure_test materialisation + plain-slot cases; fileapi_test DC-gate for
PUT and MOVE. The generic dispatch-routing test moves its ops into working/drafts/.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A history: true .zddc subtree (enabled by default on archive/<party>/working/)
routes markdown PUTs through WriteTextWithHistory: each save snapshots the
content into a hidden, immutable .history/<stem>/ store (content-addressed
blobs + an append-only log.jsonl carrying server-stamped {ts, email, sha,
prev}) before writing the live file. The live file at its natural path stays
the source of truth; no symlinks, no audit in the body/filename.
Reads: GET <file>?history=1 lists versions (newest-first, current flagged);
GET <file>?history=<sha> returns that version's bytes (hex-id guard against
traversal). Listings carry a per-file History flag so the browse client knows
where to offer the affordance.
History is subtree-inheriting and ignores inherit:false ACL fences (versioning
is a write behavior, not a permission), so fenced per-user homes under working/
are covered too. No-op saves dedup; pre-existing files lazy-seed their origin
version. Records (.yaml) keep their existing in-body-audit history path.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The app-HTML dispatch branch (/archive.html, /browse.html at root) gated
serving the tool shell on read permission for the root dir — which no
per-project-scoped user has — so the root-level multi-project archive view
(/archive.html?projects=A,B) returned 403 to anyone but a root-elevated
admin. The landing page (/) and ServeDirectory already treat the root path
as a public shell and filter data per-project; the app-HTML branch didn't
get the same bypass.
Skip the read gate when the tool's request dir is the root: the shell is a
static app carrying no data, and the tool's own per-project/per-dir fetches
stay ACL-gated (fs.ListDirectory filters per entry). Non-root tool paths
(/<project>/archive.html) keep their read gate.
Test: a non-root, un-elevated user gets 200 on /archive.html but still 403
on a project directory they can't read.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Root cause of the flaky cache tests (TestServeHTTP_DirectoryListingsCachedAsSidecar
and the other hit-path tests, ~1-in-many under parallel load): on a cache
hit, ServeHTTP launches `go c.revalidate(...)` / `go c.revalidateListing(...)`,
which write into the cache root (MkdirAll + CreateTemp + Rename). Those
goroutines outlive the request — and in tests, the test — so they race
t.TempDir's RemoveAll cleanup, recreating the dir or dropping a temp file
mid-removal. testing then reports "TempDir RemoveAll cleanup: ... directory
not empty" and marks the test failed (with a 0.00s body, no assertion line).
It only surfaced under the full parallel suite / -count because the timing
has to collide.
Fix: track these background goroutines in a sync.WaitGroup via a goBackground
helper, and expose Wait(). newTestCache registers t.Cleanup(c.Wait) — cleanups
fire LIFO and t.TempDir registered its RemoveAll first, so the drain runs
before it (upstream Close was registered earliest, so it runs last and stays
up while goroutines finish). runClient also calls cacheLayer.Wait() after
srv.Shutdown so in-flight sidecar writes complete on graceful shutdown rather
than being abandoned.
Verified: cache package at -count=200 reliably failed before, passes clean
after (0 failures, 0 cleanup errors); full `go test ./...` + vet green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
May 2026 reshape. archive/ is now the only physical project-root
directory; working/, staging/, reviewing/ move from the project root
into each archive/<party>/ folder. Six top-level URLs become virtual
aggregators served via the cascade rather than disk:
ssr/mdl/rsk tables rollups across parties with a
synthesised $party source-party column
working/staging/ browse folder-nav listings of parties with
reviewing non-empty content in the slot; per-party
URLs 302-redirect to archive/<party>/<slot>/
Mkdir at the project root is restricted to `archive` and `_`/`.`-
prefixed system names — virtual aggregator names and ad-hoc folders
return 409.
Plan Review hardcodes the scaffold convention (archive/<party>/
{reviewing,staging}/<tracking>/); the pre-reshape
on_plan_review.{reviewing_root,staging_root} cascade keys are dropped.
document_controller is now subtree-admin of every archive/<party>/
(not of project-root working/staging/ as before), so per-party
lifecycle slots inherit admin authority through the cascade.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
A fresh ZDDC deployment grants no access to anyone until an operator
populates the root .zddc (admins) and per-project .zddc files (role
members). Until now this was only documented in comments inside the
embedded defaults.zddc.yaml, surfaced via `zddc-server show-defaults`
— operators wiring up a fresh master had no obvious doc to follow and
no startup signal when the bootstrap was missing or empty.
- README.md: new "## Deploy: bootstrap config" section between Tools
and File-naming convention. Two canonical examples (root admin-only,
per-project role members), schema essentials (verb bits, principal
forms, admins-only-at-root), and the acl: { allow: [...] } footgun
that silently drops grants.
- AGENTS.md: new "### Bootstrap config (REQUIRED — unlocks the server)"
subsection at the top of ## zddc-server. Same content as README but
with file:line citations into zddc/internal/zddc/file.go for the
schema source of truth.
- zddc-server: new warnIfNoBootstrap fires a slog.Warn at startup when
the root .zddc grants nobody anything (no admins, no acl.permissions,
no role members). Master mode only; skipped under --no-auth.
- config validator's existing no-root-.zddc fail-fast error message now
also points at the new README + AGENTS sections so all three signals
(fail-fast, runtime warning, docs) converge.
Smoke-tested all paths: empty root + default (fail-fast), empty root +
--insecure (file-missing warn), admins-only / perms-only / role-members
-only (silent), title-only and acl.allow footgun (both warn), --no-auth
(suppressed). All existing go tests pass.
Follow-up (manual, separate repo): add an analogous section to
~/src/zddc-website/reference.html.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds cascade-driven schema + immutable audit history for the three table-style
record stores (mdl, rsk, ssr). Two new .zddc top-level keys carry the rules:
- field_codes: discriminated-union vocabulary (kind: enum|pattern|free) for
the components used to compose tracking-number filenames and constrain
record bodies. Map-merge across the cascade, mirror of apps: semantics.
- records: per-pattern rules (filename_format, field_defaults, locked,
row_field, row_scope_fields). Filename-pattern scoping lets the SSR rule
live at the party-folder level without bleeding onto mdl/rsk siblings.
PUTs to record YAML files route through a new WriteWithHistory orchestrator
(internal/handler/history.go) which:
- strips six client-supplied audit fields (created_at/by, updated_at/by,
revision, previous_sha) so the client can't forge them
- validates body values against the cascade-resolved field_codes
- enforces filename_format composition (URL basename must match body fields)
- checks locked: defaults (422 mismatch)
- archives prior bytes to <dir>/.history/<base>/<RFC3339Nano>-<sha8>.<ext>
- stamps server-managed audit fields and writes the live file
History-before-live ordering preserves the prior version even on mid-write
crash. previous_sha forms a hash chain across revisions for tamper evidence.
The embedded defaults.zddc.yaml now declares records: entries for mdl, rsk,
and ssr.yaml. RSK rows carry the table-tracking components + row sequence
(filename = <table-tracking>-<row>); MDL rows compose to their own
tracking number; SSR records' identity is the party folder name.
GET <record>.yaml?history=1 returns a JSON list of prior revisions, ACL
gated identically to the live record. dot-segment rejection in
resolveTargetPath protects .history/ from direct client writes.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The bwrap engine + OCI engine that lived in internal/convert/runner.go
both leak isolation policy into Go code. Replaced with a single image-
side wrapper that drop-in-shadows pandoc and chromium-browser on PATH.
zddc-server's only contract with the image is now "exec.Command(name,
args) gets you that tool's behavior" — sandboxing, resource caps, and
namespace setup live entirely in shell scripts shipped by the image.
Architecture:
- zddc/runtime/zddc-cgroup-init runs at container start. cgroup v2's
"no internal processes" constraint forbids a cgroup from having both
children and processes; the init script moves PID 1 into a child,
enables +memory +pids in subtree_control, then exec's zddc-server.
Best-effort: degrades cleanly to "no resource caps" if cgroupfs
isn't writable.
- zddc/runtime/zddc-sandbox-exec is the per-call wrapper, symlinked
from /usr/local/bin/{pandoc,chromium-browser}. Creates a transient
cgroup v2 (memory.max + pids.max), then bubblewrap-sandboxes the
real binary at /usr/bin/<name>: --unshare-all, --ro-bind /usr,
--proc /proc, --tmpfs /tmp, --clearenv. Caller's scratch dir comes
in via ZDDC_SCRATCH env and is bind-mounted at the SAME path so
absolute paths round-trip unchanged.
Go simplifications (~250 lines net deletion):
- Runner interface: Run(ctx, binary, stdin, scratchDir, cmd) — no
ToolSpec, no mount list, no engine concept. Single localRunner
implementation; bwrapRunner + containerRunner both deleted.
- health.Probe just looks up pandoc + chromium on PATH; Capabilities
drops engine kinds.
- Convert.go: ToHTML/ToPDF write to a per-call scratch dir under
TMPDIR and pass absolute paths; the wrapper bind-mounts the dir.
No more "/tpl" / "/pdf" mount-point indirection.
- Config drops --convert-pandoc-image, --convert-chromium-image,
--convert-engine, --convert-podman-socket (OCI engine gone) and
--convert-cpus (CPU caps don't apply in the new model — wall-clock
+ memory + pids is the cap set). Defaults raised to match the new
caps the user authorized: mem 512→1024 MiB, pids 100→256,
timeout 30→60 s.
Image:
- zddc/runtime.Containerfile builds the production runtime image
(alpine + bubblewrap + pandoc + chromium + font-noto). Two
COPY statements pull in the wrapper scripts; ln -s symlinks the
shadow names.
- bitnest dev image mirrors this layout under /var/lib/zddc-dev-build/.
Container privilege required:
- Nested bwrap needs the outer container to permit user + mount
namespace creation + MS_SLAVE on root. The default seccomp +
AppArmor profiles block all of these. Quadlet adds:
--cap-add=ALL
--security-opt=seccomp=unconfined
--security-opt=apparmor=unconfined
--security-opt=unmask=ALL
Helm chart sets the equivalent via securityContext (capabilities.
add: SYS_ADMIN, seccompProfile.type: Unconfined, appArmorProfile.
type: Unconfined). Trade-off documented in AGENTS.md: zddc-server
RCE now has near-root power within the container, but the bind-
mount layout still bounds blast radius; bwrap is the real boundary
between zddc-server and untrusted markdown.
Tests: convert_test.go fully rewritten for the new Runner signature.
Drops TestBwrapArgs_* (functionality moved out of Go) and
TestImageTag (no more image refs). All 15 Go test packages green.
Verified live on bitnest: pandoc --version round-trip exits 0
through the wrapper; MD→DOCX produces a valid Word 2007+ file
end-to-end.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds the risk register as a sibling of MDL under archive/<party>/, and
three project-level virtual aggregations at <project>/{ssr,mdl,rsk}:
- SSR aggregates archive/<party>/ssr.yaml; "+ Add row" materializes a
new party folder (mkdir + auto-own .zddc + ssr.yaml). Renames go
through X-ZDDC-Op: ssr-rename, which os.Rename's the party
directory so every row inside follows. Party name doubles as the
folder name (no opaque IDs) and is path-derived on read.
- MDL/RSK rollups list every deliverable / every risk across all
parties with a derived `party` column; "+ Add row" is suppressed
because party affiliation is ambiguous in the aggregate view.
All four virtual roots are declared `virtual: true` in
defaults.zddc.yaml. Spec/form bytes come from six new embedded
defaults (default-rsk.*, default-ssr.*, default-project-{mdl,rsk}.*)
served via a generalized IsDefaultSpec/IsDefaultSpecAbs that replaces
the MDL-only recognizer. Listing synthesis lives in fs/tree.go;
ACL on each synthetic row evaluates against the canonical
archive/<party>/ chain so non-owners see rows read-only. PUT/DELETE
through virtual URLs rewrite to canonical paths in fileapi.go via
sibling-shape blocks that don't touch the ACL gate. SSR row DELETE
returns 405 (delete the party folder via the archive view).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replaces the always-spawn-an-OCI-container model with a per-call
bubblewrap sandbox. Pandoc and chromium binaries are baked into the
zddc-server runtime image; each conversion runs them under bwrap's
Linux-namespace isolation. No daemon, no socket, no privileged outer
container, no OCI image pull at conversion time.
Why: the OCI engine paid ≈ 350 MB image pulls + 400 MB persistent
storage + ~300 ms per-conversion startup, plus required either an
on-host daemon socket (zddc-RCE → host-RCE in one hop) or nested
container privileges. bwrap gets the same sandbox properties
(--unshare-all, ro-bind /usr, tmpfs /tmp, clearenv, no-network) at
~5 ms per call and zero external dependencies. This is the same
primitive Flatpak uses for every app launch — battle-tested at scale
for "untrusted-input, short-lived, isolated."
Runner abstraction:
- `Runner.Run` signature: image string → ToolSpec{Image, Binary}.
Both fields populated by entry points; whichever engine is
installed reads the one it needs.
- `bwrapRunner` (new): assembles bwrap argv via `buildBwrapArgs`
helper (testable in isolation), spawns bwrap with the binary.
- `containerRunner` (renamed conceptually to "legacy fallback"):
unchanged behavior, still reachable for hosts that prefer OCI
containers per conversion.
Probe order in health.Probe: bwrap → podman → docker. First hit wins.
Engine kinds in Capabilities: "bwrap" | "podman" | "docker". The
no-engine error message now lists all three.
Config (cmd/zddc-server):
- new --convert-pandoc-binary / ZDDC_CONVERT_PANDOC_BINARY (default "pandoc")
- new --convert-chromium-binary / ZDDC_CONVERT_CHROMIUM_BINARY (default "chromium-browser")
- existing --convert-pandoc-image / --convert-chromium-image kept
for the OCI engine, doc updated to clarify they only apply there.
- --convert-engine helptext lists bwrap first.
Images:
- New `zddc/runtime.Containerfile` — alpine + bubblewrap + pandoc-cli +
chromium + font-noto. Documents build/publish workflow.
- helm/zddc-server-prod/values.yaml.example: runtimeImage default
switched to a placeholder for the new bundled runtime image; bare
alpine NO LONGER works for /.convert (clearly called out in the
comment).
- bitnest dev: /var/lib/zddc-dev-build/Containerfile mirrors the
production runtime image. Quadlet at /etc/containers/systemd/
zddc.container drops the podman-socket mount (no longer needed)
and sets ZDDC_CONVERT_ENGINE=bwrap explicitly to avoid silent
downgrades if a stray podman ends up on PATH.
Tests:
- convert_test.go: fakeRunner / recordingRunner now record ToolSpec.
- New TestToolSpecPopulation pins that both Image and Binary are
filled by every entry point.
- New TestBwrapArgs_SandboxFlagsPresent / MountTranslation /
RejectsBadMountSpec lock in the bwrap argv shape — a refactor that
drops a hardening flag or misroutes a mount fails this loud.
Docs:
- AGENTS.md § "Server-side document conversion" rewritten around
the bwrap-first model with podman/docker as legacy fallbacks.
- ARCHITECTURE.md convert reference updated.
- internal/convert package doc reflects the two-engine probe order.
Verified end-to-end on bitnest: probe reports
engine=bwrap pandoc_binary=pandoc chromium_binary=chromium-browser
on startup. All 15 Go test packages green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Four targeted test suites that pin the invariants exercised by the
preceding audit refactor. Closes the coverage gaps identified after the
admin-decider consolidation and the .zddc write-path fix.
internal/policy/principal_test.go (NEW)
TestAllowActionFromChainP_TruthTable — 11 cases × 5 actions = 55
assertions covering every (elevated × admin-at-level × action)
combination. Pins the IsActiveAdmin short-circuit: bypass requires
BOTH (in admins) AND Elevated; elevation alone confers nothing;
empty email never matches.
TestAllowActionFromChainP_AdminScopeDepth — root admin reaches every
path; subtree admin matches in their own subtree; subtree admin
does NOT match in a sibling subtree (the chain doesn't carry
sibling admins lists).
TestAllowActionFromChainP_BypassWinsOverWorm — elevated admin
escape hatch in WORM zones, plus the negative control that an
un-elevated admin does NOT bypass WORM.
internal/handler/auth_invariants_test.go (appended)
TestInvariant_ZddcPutMatrix — 16 sub-cases across (root / project /
subtree .zddc) × (root admin / subtree admin / non-admin /
anonymous) × (elevated / un-elevated). Locks down which principal
can PUT which .zddc.
TestInvariant_ZddcDeleteMatrix — 5 DELETE cases.
TestInvariant_UnelevatedAdminNoSilentBypass — 14 anti-bypass probes:
every (admin-flavour × probe-path) tuple where an un-elevated
admin must 403. Single bypass leak → loud test failure.
cmd/zddc-server/main_test.go (appended)
TestDispatchZddcWriteRouting — full dispatcher path coverage:
GET/HEAD route to ServeZddcFile (YAML or virtual placeholder);
PUT/DELETE route through the .zddc-leaf carve-out into
ServeFileAPI; intermediate .zddc.d/ segments still 404 at the
guard.
internal/handler/middleware_test.go (appended)
TestAccessLog_ChainAdminLevelAttribution — 7 cases pinning the
forensic record: root admin → chain_admin_level=0, subtree admin
in scope → chain_admin_level=N, subtree admin out of scope → -1,
un-elevated admin → -1, non-admin → -1, anonymous → -1.
Cross-checks active_admin == (chain_admin_level >= 0) so a future
refactor can't desync them.
92 new sub-cases total. Coverage delta on the policy package:
76.1% → 87.2%; AllowActionFromChainP 0% → 100%;
activeAdminForRequest 7% → 68%.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Single audit pass that removes pre-release back-compat, consolidates the
admin-policy decider, and fixes the .zddc write path.
Field removal — acl.allow / acl.deny:
- Drop ACLRules.Allow / Deny struct fields and mergeLegacyACL().
- Remove walker / lookups / validate / decider branches that read them.
- Migrate every test fixture (YAML strings and ACLRules struct literals)
to acl.permissions: { principal → verb-set }.
- Rewrite both bundled Rego policies (access.rego, access_federal.rego)
to traverse level.acl.permissions; rewrite parity-test helpers.
- Update create-project form (profile page) to collect permissions
instead of allow/deny lists.
Admin decider consolidation:
- Delete zddc.CanEditZddc — strict-ancestor rule retired. Subtree admins
own their own .zddc; the policy decider's IsActiveAdmin short-circuit
is the single bypass site.
- Migrate tablehandler.ServeTable to AllowActionFromChainP — closes the
same Forbidden bug already fixed for /browse.html.
- Drop AccessView.EditableParentChoices and treeEntry.CanEdit (always
true after the retirement). Profile page renders AdminSubtrees
directly for both lists.
- Drop the excludeLeaf parameter from AdminLevelInChain /
IsAdminForChain — no production caller passed true.
Dead code removed:
- policy.AllowWriteFromChain (zero production callers, zero tests).
- zddc.AllowedWithChain (zero production callers; tests deleted).
ModeStrict retirement — federal posture is OPA-only:
- Delete cascade_mode.go / cascade_mode_test.go and the ModeStrict
branches in cascade.go and acl.go.
- Drop --cascade-mode flag, CascadeMode config field, and the
InternalDecider.Mode field.
- Drop the mode parameter from every cascade helper:
GrantedVerbsAtLevel, AllowedAction, EffectiveVerbs,
EffectiveVerbsRange, RoleMembers, MatchesPrincipal,
MatchingPrincipals, WormZoneGrant, PolicyChain.VisibleStart.
- Strip cascade_mode from /.profile/config and
/.profile/effective-policy responses.
- Refresh README / ARCHITECTURE.md to describe federal posture as
"deploy OPA with access_federal.rego" (NIST AC-6); the bundled Rego
is the parent-deny-is-absolute variant. The in-process Go evaluator
implements only the commercial cascade.
Legacy redirects + .admin.css fallback:
- Drop /<dir>/.zddc.html → ?file=.zddc redirect and its test.
- Drop ?zip=1 retired comment + legacy test (handled by the
.zip virtual-URL path; covered by TestServeSubtreeZip).
- Drop .admin.css fallback in profile_assets.go — only .profile.css now.
- Refresh stale "retired" / "back-compat" / "legacy" comment markers.
.zddc write path fix:
- Dispatcher: route only GET/HEAD on .zddc URLs to ServeZddcFile; carve
.zddc out of the dot-prefix guard so PUT/DELETE/POST reach
ServeFileAPI. Before this, .zddc writes 405'd at ServeZddcFile and
the YAML editor's save flow had no live path.
- ServeFileAPI.resolveTargetPath: same .zddc-leaf carve-out so the file
API accepts the path; intermediate dot dirs (.zddc.d/) stay reserved.
- Listing: compute Writable per-file with ActionAdmin for .zddc
(matches the file API's gate) instead of ActionWrite for everything.
- Virtual .zddc placeholder: compute Writable via the same
parentActiveAdmin || ActionAdmin path. Was always false before.
- browse YAML editor canSave: exempt virtual .zddc — the synthetic
body is designed to materialize on PUT.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Reads (apps resolution, directory listing, file GET, archive index,
profile pages, subtree zip, form render) used policy.AllowFromChain
with email — no admin-bypass branch fired even for elevated admins,
because IsActiveAdmin only landed in AllowActionFromChainP.
Symptom: elevated admin navigating to /browse.html got 403 because
the root cascade has no explicit read grants in my refactored root
.zddc (role memberships + admins only; no acl.permissions). The
app-resolution path's AllowFromChain didn't see admin status.
Fix: new policy.AllowFromChainP that forwards to
AllowActionFromChainP(action=read). Migrate every read-path caller
to the principal-aware variant. The decider's single bypass branch
now fires uniformly across read and write decisions.
Migrated:
cmd/zddc-server/main.go (9 sites)
handler/directory.go (1)
handler/archivehandler.go (2)
handler/zddcfile.go (1)
handler/formhandler.go (3)
handler/projectshandler.go (1; EnumerateProjects sig takes Principal)
handler/subtreezip.go (1)
fs/tree.go (1; uses already-built principal)
profilehandler.go:400 stays on AllowFromChain — it probes ACL for a
DIFFERENT email (the enumeration target, not the request principal),
so admin bypass on the request's principal doesn't apply.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The access log now reports whether the elevated user actually held
admin authority on the request's target path — i.e., whether the
single bypass branch in policy.InternalDecider.Allow would have
fired here. Three states fall out:
elevated=false, active_admin=false: normal user
elevated=true, active_admin=false: opted into admin but no admin
grant on this path (subtree-
admin out of scope)
elevated=true, active_admin=true: admin authority active for
this path — WORM/ACL bypass
Implementation: AccessLogMiddleware gains a cfg parameter and calls
activeAdminForRequest at log emission, walking the closest existing
ancestor (same logic the file API uses to build its ACL chain).
The cascade is mtime-cached upstream so the per-request cost is one
map lookup in the common case.
Audit value: a reviewer can spot at a glance whether a destructive
write was authorized by ACL or by admin bypass. Plus "elevated=true
active_admin=false" rows surface users who tried to elevate outside
their actual scope.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two layers shipped together since the second builds on the first.
LAYER 1 — reviewing/ + Plan Review scaffolding
- reviewing/ is now a real folder under each project, populated by the
Plan Review composite endpoint. The old reviewing/ virtual aggregator
handler is retired.
- POST /<project>/archive/<party>/received/<tracking>/ with X-ZDDC-Op:
plan-review scaffolds physical workflow folders under reviewing_root
and staging_root, each carrying .zddc.received_path pointing back at
the canonical submittal. Idempotent re-runs match by received_path
and re-converge the ACL.
- Virtual received window: when listing or writing under
<workflow>/received/, the server resolves through the canonical
archive/<party>/received/<tracking>/ via the workflow's
.zddc.received_path. Writes get rewritten to
<workflow>/<base>+C<n><suffix> so review comments land in the
workflow folder and never touch the WORM archive.
- Cascade defaults declare on_plan_review per project so the
reviewing_root and staging_root are configurable.
LAYER 2 — browse context-menu workflows
- Accept Transmittal: right-click a transmittal folder in
archive/<party>/incoming/ → validates ZDDC folder + filename
conformance, atomic-renames the folder to
archive/<party>/received/<tracking>/ (WORM zone), and optionally
chains into Plan Review in the same composite request. Re-acceptance
with a different revision merges file-by-file; WORM forbids
overwrite of an existing filename.
- Stage / Unstage: right-click files in working/<…>/ → "Stage to…"
with picker of existing staging transmittal folders + inline
"New transmittal folder…" create; right-click files in
staging/<…>/ → "Unstage to working/" defaulting to the user's
working/<email>/ home. Reuses the file-API move primitive.
- Create Transmittal folder: right-click the staging/ pane → prompts
for a ZDDC-conforming folder name with live validation; mkdir,
then navigate to the new folder URL where the transmittal tool
serves the editor.
- Supporting infrastructure: new CanonicalFolderAt cascade lookup +
X-ZDDC-Canonical-Folder response header so the browse SPA can
scope-gate menu items without re-implementing the cascade
client-side.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replace `?zip=1` / `?convert=docx|html|pdf` query forms with path-suffix
URLs that look like ordinary files. `<dir>.zip` and `<file>.docx` /
`.html` / `.pdf` are virtual files served by the dispatcher when stat
fails at the requested path AND the corresponding base resource exists:
GET /Project-1/archive.zip ← if archive/ is a real directory
GET /Project-1/notes.docx ← if notes.md exists
Real on-disk files always win — a genuine archive.zip in the tree
serves its bytes normally. The virtual forms only fire when nothing
real is there.
Why: the URL form lets clients emit plain <a href> without query-
string handling; `curl -O` writes a sensible filename; mirror tools
pick up the path through normal recursion; the protocol surface
becomes "every URL is a file". Bash + filesystem mental model.
Server:
- New helpers handler.RecognizeVirtualSubtreeZip /
RecognizeVirtualConvert (in subtreezip.go and converthandler.go).
- Dispatcher's stat-fails branch checks them between IsDefaultMdlSpec
and MatchAppHTML. ACL is enforced on the base resource (the source
directory for zip, the .md source for convert).
- Three legacy query-form branches removed from main.go.
Client:
- browse/js/download.js: `dir + '.zip'` instead of `dir + '/?zip=1'`.
- browse/js/preview-markdown.js: convert anchor hrefs become
`<mdUrl-minus-.md>.<fmt>` instead of `<mdUrl>?convert=<fmt>`.
- shared/zddc-source.js downloadConverted: same transform.
Tests: subtreezip_test.go test URLs cosmetically updated to the new
shape (the handler is exercised directly, so the URL is metadata only,
but the test reads better).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three coordinated changes that share the same files. Common theme:
convention beats exception. Where the codebase had a bespoke wire shape
or a special-case route, replace it with the generic shape every other
client already speaks.
== Listing protocol ==
GET / Accept: application/json used to dispatch to a bespoke
ServeProjectList handler returning {name, url, title} per project — a
shape that diverged from every other directory's listing.FileInfo
response. Now:
- listing.FileInfo gains an optional `title` field (read from each
directory's own .zddc title:). Generic clients (landing, browse)
read the same shape from every URL.
- appfs.ListDirectory emits a virtual `.zddc` entry (is_dir:false,
virtual:true) when no on-disk file exists at that path and the
caller asked for ?hidden=1. Opens an editable view of the cascade
defaults; PUT-saving its bytes materialises a real file.
- The bespoke GET / JSON branch in cmd/zddc-server/main.go is gone.
The bare-root landing serve is Accept-gated: HTML requests get the
landing tool (project picker), JSON requests fall through to
ServeDirectory and get the generic listing.
- landing's fetchProjects filters the new generic shape (is_dir,
strip trailing slash) — same pattern fetchParties already used at
/<project>/archive/.
== Form editor retirement ==
`<dir>/.zddc.html` was a server-rendered form for editing per-directory
.zddc files (~900 LOC across zddceditor.go, zddchandler.go, zddc_assets.go).
Browse's YAML/CodeMirror editor (with .zddc-schema lint) already edits
the same files via the generic file-API. Two ways to edit the same data
is exception, not convention.
- Delete zddceditor.go, zddchandler.go, zddc_assets.go and tests.
- `/<dir>/.zddc.html` → 302 redirect to `/<dir>/?file=.zddc` (browse
opens the .zddc in its editor pane).
- /.profile/zddc/* namespace deleted (REST API + assets sub-route).
- Profile page's "Editable .zddc files" list links to browse.
- ServeZddcFile's 405 message + virtual-body comment point at the
browse URL instead of the dead form.
== Admin elevation (Principal model) ==
Sudo-style: admins are treated as normal users by default; opting into
admin powers is per-request and gated by a `zddc-elevate=1` cookie.
- zddc.Principal{Email, Elevated} replaces bare-email arguments on
IsAdmin / IsSubtreeAdmin / CanEditZddc. The signature change makes
the elevation gate compiler-enforced at every admin call site —
audit-fragility is gone. The empty-email short-circuit is no longer
load-bearing for elevation; Principal.gate() is the explicit check.
- handler.ACLMiddleware derives Elevated per request: bearer tokens
are implicitly elevated (CLI clients can't toggle a cookie); browser
sessions elevate only when zddc-elevate=1 is set. PrincipalFromContext(r)
is the one-call-per-site bundling helper.
- Every admin-check call site updated to pass a Principal.
- /.auth/admin (forward_auth target for the dev-shell IDE) explicitly
bypasses elevation with a synthetic-elevated Principal — different
cookie scope than zddc-server origin, documented inline.
- AccessView gains CanElevate (elevation-independent "does this email
have admin authority anywhere?") so the header toggle can render
itself for an un-elevated admin who hasn't opted in yet.
- ServeProjectList is removed; ProjectInfo + EnumerateProjects stay
for the profile page's server-rendered project list.
- MatchAppHTML stays — still used by main.go to route <dir>/<tool>.html
URLs to the apps subsystem when no real file exists.
- Test helpers carry Elevated=true by default (matches the
pre-elevation default; tests for the un-elevated gate use the
explicit form).
Go tests pass across all 14 internal packages. Browse + every other
tool rebuilds clean.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds a UI checkbox next to the existing Sort dropdown that surfaces
hidden entries when ACL would otherwise allow read. Default off
(matches today's filtered behavior). On toggle, browse re-fetches
the current directory with ?hidden=1 and re-renders.
┌─ browse toolbar ─────────────────────────────────────────────┐
│ Sort: [Name (A→Z) ▾] ☐ Show hidden │
└──────────────────────────────────────────────────────────────┘
Server-side surface:
- internal/fs/tree.go ListDirectory gains an `includeHidden bool`
parameter. The .-prefix filter (previously hard-coded) now also
drops _-prefix entries (matches dispatch's reserved-prefix guard)
and honors the new flag.
- internal/handler/directory.go reads `?hidden=1` from the request
and threads it through.
- cmd/zddc-server/main.go dispatcher relaxes its dot-prefix and
_-prefix guards for GET/HEAD when `?hidden=1` is set, so clicking
a hidden entry's link works. `_app/` (apps cache) stays
unconditionally reserved — those bytes must go through the apps
resolver. Writes to hidden paths stay blocked (the file API has
its own segment check that the flag does NOT relax).
- internal/listing/listing.go: signature parity (the lower-level
helper that's used by tests + non-cascade listing paths).
Security model unchanged: the ACL chain on the parent dir is the only
real gate. Whoever can read the dir can see its contents — toggling
"Show hidden" just stops the client-side filter from masking
.-prefixed and _-prefixed entries. Hidden paths today:
• <dir>/.zddc ACL YAML — already exposed via /.profile/zddc
• <dir>/.converted/<base> cached MD→DOCX/HTML/PDF, same sensitivity as source
• <root>/.zddc.d/tokens/ per-token metadata; filename = sha256(token)
so not bearer-usable. Default root ACL
restricts to admins; matches /.tokens UI.
• <root>/.zddc.d/logs/ access logs; same admins-only audience
• <root>/_app/ cached upstream tool HTML (public)
• <root>/_template/ install.zip scaffolding (public)
None of these contain bearer credentials or secret material that the
existing ACL doesn't already gate. The walls are still the cascade.
zddc-server can now invoke podman as a CLIENT against a remote socket
instead of creating containers in its own process. The sidecar pattern
in tnd-zddc-chart will use this so zddc-server's own pod stays
unprivileged (only the podman-system-service sidecar runs privileged).
New surface:
--convert-podman-socket / ZDDC_CONVERT_PODMAN_SOCKET
e.g. unix:///var/run/podman/podman.sock
Empty (default) → local mode (podman creates containers in
zddc-server's own filesystem namespace).
Non-empty → remote mode: `podman --remote --url=<this> run …`
dispatches each container request to whatever process owns the
socket. Typically a `podman system service` sidecar in the same
Kubernetes pod.
--convert-scratch-dir / ZDDC_CONVERT_SCRATCH_DIR
Host-side directory for per-conversion intermediates (template,
HTML, PDF). In remote mode this MUST be a path the sidecar sees
at the same mountpoint — typically a shared emptyDir at /work
in both containers. Empty = $TMPDIR (local-mode default).
Runner behaviour:
local mode → unchanged. `podman run --userns=host --rm --pull=missing
--network=none --read-only …`. `--userns=host` stays so nested-podman
on a privileged host (the previous chart shape) keeps working for
anyone still using it.
remote mode → `podman --remote --url=<sock> run --rm --pull=missing
--network=none --read-only …`. `--userns=host` is dropped because
the sidecar is rootful inside its own privileged container and
doesn't need userns juggling.
Health probe gains a Mode field ("local" | "remote") and, in remote
mode, runs `podman --remote --url=<sock> version` to confirm the
sidecar's socket is reachable. Unreachable-socket → 503 with a clear
reason (sidecar may still be starting up); reachable → ready.
Capabilities log now includes engine_version + mode + remote_url for
easier debugging of "which podman is actually doing the work".
No tests removed — the existing fake-runner table covers both modes
since the runner's args are uniform (remote prefix is the only thing
that differs).
Flip default_tool from `mdedit` to `browse` (which now ships a Toast UI
markdown editor plugin in its preview pane) at:
• paths."*".paths.working
• paths."*".paths.working.paths."*" (per-user homes)
• paths."*".paths.reviewing
available_tools at those levels drops `mdedit` and adds `browse` next
to `classifier`. Operator overrides per .zddc cascade still work; only
the embedded baseline changes.
Test fixtures updated:
• lookups_test.go — DefaultToolAt assertions for working/+reviewing/
• availability_test.go — AppAvailableAt + DefaultAppAt for working/+
reviewing/+per-user home
• main_test.go — dispatch route asserts "ZDDC Browse" (was "ZDDC
Markdown"); Apps cascade fixture swaps mdedit
for browse so the live route fetches the right
embedded HTML
New endpoint GET /<path>/foo.md?convert=docx|html|pdf renders a markdown
source on demand. Surfaced as the Download buttons in browse's markdown
editor (separate commit).
Execution model — two upstream container images, lazy-pulled:
• docker.io/pandoc/latex:latest — MD→DOCX, MD→HTML (entrypoint pandoc)
• docker.io/zenika/alpine-chrome — HTML→PDF (entrypoint chromium-browser)
No custom image build. The runner passes --pull=missing on every podman/
docker invocation so the operator only needs the runtime installed —
first request pulls the image, subsequent requests use the local cache.
Overrides: --convert-pandoc-image / --convert-chromium-image (and the
matching ZDDC_CONVERT_* env vars). Engine: --convert-engine (podman
preferred, docker fallback). Resource caps: --convert-mem-mib (512),
--convert-cpus (2), --convert-pids (100), --convert-timeout (30s).
PDF flow is two-stage: pandoc renders the markdown through the embedded
viewer-template.html to standalone HTML, then chromium prints that HTML
via --print-to-pdf. Preserves the print-media CSS already authored in
viewer-template.html rather than going through pandoc's LaTeX template.
Each conversion runs in a throw-away container with --rm --network=none
--read-only --tmpfs=/tmp --cap-drop=ALL --security-opt=no-new-privileges
--env=HOME=/tmp plus a bind-mounted scratch dir for I/O. Pandoc reads
markdown from stdin / writes to stdout; the viewer template lives at
/tpl (ro). Chromium reads HTML from a read-write bind mount at /pdf
and writes the PDF to the same mount; the host reads it back. No shell
wrappers, no shell quoting — argv flows straight into each image's
entrypoint.
On-disk cache at <dir>/.converted/<base>.<ext> with mtime synced to the
source. Fast path is a stat-and-serve with no exec; slow path
singleflights concurrent requests for the same target. PUT/DELETE/MOVE
on the source .md purges the .converted/ sidecars.
Per-project template variables (client/project/contractor/project_number)
come from a new .zddc `convert:` cascade block, walked leaf→root with
per-key latest-wins. Filename-derived variables (title, tracking_number,
revision, status, is_draft) come from a new zddc.ParseFilename helper.
If neither podman nor docker is on PATH, the endpoint serves 503 with
a clear Retry-After. The rest of the server keeps working.
This is the first os/exec site in the codebase. The hardening in
internal/convert/runner.go — context.CancelFunc → process kill,
cmd.WaitDelay, platform-specific SysProcAttr (Setpgid + Pdeathsig on
Linux), minimal env, stdout cap via limitWriter, stderr ring buffer —
sets the pattern for any future shell-outs.
Public surface:
convert.ToDocx(ctx, source, meta) / .ToHTML / .ToPDF
convert.Probe(ctx, engineOverride) → install Runner if engine present
convert.SetImages(pandoc, chromium)
convert.ConfigureLimits(memMiB, cpus, pids, timeout)
convert.Available()
Container handler at internal/handler/converthandler.go; dispatcher
branch in cmd/zddc-server/main.go inserts the convert lookup after the
existing ACL gate, reusing the source file's read policy verbatim.
zddc-server can now hand back a whole directory subtree as a single
streamed application/zip download: GET /some/dir/?zip=1 (works on both
/dir and /dir/) → Content-Type: application/zip + Content-Disposition:
attachment; filename="<dir>.zip", containing every readable file under
/some/dir/, recursively.
handler.ServeSubtreeZip walks the tree with filepath.WalkDir, ACL-gates
each file by the .zddc chain of its containing directory (per-dir
decision cache, same shape as serveArchiveListing), skips hidden
entries ("." and "_" prefixes — .zddc, _template, _app), and adds a
.zip *file* it encounters as opaque bytes (it does not recurse into it
— that's the navigable-virtual-surface feature, a different thing).
The response is streamed (zip.NewWriter straight onto the
ResponseWriter, Store for already-compressed extensions, Deflate
otherwise), so a fully-ACL-denied or empty subtree just yields a valid
empty zip rather than a 403 (a stream can't change status after the
headers go out; empty leaks no more than 403). HEAD sends the headers
and no body. The dispatch's directory ACL gate still runs first, so a
viewer who can't read the directory gets 403 before the handler.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
zddc-server can now browse into a .zip file without the client
downloading the whole archive:
- GET …/Foo.zip/ → JSON listing of the zip's members
(Accept: application/json), or the
browse SPA (HTML) — same content
negotiation as ServeDirectory/.archive
- GET …/Foo.zip/sub/doc.pdf → extracts and streams that one member
(Range / ETag / conditional GET via
http.ServeContent)
- GET …/Foo.zip → unchanged: the raw .zip download
- PUT/DELETE/POST …/Foo.zip/… → 405 (zip access is read-only)
New internal/zipfs package reconstructs directory levels from the zip's
flat central directory (synthesising intermediate dirs with no explicit
"<dir>/" entry, mirroring what browse does client-side with JSZip) and
drops zip-slip-unsafe entries ("..", absolute, backslash). New
handler.ServeZip wraps it. The dispatcher gets splitZipPath + an
intercept placed before the file-API branch (so a write to a path under
a .zip is refused, not silently mkdir'd); ACL is the chain of the
directory CONTAINING the zip — a zip carries no .zddc of its own, same
as the .archive virtual surface. The os.Stat-per-segment walk is gated
by a cheap ".zip/" substring check so ordinary requests are unaffected.
Also fixes two pre-existing dispatch-test failures uncovered along the
way: a non-existent top-level "*.html" URL was 302'ing to its slash
form (because the bare "*" project glob makes every first-level segment
"declared") — the cascade-declared no-slash block now requires a
directory-shaped URL (trailing slash, or no file extension); and the
stale TestDispatchSlashRouting expectation that archive/<party>/mdl/
302s to mdl/table.html was updated to match the intended behaviour
(the default-MDL virtual fallback shows the browse listing there; only
a real on-disk tables: + *.table.yaml triggers the bounce).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The trailing-slash directory form was hardcoded to serve `browse`. Add a
`dir_tool` .zddc key (cascades leaf→root, floors at `browse`) so an
operator can point a subtree's slash form at another directory-oriented
tool — the symmetric counterpart to `default_tool` (the no-slash
"specialized app"). handler.ServeDirectory now resolves it via
zddc.DirToolAt; JSON listing requests are unaffected (raw listing
always served, so browse can still enumerate).
Also collapse the no-slash dispatch: the on-disk-directory and the
virtual-declared-path branches in main.go each carried their own copy
of "default_tool → tables-carveout-or-apps.Serve → 302", with
inconsistent ACL checks. Extract one chokepoint, serveSpecializedNoSlash,
that enforces ACL uniformly for every default_tool route.
Updates ARCHITECTURE.md and AGENTS.md: the stale "Special folders" /
hardcoded-availability sections now describe the .zddc-cascade model
(defaults.zddc.yaml, the schema-key table, the slash/no-slash
convention, WORM, standard roles).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Final consumer migration. The Go-coded lists that previously encoded
the ZDDC convention all defer to the .zddc cascade now.
Schema added:
available_tools: [tool1, tool2, ...] concat-union across cascade;
tools not in the union are
denied auto-route at that path
auto_own_fenced: true|false generated auto-own .zddc
carries inherit:false (private
to creator)
Lookups added:
AvailableToolsAt(root, dir) union of available_tools across cascade
IsToolAvailableAt(root, dir, tool)
AutoOwnFencedAt(root, dir) leaf-only
Cascade semantics finalised (per field):
default_tool → leaf→root walk (parent applies to descendants)
available_tools → leaf→root union (each level adds; baseline at root)
auto_own → leaf-only (creating THIS dir specifically)
auto_own_fenced → leaf-only (same)
virtual → leaf-only (THIS dir is virtual, not subtree)
Consumers migrated:
apps.DefaultAppAt → zddc.DefaultToolAt
apps.AppAvailableAt → zddc.IsToolAvailableAt (+ landing special)
EnsureCanonicalAncestors → AutoOwnAt + AutoOwnFencedAt
fs.ListDirectory empty-list fallback → zddc.IsDeclaredPath
fs.virtualCanonicalFolders → zddc.ChildrenDeclaredAt
dispatcher canonical-folder branches → unified into one
cascade-declared block
Hardcoded helpers REMOVED (dead code):
apps.inAncestorWithName
zddc.autoOwnDepthMatch / isAutoOwnDepthMatch
Hardcoded lists kept as data sources for the cascade walker but
no longer drive routing logic:
ProjectRootFolders / PartyFolders / AutoOwnCanonicalNames /
VirtualOnlyCanonicalNames / IsProjectRootFolder / IsArchivePartyFolder /
IsArchivePartyMdlDir — all still defined; only `ProjectRootFolders`
is used by special.go's IsProjectRootFolder. The rest are dead.
Dispatcher unified: the previously-two branches (per-party folder vs
project-root folder) collapse into one cascade-declared-path block
that handles the slash/no-slash convention uniformly:
- no-slash, default_tool=tables → ServeTable (default-MDL fallback)
- no-slash, default_tool set → apps.Serve(tool)
- no-slash, no default_tool → 302 to slash form
- slash, any → ServeDirectory empty-list fallback
The IsDir branch's switch also un-hardcoded — any cascade tool is
served (not just the legacy 3 names), so e.g. /Project/archive/<party>
/incoming (no slash) now serves classifier directly rather than 302'ing
to the slash form.
defaults.zddc.yaml populated with the canonical convention as the
recipe. Operators edit it (or override per-directory on disk) to
change any behaviour — no Go code changes required.
Browse drag-drop scope (working/staging/incoming) is the one remaining
client-side hardcoded regex; cascading that requires the cascade JSON
to be served to the client, which is its own Phase 4 piece.
Tests updated for the new no-slash mdl URL convention (landing MDL
card test) and no-slash stage URLs (nav strip test). All 248
Playwright + all Go tests green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
First step of the .zddc-first-configuration rollout: pure plumbing
that makes the future move-everything-out-of-Go work mechanically
possible without changing any current behaviour.
New pieces:
1. zddc/internal/zddc/defaults.zddc.yaml — a real YAML file in the
repo. Single source of truth for the baked-in baseline; intentionally
minimal in Phase 1 (just title + empty acl) so existing deployments
stay bit-identical until Phase 2 starts populating the schema.
2. //go:embed (defaults.go) bakes the bytes into the binary so
shipped deployments don't need the file. Operators who want a
starting point export with:
zddc-server show-defaults > /var/lib/zddc/root/.zddc
3. PolicyChain gains an Embedded ZddcFile field. EffectivePolicy
layers in the embedded defaults as a baseline below the on-disk
chain. Consumers that want the full effective view consult both;
existing consumers that only read chain.Levels keep working
bit-identically (the new field is additive).
4. New top-level `inherit:` key on ZddcFile. Default true. Set
`inherit: false` on any on-disk .zddc to zero out chain.Embedded
— the operator owns every rule from that level outward. Useful at
the on-disk root to fully reject the embedded defaults; useful at
deeper levels for sandbox subtrees.
5. `zddc-server show-defaults` (also accepts --show-defaults) subcommand
dumps the embedded bytes to stdout — same shape as --print-rego.
No flag plumbing needed beyond the existing args walk.
6. Tests: parse-roundtrip on the embedded file, presence in chain by
default, inherit:false drops it, explicit inherit:true is a no-op
versus the default.
Phase 2 (next): add a `paths:` recursive map + `default_tool:` /
`auto_own:` / `virtual:` keys, populate defaults.zddc.yaml with the
canonical ZDDC convention, and migrate apps.DefaultAppAt /
AutoOwnCanonicalNames / VirtualOnlyCanonicalNames to cascade lookups.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three coupled fixes:
1. landing MDL card: Open button now navigates to /<project>/archive/
<party>/mdl (no trailing slash) so the tables tool loads. The
slash form would route to browse instead, which is not what users
want when they click "Open MDL".
2. zddc-server canonical-folder fallback extended to
archive/<party>/{mdl,incoming,received,issued}. New
zddc.IsArchivePartyFolder() recognises any of the four party
folders at depth 4. fs.ListDirectory returns [] for missing
on-disk variants (mirroring the project-root behavior added in
commit 3fc3717); the dispatcher routes slash forms to
ServeDirectory and the no-slash mdl form to ServeTable, with
non-mdl no-slash forms 302'ing to the slash form.
So /Project-N/archive/<party>/incoming/ now lands on an empty
browse listing rather than 404 when nobody has dropped files yet.
3. Fixture seeded with 3 files per party under incoming/ — naming
intentionally NOT in transmittal-envelope form, so classifier
(loaded automatically by browse's grid mode at /incoming/
per the URL-driven view convention) has something to rename.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two related routing fixes:
1. /<project>/archive/<party>/mdl[/] now follows the slash/no-slash
convention uniformly with the rest of the system:
- mdl (no slash) → tables app (default tool for mdl/)
- mdl/ (slash) → browse (ServeDirectory empty-listing fallback)
Previously the slash form auto-redirected to mdl/table.html, which
forced the user into the table view from any party-folder click and
produced a confusing "Unrecognized table URL" error when the
redirect race-conditioned. tableRowsRedirect now only redirects
when a real on-disk table.yaml exists; the default-MDL virtual case
stays in browse via the convention.
New zddc.IsArchivePartyMdlDir helper recognises the canonical
<project>/archive/<party>/mdl pattern at depth 4 (relative path).
fs.ListDirectory uses it to return [] for the missing-on-disk case
so browse renders the empty workspace cleanly. Test updated
(TestServeDirectoryRedirectsDefaultMdl → TestServeDirectoryDefaultMdlNoRedirect).
2. <dir>/.zddc URLs now work at every directory depth.
The dispatcher previously 404'd anything beginning with a dot
(except /.archive and /<dir>/.zddc.html). New IsZddcFileRequest +
ServeZddcFile handlers carve out the raw .zddc leaf so an operator
can navigate to /Project-1/archive/PartyA/mdl/.zddc and inspect
the rules effective at that depth.
Semantics:
- Method: GET / HEAD only. Writes go through the existing admin-
gated form at <dir>/.zddc.html (unchanged).
- ACL: parent directory's read permission gates access; 404
(not 403) is returned to non-readers so existence isn't leaked.
- On disk: file bytes served verbatim with
Content-Type: application/yaml and X-ZDDC-Source: file:<rel>.
- Virtual: when no file exists at this level, a synthetic
placeholder body is returned with a YAML-comment cascade
summary so the reader sees exactly what rules apply here from
ancestors. X-ZDDC-Source: virtual:zddc distinguishes it.
The virtual body parses as valid YAML (`{}` after the comments) so
downstream tooling that consumes the URL isn't confused.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Four user-reported items:
1. landing: remove the standalone-tool strip from the site picker.
Per user, it was awkward — links pointing at zddc.varasys.io
releases from inside a deployment is a layering confusion. The
nav.tool-strip block in landing/template.html and its CSS are
gone.
2. zddc-server: route /Project/archive/<party>/mdl[/] to the tables
app for the virtual-MDL case where the on-disk folder doesn't
exist yet. Previously fell through to 404 because the dispatcher
only routed virtual mdl/ via the IsDir branch — the IsNotExist
branch was missing the equivalent check. Now both shapes (with
and without trailing slash) hit RecognizeTableRequest's default-
MDL fallback and ServeTable serves the embedded tables.html.
3. browse: re-layout the markdown editor to mirror mdedit's layout.
Was: sidebar on right with TOC top + front-matter bottom.
Now: sidebar on LEFT with YAML front matter top + Outline bottom,
content on RIGHT with an informational header (file title +
save controls + status + source) above the Toast UI editor.
New horizontal resizer between the front-matter and outline
sections inside the sidebar (drag the row boundary; arrow keys
step by 24 px). Browse test selectors updated.
4. zddc-server reviewing aggregator: extend to depth ≥ 2 so the
user can preview files inside virtual reviewing/<tracking>/
received/ and staged/ folders. IsReviewingPath now returns a
sidePath ("received[/rest]" or "staged[/rest]"); ServeReviewing's
depth-2 branch proxies the underlying real folder's listing,
emitting folder entries with virtual reviewing/ URLs (so
navigation stays in the aggregator) and file entries with
canonical archive/ or staging/ URLs (so byte fetches resolve
directly). ACL is enforced against the real path; depth-1
received/ + staged/ URLs are now virtual too (was canonical),
so the user smoothly descends into the depth-2 listing.
Tests updated for the new IsReviewingPath signature and the depth-1
URL shape.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Wires up live alpha-dev iteration on bitnest. With this change a
`.zddc apps: <tool>: <path>` entry overrides the embedded copy for any
of the eight tools, not just five.
Two coupled fixes:
1. zddc.AppNames had a five-entry list (archive/transmittal/
classifier/mdedit/landing) — predating browse/form/tables.
ResolveWithOverride's `if !IsKnownApp(app)` gate silently rejected
those three before ever looking at the cascade, falling back to
embedded with an "unknown app" error.
2. handler.ServeDirectory hard-coded `apps.EmbeddedBytes("browse")`
for the HTML directory-listing fallback, bypassing the apps
subsystem entirely. Now takes an optional *apps.Server and
delegates to appsSrv.Serve(w, r, "browse", chain, absDir) when
wired, so the cascade is honored at bare directory URLs too
(the most common way browse gets surfaced).
Both call sites in main.go and the test signatures in
directory_test.go updated. ValidateFile error message now lists all
eight known apps.
Verified end-to-end on bitnest with a root .zddc apps cascade
pointing at /srv/.zddc.d/source/<tool>/dist/<file>: every `./build`
on the host is now immediately visible after a hard refresh. Iteration
loop is `./build` (or `sh tool/build.sh`) then reload — no container
restart needed, since the apps subsystem reads the path source on
each request.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
301 Moved Permanently is cached by browsers effectively forever — when
we changed /<project> no-slash from "redirect to slash form" to
"serve project landing" earlier today, anyone who had visited the URL
under the prior behavior got stuck on the cached 301 indefinitely. No
server-side fix is possible after the fact; only a manual cache clear
in each user's browser releases the binding.
Demote every routing-shape redirect to 302 Found, which browsers do
not cache by default. Five sites:
- handler/directory.go: no-trailing-slash → slash on directory URLs
- main.go (4 sites):
.archive/ canonicalization (deep /<project>/<sub>/.../.archive/
path collapses to /<project>/.archive/)
reviewing/<tracking> no-slash → slash
reviewing/ default-app fallback to slash form
generic IsDir + no-slash + no-default-tool fallback
301 → 302 trades "permanent semantics in the protocol" for "we can
change our mind later without trapping users on old behavior." For
these routes — all of which are convention-driven shapes the server
owns — the latter is what we want.
Test updates: five httptest assertions switch from
http.StatusMovedPermanently → http.StatusFound, plus five comment
strings ("301" → "302").
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The /<project> landing page was server-rendered via
internal/handler/projecthandler.go's html/template — an inconsistency
against the project's "every tool is a single-file HTML" convention.
Convert it to a mode of the existing landing/ tool: same bundle now
serves both / (project picker) and /<project> (project workspace).
Mechanics:
- landing/template.html: pickerView (existing markup) + projectView
(new: stage cards, browse-all, MDL section, party-list slot).
Mode toggles by adding/removing .hidden on the two containers.
- landing/js/landing.js: detectMode() reads location.pathname;
renderProjectMode() populates stage hrefs from the project segment
and fetches /<project>/archive/?json=1 for the party list. init()
forks based on mode; picker init was extracted to initPicker().
Existing public API + behaviour unchanged for picker mode.
- landing/css/landing.css: appended ~115 lines for the project view
(.stages grid, .stage-card hover, .party-list, MDL formatting).
- cmd/zddc-server/main.go: dispatcher's IsProjectRootURL fork now
calls appsSrv.Serve(w, r, "landing", chain, absPath) rather than
the deleted ServeProjectLanding handler.
- internal/handler/projecthandler.go: trimmed to just the
IsProjectRootURL predicate (the dispatcher still needs it for
routing). Template + render code (~220 lines) deleted.
Net effect: same UI as before — same logo wrapping (now via
shared/logo.js, no longer a hand-rolled inline anchor), same stage
cards, same MDL instructions with party links — but the page is now a
single-file SPA that themes like the rest, follows the same logo and
stage-strip conventions, and could in principle be downloaded and
served standalone.
Tests:
- 3 new tests/landing.spec.js cases: detectMode exposure, project
workspace renders at /<project> with correct stage hrefs + title,
party listing populates from JSON fetch and filters dot-prefixed
entries.
- The dispatcher test for /Project no-slash still asserts 200 +
no-redirect; the served body is now landing.html instead of the
server-rendered template, but both pass the assertion.
LOC: roughly net-neutral. -220 in projecthandler.go, +115 in
landing.css, +130 in landing.js, +60 in template.html.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
GET /<project> (no trailing slash) used to 301 to /<project>/ which
served the browse listing. Now it serves a small server-rendered
landing page with:
- Four lifecycle-stage cards (archive/working/staging/reviewing)
linking to the no-slash form of each canonical folder, so each
card opens its default tool (archive view, mdedit sandboxed to
working/, transmittal at staging/, mdedit at reviewing/).
- A "Browse all files" link to the slash form for the generic
file tree.
- A "Master Deliverables List" section with step-by-step
instructions for editing any party's MDL plus direct links to
the MDL of each party already present under archive/ (sorted,
case-preserved). Falls back to a friendly "no parties yet"
message when the archive is fresh.
Trailing-slash form (/<project>/) is unchanged — still 200 +
embedded browse.html. The slash-vs-no-slash convention now extends
all the way up the URL tree:
/ → landing tool (project picker)
/<project> → project landing (this commit)
/<project>/ → browse
/<project>/working → mdedit
/<project>/working/ → browse
... etc.
Implementation:
- new internal/handler/projecthandler.go — IsProjectRootURL
predicate + ServeProjectLanding rendering an inlined html/template.
Page styles are inline; tokens mirror shared/base.css and
auto-flip on prefers-color-scheme: dark.
- dispatcher in cmd/zddc-server/main.go: at the IsDir branch's
no-slash fork, intercept depth-1 single-segment URLs before
the historical 301. Other depths still 301 unchanged.
Tests:
- internal/handler/projecthandler_test.go (4 cases): predicate
coverage; landing page renders project name + four stage cards;
on-disk parties surface as MDL links with case preserved; fresh
project falls back to the no-parties-yet copy.
- cmd/zddc-server/main_test.go TestDispatchSlashRouting: the
"project root no-slash → 301" case becomes "→ landing (200)".
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Apply the same slash-vs-no-slash convention to reviewing/ that already
governs the other three canonical project folders:
/<project>/reviewing → mdedit (default tool, via DefaultAppAt)
/<project>/reviewing/ → browse (HTML) — shows the aggregator's
virtual <tracking>/ entries as a tree
/<project>/reviewing/?json → aggregator JSON (handler.ServeReviewing)
Browse fetches the JSON listing for the URL it was loaded from, so
loading browse.html at /<project>/reviewing/ triggers a JSON request
back through the dispatcher → ServeReviewing → aggregator output.
Browse then renders the virtual <tracking>/ entries as clickable
folders. Clicking a tracking folder navigates to the per-submittal
view; clicking received/ or staged/ exits the virtual subtree
into canonical archive/ or staging/ paths via the polyfill's
explicit-url support.
The HTML branch in the reviewing dispatcher block was previously
calling appsSrv.Serve(..., "mdedit", ...) for trailing-slash URLs;
now it falls through to the canonical-folder block which routes to
ServeDirectory's HTML default (embedded browse.html).
Test: TestDispatchEmptyCanonicalProjectFolders extended with the
slash/<stage> → browse subtests, mirroring the no-slash → default
app set. All four canonical folders now have symmetric coverage of
both shapes.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Implements the reviewing/ aggregator described in the saved
project memory (~/.claude/projects/-home-user-src-zddc/memory/
project_reviewing_folder_design.md). reviewing/ stays in
VirtualOnlyCanonicalNames — never materialised on disk — and is
served as a join over archive/<party>/received/, archive/<party>/
issued/, and staging/, recomputed on every read.
Two depths, both trailing-slash:
GET <project>/reviewing/?json=1
→ array of virtual <tracking>/ entries, one per submittal in
archive/<party>/received/ that doesn't yet have a matching
archive/<party>/issued/ entry. Sorted by tracking. URLs stay
under reviewing/ so the user can drill into the per-submittal
view. ACL: per-party, filtered like fs.ListDirectory.
GET <project>/reviewing/<tracking>/?json=1
→ array of two virtual entries, received/ + staged/, with
canonical URLs pointing back to archive/<party>/received/...
and staging/... respectively. staged/ is omitted when no
response draft exists yet.
When the response moves staging/ → archive/<party>/issued/, the
entry vanishes from depth-0 on the next listing. No mutation of
the reviewing/ subtree itself; pure join, recomputed on read.
Front-end at <project>/reviewing[/<tracking>/] is mdedit (per
user request). DefaultAppAt + AppAvailableAt extended to recognise
"reviewing" as a canonical mdedit-bearing folder. The polyfill in
shared/zddc-source.js is updated to follow listing entries' explicit
url field when present (absolute or root-relative) — that's how
mdedit's tree follows the depth-1 received/ + staged/ links into
the canonical archive/staging subtrees.
Dispatcher routing in zddc-server/main.go:
- GET <project>/reviewing/[<tracking>/] with Accept: json
→ ServeReviewing
- GET <project>/reviewing/[<tracking>/] with Accept: html
→ mdedit (rooted at the virtual path; polyfill fetches the
JSON listing on its own)
- GET <project>/reviewing (no slash) → mdedit (via DefaultAppAt)
- GET <project>/reviewing/<tracking> (no slash) → 301 to slash form
Tests:
- handler/reviewinghandler_test.go (6 cases): IsReviewingPath
classification + ServeReviewing depth-0/depth-1 with and without
staged drafts + 404 on unknown tracking + empty when archive/ is
absent.
- apps/availability_test.go updated: reviewing/ now expects mdedit
rather than "" (no default).
- cmd/zddc-server/main_test.go: TestDispatchEmptyCanonicalProjectFolders
extended to assert reviewing → mdedit at the no-slash form;
older "no-slash/reviewing → 301" test removed.
Future work (not in this commit): write translation. Editing a file
under reviewing/<tracking>/staged/<f>.md works today because the
polyfill rewrites to /<project>/staging/<response>/<f>.md before
fetching — the user's URL bar moves to the canonical path on click.
A virtual-filesystem mode where the URL bar stays under reviewing/
throughout would require server-side write rewriting (translate
PUT/DELETE on reviewing/.../staged/... into the canonical staging/
path). Not needed for the MVP — links in mdedit's tree work.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
when missing on disk
Mirror of the existing IsDir-branch behavior at line 873
(<project>/working → mdedit, <project>/staging → transmittal,
<project>/archive → archive) for the case where the folder doesn't
exist on disk yet. Without this, GET <project>/working on a fresh
project 404s instead of opening mdedit rooted at the (virtual)
working directory.
Behavior matrix for canonical project-root folders that don't yet
exist on disk:
GET <project>/archive → archive tool (project-root mode)
GET <project>/archive/ → empty browse listing
GET <project>/working → mdedit rooted at working/
GET <project>/working/ → empty browse listing (with synthetic
<viewer-email>/ home entry)
GET <project>/staging → transmittal rooted at staging/
GET <project>/staging/ → empty browse listing
GET <project>/reviewing → 301 to /reviewing/ (no default app)
GET <project>/reviewing/ → empty browse listing
GET <project>/random → 404 (still — non-canonical)
GET <project>/random/ → 404 (still — non-canonical)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The previous fix in fs.ListDirectory was insufficient — main.go's
dispatcher calls os.Stat(absPath) before reaching ServeDirectory,
and 404s on the missing path before the listing code ever runs.
Symptom: GET <project>/working/ on a fresh project still returned
"Not Found" despite the read-side fallback being committed.
Add the same fallback at the dispatcher level: when os.Stat returns
NotExist AND the URL ends with "/" AND the path matches
IsProjectRootFolder, fall through to ServeDirectory rather than
404. ServeDirectory's ACL check + ListDirectory's empty-listing
behavior take it from there.
Separately, fs.ListDirectory now initializes its result slice to
make([]listing.FileInfo, 0) instead of `var result []listing.FileInfo`,
so the JSON encoder emits "[]" rather than "null" for empty
listings — clients (browse, archive) expect an array and choke on
null.
New test TestDispatchEmptyCanonicalProjectFolders covers the four
canonical names (archive/working/staging/reviewing) on a project
where none of them exist on disk yet, plus the negative case (a
non-canonical missing path still 404s).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two intertwined refactors that share too many files to split cleanly.
Both are described separately below.
PART 1 — in-dir convention for table+form spec files
Old layout had the spec at the parent and rows in a child:
archive/<party>/
mdl.table.yaml spec
mdl.form.yaml row-edit form
mdl/ rows-dir
row-001.yaml ...
URLs were /<dir>/mdl.table.html and /<dir>/mdl.form.html. Copying
mdl/ elsewhere lost the spec and form because they lived next door.
New layout collapses everything into the rows-dir:
archive/<party>/mdl/ self-contained
table.yaml spec
form.yaml row-edit form
row-001.yaml ... rows
URLs become /<dir>/mdl/table.html and /<dir>/mdl/form.html. The
"copying-the-folder-takes-everything" property the user asked for
falls out by construction; the row-edit URL /<dir>/<id>.yaml.html
keeps the same shape (spec is now in the same dir, not the
grandparent).
Server changes:
- internal/handler/tablehandler.go RecognizeTableRequest fires on
/<dir>/table.html when <dir>/table.yaml exists. The .zddc.tables
alias map is gone — pure presence-based discovery now matches
the form system's existing convention. Default-MDL fallback at
archive/<party>/mdl/ stays for the virgin-archive case (the
rows-dir need not exist on disk; the URL renders fully virtually).
- internal/handler/formhandler.go RecognizeFormRequest fires on
/<dir>/form.html and /<dir>/<id>.yaml.html with spec at
<dir>/form.yaml. specEligible accepts on-disk files OR the
default-MDL virtual path so an empty mdl/ dir still surfaces the
add-row form.
- internal/handler/tablehandler.go IsDefaultMdlSpec moves to
serving archive/<party>/mdl/{table,form}.yaml (5 segments after
ZDDC_ROOT). New isAtArchivePartyMdlLevel predicate; new
isAtArchivePartyMdlDir for directory-based recognition. New
IsDefaultMdlSpecAbs accessor for callers that hold an abs path
rather than a URL (formhandler).
- internal/handler/formhandler.go loadFormSpec(fsRoot, path) falls
back to embedded default-MDL bytes when os.ReadFile returns
NotExist AND the path matches the archive-party-mdl shape. Three
call sites updated to pass cfg.Root.
- internal/handler/formhandler.go serveFormCreate writes
submissions to filepath.Dir(req.SpecPath) — the spec, the form,
and rows all live in one directory. The submissionsDir creation
is idempotent (MkdirAll); cascade falls back one level for ACL
evaluation when the dir hasn't been materialized yet.
- internal/handler/tablehandler.go tableRowsRedirect now points at
/<dir>/table.html (was /<dir>.table.html) when the directory
request maps to a recognized table.
- cmd/zddc-server/main.go dispatch synth flips from
urlPath + ".table.html" to urlPath + "/table.html" for the
no-trailing-slash → tables-app routing.
- internal/apps/availability.go DefaultAppAt comment clarified
that the dir at archive/<party>/mdl/ IS the table (not a child).
Client changes:
- tables/js/context.js walkServer fetches <currentdir>/table.yaml
directly — no .zddc walk for table declarations. Rows are every
*.yaml in current dir EXCLUDING table.yaml and form.yaml. The
.zddc fetch-for-aliases is gated on file:// (online mode 404s
on .zddc reads via the dispatcher's reserve guard, so skipping
the request avoids browser console noise).
- tables/js/main.js add-row button links to relative form.html
(same dir).
- tables/js/render.js + filters.js: every column's autofilter is
uniformly a text-contains input, even enum columns — keeps the
filter row visually consistent and doesn't constrain users to
the enum vocabulary.
PART 2 — unified table+form HTML bundle
The form-render and table-render code paths share field schemas,
the cell editor for excel-mode IS a form widget, and the form
system's POST-back / validation already exists. Combining the two
HTMLs eliminates duplicating jsyaml/jsonschema/theme/source-
detection/.zddc-parsing across two single-file tools.
- tables/template.html grows two top-level mode containers:
#table-mode (toolbar + sortable table) and #form-mode (form +
submit button). Both hidden at parse time; the dispatcher
unhides one. The shared #form-context placeholder was added
here so the server's existing injectFormContext target
resolves.
- tables/js/mode.js (new) sets window.zddcMode synchronously
based on URL pattern: /form.html or /<id>.yaml.html → form,
/table.html → table, else inline-context fallback for
file:// (whichever context blob is non-empty wins). Unhides
the matching container at DOMContentLoaded.
- tables/js/main.js init() and form/js/main.js boot() each guard
early when mode isn't theirs. Both apps live on different
globals (window.tablesApp vs window.formApp) so module
registration doesn't collide.
- form/js/main.js title write falls back from #form-title to
#table-title (the unified bundle's shared header element)
when the dedicated id isn't present.
- tables/build.sh concatenates form modules (widgets, render,
object, array, errors, post, serialize, util) and form CSS.
No new external deps. Bundle grows from ~95KB to ~120KB.
- internal/handler/formhandler.go drops the //go:embed form.html
directive; serveFormRender now writes embeddedTablesHTML via
a small formRenderHTML() accessor (var declared in
tablehandler.go, same package). The embedded form.html file
is removed.
- build script: cp form/dist/form.html → internal/handler/form.html
step is gone (file no longer exists in the source tree). cp
tables/dist/tables.html → internal/handler/tables.html now
runs unconditionally rather than only on beta/stable cuts —
the renderer is a fixed binary component and dev iteration
needs the embedded copy refreshed every build. Channel-cascaded
apps (internal/apps/embedded/) stay channel-gated as before.
- form/dist/form.html still builds for standalone offline-only
use (downloadable from /releases/), but no longer goes into
the binary.
Tests:
- internal/handler/tablehandler_test.go and formhandler_test.go
rewritten for the in-dir layout. New test
TestRecognizeFormRequest_DefaultMdlAtArchiveParty covers
empty-form, create POST, re-edit row, and the negative cases
(Working/, non-mdl name) where the fallback must NOT fire.
- internal/handler/directory_test.go updated for the new
/<dir>/table.html redirect target.
- cmd/zddc-server/main_test.go TestDispatchSlashRouting Location
expectation updated.
- tests/form-safety.spec.js loads tables/dist/tables.html
(named form.html in the temp dir to trigger form-mode in the
dispatcher) so it tests the same bytes the server returns.
Title-element selector switches to #table-title.
- tests/tables.spec.js updates the status-filter test for the
uniform text-input filter.
Docs:
- AGENTS.md form-data system rewrites the URL conventions and
storage layout for in-dir; gains a Tables system section
parallel to forms describing the self-contained-directory
property; subfolder rules ("one table per folder by
construction; subfolders allowed and silently ignored as rows
— legitimate uses: nested sub-tables, per-row attachments,
drafts, future history sidecars") so we don't re-derive this.
Not included (deferred):
- ACL gating on cell-level writes — not relevant until Phase 3.
- Editable cells UI — separate commit (Phase 1).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
URLs are now case-insensitive against the on-disk casing under
ZDDC_ROOT, with a lowercase-wins tiebreak when sibling case variants
exist. File and folder names preserve case on disk — the change is a
pure URL→FS-name mapping; nothing renames anything.
internal/fs/resolve.go ResolveCanonical walks segments left-to-right
under fsRoot. Per segment: try lowercase first (canonical / cheap
lstat fast-path), then exact-case, then readdir+CI scan with the
all-lowercase variant winning the tiebreak. Walk stops at the first
segment that doesn't exist on disk so virtual prefixes (.archive,
.profile, .tokens, .auth) and 404 paths flow through with their tail
preserved verbatim. Path-escape safety check on the resolved abs
path matches the existing safeJoin pattern.
Wired in at the top of cmd/zddc-server/main.go dispatch(), which
rewrites r.URL.Path before any handler runs. Downstream handlers
(plus their existing safeJoin calls and the cascade walker) pick up
canonical case automatically — no per-handler changes. The ACL
cascade benefits from this for free since EffectivePolicy is keyed
by the now-canonical absolute path.
internal/handler/middleware.go AccessLogMiddleware snapshots the
as-typed URL path before the rewrite. The audit log's `path` field
records what the client actually sent; a `resolved_path` field is
added only when canonicalization changed it. Operators reading the
log can see both the raw request and what was served.
Lowercase as the project-wide canonical convention is already
honoured by the auto-created folders in internal/zddc/ensure.go
(working/, staging/, archive/<party>/incoming/) and the server's
own state dirs (_app/, .zddc.d/tokens/, .zddc.d/outbox/,
.zddc.d/logs/). Operators who drop a Mixed-Case-Folder/ on disk
keep that casing — the resolver finds it via the readdir tier.
Performance: the lowercase-first lstat is one syscall on the hot
path. Only mismatches (mixed-case URL where on-disk is also
mixed-case) pay the readdir+EqualFold scan, and Linux page-caches
small-dir readdirs aggressively. Apache mod_speling uses the same
"try then fallback" pattern.
Tests:
- internal/fs/resolve_test.go — 9 unit tests: exact-case,
mixed-case-URL-with-lowercase-on-disk, mixed-case-URL-with-
mixed-case-on-disk, both-cases-exist-lowercase-wins, nonexistent
segment preserves remainder, file-segment terminates walk, escape
rejection, trailing-slash normalization, root.
- cmd/zddc-server/main_test.go TestDispatchCaseInsensitiveURL —
end-to-end through the dispatcher with sibling Archive/ and
archive/ on disk; all four URL casings of the same path serve the
lowercase variant's content (proves the tiebreak fires through
every layer).
- Full Go suite green.
Docs: AGENTS.md gains a "URL handling" subsection in the
zddc-server section; ARCHITECTURE.md security-model table gains a
"URL canonicalization" row.
Out of scope (separate decisions, can revisit if needed):
- ACL glob CI-matching. If .zddc rules use mixed-case URL globs,
they won't match the canonical lowercase URL. Workable today by
writing rules in lowercase. Touches a different package.
- Redirect-to-canonical (303). Server serves under whichever case
the client used; canonicalization is internal. Could 301 to
canonical for SEO/bookmark hygiene as a follow-up.
- Client-mode (proxy/cache). Only master mode is wired so far.
Cache-handler CI lives in internal/cache/cache.go cachePathFor
and is a separate code path.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
PUT / POST / DELETE in client mode now work end-to-end. Online: the
cache layer forwards to upstream and (on success) drops any cached
entry for the path so the next read fetches fresh. PUT/DELETE include
If-Unmodified-Since derived from the cached file's mtime so the master
can reject conflicting writes with 412 Precondition Failed.
When upstream is unreachable, the request is captured in the outbox
at <root>/.zddc-outbox/<id>/ — directory per queued write, mode 0700,
containing meta.json (method, RawURI, Content-Type, base mtime,
queued-at) and body.bin (request body, capped at 256 MiB). The client
gets 202 Accepted + X-ZDDC-Cache: queued and a JSON envelope.
A background replay loop started by runClient processes the queue:
- 2xx → delete entry; drop cached path so next read fetches fresh
- 412 → rename to <id>.conflict-<RFC3339>/ for manual reconciliation
(body + meta intact for inspection or re-submit)
- 4xx other → drop (retry won't help; logged at WARN)
- 5xx / transport error → leave for next pass
Replay schedule: eager at startup, then 30s while pending falling
back to 5min while idle. Loop honors graceful-shutdown context.
Disabled in --mode=proxy (proxy persists nothing by design — offline
writes return 503 instead of queueing).
Outbox IDs are <unix-nano-base16>-<hex-random> so lex-sort = queue
order; concurrent enqueues never collide. Conflict-rename appends a
4-char random suffix on the unlikely same-second collision.
The local cache is intentionally not updated for offline writes:
until upstream confirms the user reads still see the upstream-cached
version (or 503 if uncached). Trade-off: no "did my queued write
actually win?" ambiguity, at the cost of not seeing one's own
offline edits immediately. Phase 5 will surface .conflict-<ts>/
directories in browse views.
Tests (20 new in outbox_test.go, 5 new in cache_test.go covering
the write path): NewOutbox creates 0700 dir, Enqueue persists meta
+ body, Pending returns lex-sorted entries excluding conflicts,
Replay deletes on 2xx / renames on 412 / leaves on transport error
/ leaves on 5xx / drops on 4xx-other, IUS sent only for PUT/DELETE
with base mtime, query string preserved, ServeHTTP online write
forwards + evicts cache, ServeHTTP offline write queues with 202,
ServeHTTP offline + no outbox returns 503, ServeHTTP PUT sends IUS
from cached mtime, oversize body rejected, IDs lex-sortable,
RunReplayLoop stops on context cancel, concurrent Enqueue 30×
no collisions. Full suite + go vet clean.
Doc updates: zddc/README.md gains a "Writes (online + offline
outbox)" subsection covering both paths and replay outcomes;
"What client mode is NOT, yet" now lists only conflict UI and
multi-tenancy. AGENTS.md client-mode pipeline gains writes +
mirror-mode bullets. ARCHITECTURE.md adds a "Writes: outbox +
offline replay" subsection with the trade-off rationale and the
phase-5-deferred conflict UI hand-off.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
