Two layers shipped together since the second builds on the first.
LAYER 1 — reviewing/ + Plan Review scaffolding
- reviewing/ is now a real folder under each project, populated by the
Plan Review composite endpoint. The old reviewing/ virtual aggregator
handler is retired.
- POST /<project>/archive/<party>/received/<tracking>/ with X-ZDDC-Op:
plan-review scaffolds physical workflow folders under reviewing_root
and staging_root, each carrying .zddc.received_path pointing back at
the canonical submittal. Idempotent re-runs match by received_path
and re-converge the ACL.
- Virtual received window: when listing or writing under
<workflow>/received/, the server resolves through the canonical
archive/<party>/received/<tracking>/ via the workflow's
.zddc.received_path. Writes get rewritten to
<workflow>/<base>+C<n><suffix> so review comments land in the
workflow folder and never touch the WORM archive.
- Cascade defaults declare on_plan_review per project so the
reviewing_root and staging_root are configurable.
LAYER 2 — browse context-menu workflows
- Accept Transmittal: right-click a transmittal folder in
archive/<party>/incoming/ → validates ZDDC folder + filename
conformance, atomic-renames the folder to
archive/<party>/received/<tracking>/ (WORM zone), and optionally
chains into Plan Review in the same composite request. Re-acceptance
with a different revision merges file-by-file; WORM forbids
overwrite of an existing filename.
- Stage / Unstage: right-click files in working/<…>/ → "Stage to…"
with picker of existing staging transmittal folders + inline
"New transmittal folder…" create; right-click files in
staging/<…>/ → "Unstage to working/" defaulting to the user's
working/<email>/ home. Reuses the file-API move primitive.
- Create Transmittal folder: right-click the staging/ pane → prompts
for a ZDDC-conforming folder name with live validation; mkdir,
then navigate to the new folder URL where the transmittal tool
serves the editor.
- Supporting infrastructure: new CanonicalFolderAt cascade lookup +
X-ZDDC-Canonical-Folder response header so the browse SPA can
scope-gate menu items without re-implementing the cascade
client-side.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three coordinated changes that share the same files. Common theme:
convention beats exception. Where the codebase had a bespoke wire shape
or a special-case route, replace it with the generic shape every other
client already speaks.
== Listing protocol ==
GET / Accept: application/json used to dispatch to a bespoke
ServeProjectList handler returning {name, url, title} per project — a
shape that diverged from every other directory's listing.FileInfo
response. Now:
- listing.FileInfo gains an optional `title` field (read from each
directory's own .zddc title:). Generic clients (landing, browse)
read the same shape from every URL.
- appfs.ListDirectory emits a virtual `.zddc` entry (is_dir:false,
virtual:true) when no on-disk file exists at that path and the
caller asked for ?hidden=1. Opens an editable view of the cascade
defaults; PUT-saving its bytes materialises a real file.
- The bespoke GET / JSON branch in cmd/zddc-server/main.go is gone.
The bare-root landing serve is Accept-gated: HTML requests get the
landing tool (project picker), JSON requests fall through to
ServeDirectory and get the generic listing.
- landing's fetchProjects filters the new generic shape (is_dir,
strip trailing slash) — same pattern fetchParties already used at
/<project>/archive/.
== Form editor retirement ==
`<dir>/.zddc.html` was a server-rendered form for editing per-directory
.zddc files (~900 LOC across zddceditor.go, zddchandler.go, zddc_assets.go).
Browse's YAML/CodeMirror editor (with .zddc-schema lint) already edits
the same files via the generic file-API. Two ways to edit the same data
is exception, not convention.
- Delete zddceditor.go, zddchandler.go, zddc_assets.go and tests.
- `/<dir>/.zddc.html` → 302 redirect to `/<dir>/?file=.zddc` (browse
opens the .zddc in its editor pane).
- /.profile/zddc/* namespace deleted (REST API + assets sub-route).
- Profile page's "Editable .zddc files" list links to browse.
- ServeZddcFile's 405 message + virtual-body comment point at the
browse URL instead of the dead form.
== Admin elevation (Principal model) ==
Sudo-style: admins are treated as normal users by default; opting into
admin powers is per-request and gated by a `zddc-elevate=1` cookie.
- zddc.Principal{Email, Elevated} replaces bare-email arguments on
IsAdmin / IsSubtreeAdmin / CanEditZddc. The signature change makes
the elevation gate compiler-enforced at every admin call site —
audit-fragility is gone. The empty-email short-circuit is no longer
load-bearing for elevation; Principal.gate() is the explicit check.
- handler.ACLMiddleware derives Elevated per request: bearer tokens
are implicitly elevated (CLI clients can't toggle a cookie); browser
sessions elevate only when zddc-elevate=1 is set. PrincipalFromContext(r)
is the one-call-per-site bundling helper.
- Every admin-check call site updated to pass a Principal.
- /.auth/admin (forward_auth target for the dev-shell IDE) explicitly
bypasses elevation with a synthetic-elevated Principal — different
cookie scope than zddc-server origin, documented inline.
- AccessView gains CanElevate (elevation-independent "does this email
have admin authority anywhere?") so the header toggle can render
itself for an un-elevated admin who hasn't opted in yet.
- ServeProjectList is removed; ProjectInfo + EnumerateProjects stay
for the profile page's server-rendered project list.
- MatchAppHTML stays — still used by main.go to route <dir>/<tool>.html
URLs to the apps subsystem when no real file exists.
- Test helpers carry Elevated=true by default (matches the
pre-elevation default; tests for the un-elevated gate use the
explicit form).
Go tests pass across all 14 internal packages. Browse + every other
tool rebuilds clean.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
New endpoint GET /<path>/foo.md?convert=docx|html|pdf renders a markdown
source on demand. Surfaced as the Download buttons in browse's markdown
editor (separate commit).
Execution model — two upstream container images, lazy-pulled:
• docker.io/pandoc/latex:latest — MD→DOCX, MD→HTML (entrypoint pandoc)
• docker.io/zenika/alpine-chrome — HTML→PDF (entrypoint chromium-browser)
No custom image build. The runner passes --pull=missing on every podman/
docker invocation so the operator only needs the runtime installed —
first request pulls the image, subsequent requests use the local cache.
Overrides: --convert-pandoc-image / --convert-chromium-image (and the
matching ZDDC_CONVERT_* env vars). Engine: --convert-engine (podman
preferred, docker fallback). Resource caps: --convert-mem-mib (512),
--convert-cpus (2), --convert-pids (100), --convert-timeout (30s).
PDF flow is two-stage: pandoc renders the markdown through the embedded
viewer-template.html to standalone HTML, then chromium prints that HTML
via --print-to-pdf. Preserves the print-media CSS already authored in
viewer-template.html rather than going through pandoc's LaTeX template.
Each conversion runs in a throw-away container with --rm --network=none
--read-only --tmpfs=/tmp --cap-drop=ALL --security-opt=no-new-privileges
--env=HOME=/tmp plus a bind-mounted scratch dir for I/O. Pandoc reads
markdown from stdin / writes to stdout; the viewer template lives at
/tpl (ro). Chromium reads HTML from a read-write bind mount at /pdf
and writes the PDF to the same mount; the host reads it back. No shell
wrappers, no shell quoting — argv flows straight into each image's
entrypoint.
On-disk cache at <dir>/.converted/<base>.<ext> with mtime synced to the
source. Fast path is a stat-and-serve with no exec; slow path
singleflights concurrent requests for the same target. PUT/DELETE/MOVE
on the source .md purges the .converted/ sidecars.
Per-project template variables (client/project/contractor/project_number)
come from a new .zddc `convert:` cascade block, walked leaf→root with
per-key latest-wins. Filename-derived variables (title, tracking_number,
revision, status, is_draft) come from a new zddc.ParseFilename helper.
If neither podman nor docker is on PATH, the endpoint serves 503 with
a clear Retry-After. The rest of the server keeps working.
This is the first os/exec site in the codebase. The hardening in
internal/convert/runner.go — context.CancelFunc → process kill,
cmd.WaitDelay, platform-specific SysProcAttr (Setpgid + Pdeathsig on
Linux), minimal env, stdout cap via limitWriter, stderr ring buffer —
sets the pattern for any future shell-outs.
Public surface:
convert.ToDocx(ctx, source, meta) / .ToHTML / .ToPDF
convert.Probe(ctx, engineOverride) → install Runner if engine present
convert.SetImages(pandoc, chromium)
convert.ConfigureLimits(memMiB, cpus, pids, timeout)
convert.Available()
Container handler at internal/handler/converthandler.go; dispatcher
branch in cmd/zddc-server/main.go inserts the convert lookup after the
existing ACL gate, reusing the source file's read policy verbatim.
The file API's mkdir post-hook still seeded auto-own .zddc files via the
hardcoded IsAutoOwnPath path-segment predicate, while
EnsureCanonicalAncestors had already moved to the cascade's auto_own:
flag. Point the hook at AutoOwnAt / AutoOwnFencedAt so both paths agree
and an operator's .zddc reshaping actually takes effect — fenced when
the new directory's own cascade level declares auto_own_fenced (per-user
working homes), unfenced otherwise.
Retires IsAutoOwnPath and WormMask (the latter already superseded by
WormZoneGrant's & VerbsRC) plus their tests, and the now-unused
path/filepath import in special.go.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Phase 3 + 4 live two-instance smoke tests against the synthetic
~/zddc-test-data fixture surfaced three real bugs that the unit
tests missed. All three are fixed in this commit.
1. walker: filenames with spaces/parens land on disk percent-encoded
walkSubtree was passing the URL-encoded child URL (built via
url.PathEscape) to fetchFileIfNeeded → cachePathFor, so a file
named "Foo (IFI) - Bar.md" landed at <root>/.../Foo%20%28IFI%29
%20-%20Bar.md on disk. Then purgeOrphans iterated os.ReadDir
(which sees the encoded names) and compared against upstreamNames
(decoded names from the listing JSON). Every fetched file was
classified as an orphan and immediately deleted: a 180-file walk
produced "fetched=180 purged=111" with only 70 files remaining.
Fix: walker now maintains two parallel path strings — dirURL
(URL-encoded for HTTP requests) and dirPath (decoded for disk
keys). fetchFileIfNeeded, fetchListing, persistOnly, and
purgeOrphans all take the decoded path. listingCachePathFor
gets dirPath too. Smoke confirmed: dirs=29 files=180 fetched=179
purged=0 (one file already cached from the user's GET that
triggered the walk).
2. outbox: replay loop sleeps 5min after eager startup pass
RunReplayLoop's idle-poll interval is 5min. After the eager
startup pass with 0 entries, the loop sleeps 5min — even if a
PUT-while-offline arrives 1 second later, replay won't fire for
~5 min. The cache returned 202 promptly but the queued write sat
on disk until either a 5min nap elapsed or another PUT happened.
Fix: Outbox gains a wake chan (buffered=1, drop-on-full).
Enqueue posts to it after writing meta.json. RunReplayLoop selects
on wake alongside the timer, so a new offline write triggers an
immediate replay attempt. Smoke confirmed: PUT queued at T+0,
master back at T+3, replay completes at T+3 (was previously a
30s wait through the timer-based poll).
3. master: PUT/DELETE didn't honor If-Unmodified-Since
The cache's outbox sends If-Unmodified-Since: <cached-mtime> on
replay so the master can reject conflicting writes with 412. The
master's checkIfMatch only evaluated If-Match (ETag-based), so
the cache's mtime-based precondition was silently ignored. Result:
an offline PUT staged before an external mod would clobber the
newer external content on replay — silent data loss in the exact
scenario the outbox is designed to detect.
Fix: checkIfMatch now also evaluates If-Unmodified-Since per
RFC 7232 §3.4, returning 412 when the file's current mtime is
strictly later than the header value (1-second resolution to
match HTTP-Date precision). Smoke confirmed: cache GET → external
mod via direct file write → cache offline PUT → master back →
replay sends IUS → master 412 → outbox entry renamed to
<id>.conflict-<RFC3339>/ → master content preserved (the
external mod, not the stale offline write).
Also added an info-level "outbox: replay attempt" log to tryReplay
so an operator watching the cache logs sees the replay loop is
alive even when every entry defers (transport error). Previously
the loop was silent unless a replay actually completed (200) or
conflicted (412).
go vet + go test ./... + go test -race ./internal/{cache,auth,handler}/...
all green. Synthetic ~/zddc-test-data fixture (553 files, 144 PDFs)
exercises the walker against realistic ZDDC filenames including
spaces, parens, and accented characters that the unit tests'
"a.txt" / "b.txt" inputs never hit.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
When a folder is created under <project>/staging/ whose name parses as a
ZDDC transmittal folder (YYYY-MM-DD_<tracking> (<status>) - <title>) and
whose tracking number contains -TRN- or -SUB-, also create the same-
named folder under <project>/working/ as a drafting space for staff.
The mirror is one-way and one-shot: created at staging-mkdir time only.
Renames and deletions of either side are not propagated. The
transmittal client orchestrates cleanup at issue time (move files to
archive/<recipient>/issued/, then delete both staging and working
siblings) — the server stays out of that decision.
-MDL- tracking deliberately skips the mirror; MDL deliverables live in
archive/<party>/mdl/ rows, not via the working↔staging pairing.
Implementation: mirrorStagingToWorking() in fileapi.go, called after a
successful serveFileMkdir. EnsureCanonicalAncestors handles working/'s
own auto-own .zddc; the mirror folder gets its own creator-grant on top.
Six new tests cover -TRN-/-SUB- mirror, -MDL- skip, non-transmittal
name skip, deep-path skip, and idempotency over a pre-existing sibling.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
New helper pair:
- ResolveCanonicalPath(fsRoot, target) — case-fold path resolution, no side effects
- EnsureCanonicalAncestors(fsRoot, target, email…) — case-fold + MkdirAll + auto-own .zddc seeding
For each canonical position along the requested path the helpers
substitute on-disk casing (so /Project/working/foo lands in an existing
Working/ rather than a new sibling) and materialise missing
working/staging/archive/<party>/{mdl,incoming,received,issued}/ folders.
working/, staging/, and archive/<party>/incoming/ get a creator-owned
.zddc seeded automatically; received/, issued/, and mdl/ are created
without auto-own (WORM and data-store concerns respectively).
reviewing/ is rejected — purely virtual, never on disk.
Wired into the file API:
- serveFilePut — resolve before auth, ensure after auth
- serveFileMkdir — resolve before auth, ensure after auth, with
two auto-own checks (target-is-canonical OR
parent-is-canonical)
- serveFileMove (POST) — resolve src+dst, ensure dst before rename so
a move from working/<draft> →
archive/<recipient>/issued/<draft> creates
the per-party folders on the way in
7 new unit tests in zddc/internal/zddc/ensure_test.go cover lazy
creation, case-fold reuse, per-party incoming auto-own, WORM no-auto-own,
empty-principal skip, reviewing rejection, and traversal rejection.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
BREAKING CHANGE. Project-level Issued/Received/Incoming folders no
longer carry special semantics. WORM enforcement and auto-ownership
move to the per-party canonical layout:
- WORM mask now triggers on archive/<party>/received/ and
archive/<party>/issued/ (any case, any party)
- Auto-own .zddc writes on first mkdir under working/, staging/,
or archive/<party>/incoming/ (any case)
Predicate API:
- IsAutoOwnPath(parentDir, fsRoot) — replaces IsAutoOwnParent(name)
- IsWormPath(requestPath) — same name, new pattern
- WormFolderLevelIndex unchanged signature, new pattern
Legacy SpecialFolderNames / AutoOwnFolderNames / WormFolderNames /
IsAutoOwnParent are deleted (no Deprecated: stubs — early-development
project, no back-compat to preserve).
Tool availability (apps/availability.go) is case-fold throughout:
- mdedit: descendants of working/
- transmittal: descendants of staging/
- classifier: descendants of working/, staging/, or
archive/<party>/incoming/
Working/, WORKING/, working/ all match identically.
Test fixtures rewritten:
- special_test.go: covers IsAutoOwnPath / IsWormPath /
WormFolderLevelIndex / ResolveCanonical / canonical lists
- availability_test.go: per-party rules, case-fold scenarios
- fileapi_test.go: rolePermissionsTestSetup now seeds
Project-X/archive/Acme/{incoming,issued,received}/ rather than
Vendor/{Incoming,Issued,Received}/ at the project root
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Pure refactor. The mkdir post-hook in handler/fileapi.go duplicated
zddc-package types; lifting the body into the package itself lets the
upcoming EnsureCanonicalAncestors helper share it without re-exposing
the file API's internals.
No behaviour change. The grant shape (creator email → rwcda + CreatedBy
audit field) and the atomic-write path through zddc.WriteFile are
unchanged.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replaces the binary acl.allow/deny model with five permission verbs
(r/w/c/d/a) and first-class roles, and adds an authenticated file API
(PUT/DELETE/POST move/mkdir) so the HTML tools can edit-in-place over
HTTP. Closes the AC-3(7) and AC-6 federal-readiness gaps.
File API (zddc/internal/handler/fileapi.go)
- PUT <new> → action c
- PUT <existing> → action w
- PUT <.zddc> → action a (CanEditZddc strict-ancestor rule)
- DELETE → action d
- POST mkdir → action c (auto-writes creator-owned .zddc when the
parent is Incoming/Working/Staging)
- POST move → action w on src + c on dst, atomic via os.Rename
- Optional If-Match for optimistic concurrency, --max-write-bytes cap,
audit log emits a structured file_write event per operation.
Permission model (zddc/internal/zddc/{acl,file,roles,cascade_mode}.go)
- acl.permissions: { principal → verb-set } map; principals are email
patterns or role names. Empty verb set is an explicit deny.
- roles: { name → members } definitions, available at the level they
declare and all descendants. Closer-to-leaf shadows ancestor.
- Legacy acl.allow/deny still work; they fold into permissions at
parse time (allow → "rwcd", deny → "").
- Cascade walks leaf→root; first level with any matching entry wins;
the union of matching verb sets at that level decides.
- --cascade-mode=strict adds a root→leaf ancestor-deny pre-pass so an
ancestor explicit-deny is absolute (NIST AC-6). Default delegated
preserves the existing commercial behavior.
Special folders (zddc/internal/zddc/special.go)
- Incoming / Working / Staging: mkdir auto-writes a .zddc into the new
subdir granting created_by + that email rwcda directly. Same form
operators write by hand; creator can edit it later to add others.
- Issued / Received: server-enforced WORM split. Cascade grants
inherited from above the WORM folder are masked to r only; grants
placed at-or-below the WORM folder retain r,c. Operators grant
write-once (cr) to the doc controller via an explicit .zddc at the
Issued/Received folder. Admins exempt — only escape hatch.
Browser polyfill (shared/zddc-source.js)
- HttpDirectoryHandle + HttpFileHandle implement the FS Access API
surface (values, getFileHandle, createWritable, removeEntry,
queryPermission/requestPermission) over zddc-server's listing JSON
and file API. Existing tools written against showDirectoryPicker
work unchanged.
- detectServerRoot() returns { handle, status }: tools auto-load on
HTTP, surface a clear "no permission to list" message on 403, and
fall back to the welcome screen on 0.
- classifier renames take the atomic POST move path on HTTP-backed
handles; mdedit and transmittal route reads/writes through the
polyfill so prior FS-API code paths cover both modes.
Tests
- zddc/internal/zddc/{cascade_mode,roles,special,acl}_test.go cover
delegated vs strict, role membership / shadowing / legacy fallback,
WORM split semantics, verb-set parser round-trip.
- zddc/internal/handler/fileapi_test.go now also covers role-based
vendor scenarios, WORM blocking vendor & doc controller writes,
explicit Issued .zddc unlocking the cr drop-box, admin bypass,
auto-ownership on mkdir, and strict-mode lockouts.
Docs
- ARCHITECTURE.md + zddc/README.md document the verb model, role
syntax, special-folder behaviors, cascade-mode flag, and full file
API surface. Federal-readiness gap analysis strikes AC-3(7) and
AC-6.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
