VARASYS/ZDDC
1
1
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
ZDDC/zddc/README.md
ZDDC 707f1d8ec2 feat(client): mirror mode — access-triggered subtree walker + listing cache 
--mode mirror layers an access-triggered walker on top of the cache
pipeline. When an incoming request's URL falls under one of the
configured --mirror-subtree paths, the scheduler kicks off a recursive
walk of that subtree iff (a) no walk for that subtree is in flight and
(b) now - last_walk_at >= --mirror-min-interval (default 1h). Walks
run in a goroutine; the user's request never blocks on scheduling.

Why access-triggered: a naive "walk on a fixed timer" would produce
thundering-herd polls on a master from many vendor mirrors most of
which are idle most of the time. Demand-triggering means idle mirrors
generate zero upstream traffic until someone hits them; active
mirrors stay current as a side effect of normal use.

The walk:
1. Recursively fetches JSON listings under the subtree, persisting
   each at <dir>/.zddc-listing.json so directory browsing works
   offline for walked subtrees.
2. For each file, fires a conditional If-Modified-Since GET (bounded
   parallelism; default 4 concurrent) — 304 no-op, 200 overwrites,
   403/404 purges the local cache.
3. After enumeration, per-directory orphan purge: local files absent
   from upstream's filtered listing are removed (handles upstream
   deletes + ACL revocations).

State persists at <root>/.zddc-mirror-state.json as
{subtrees: {<path>: {last_walk_at}}}. In-flight tracking is in-memory
only — a crash mid-walk lets the next access retry without manual
cleanup. Subtree path matching is longest-prefix-wins; "/" is a
catch-all (full mirror, the default when --mode=mirror is set without
explicit --mirror-subtree).

The cache layer also gained directory-listing caching (independent of
mirror mode but enabled by it). Directories are now stored at
<dir>/.zddc-listing.<html|json> sidecars, varied by Accept header.
Hit/miss/offline semantics mirror the file pipeline. Phase 2's
limitation that directories always proxied live (no offline browse)
is now resolved for any directory the user has visited or that mirror
mode has walked.

Mirror scope falls out of auth: the walker uses the local instance's
bearer, so it sees exactly what the user can see at upstream. Admin
bearer → full mirror; vendor bearer → vendor's permitted subtree;
no code distinguishes the cases.

New flags (also as ZDDC_* env vars), ignored when --mode != mirror:
- --mirror-subtree <csv> — repeatable subtrees (comma-separated);
  empty + --mode=mirror = "/" (full mirror)
- --mirror-min-interval <duration> — default 1h

Tests (15 new in walker_test.go, 3 new in cache_test.go): subtree
normalization, longest-prefix matching, root-as-catch-all, walk
fetches all files in scope, out-of-scope URLs are no-op, rate-
limiting prevents double-walks within min-interval, walks re-fire
after interval elapses, orphan purge removes local-only files,
state file survives restart, concurrent triggers don't double-walk,
end-to-end ServeHTTP-kicks-mirror-on-access, listing format varies
by Accept, listing offline serves stale, persisted state atomic
write + corrupt-input handling. Full suite + go vet clean.

Doc updates: zddc/README.md flags table gains the two new entries
plus a "Mirror mode (access-triggered subtree walker)" subsection
with trigger semantics and properties; the "What client mode is NOT,
yet" list shrinks accordingly. AGENTS.md env-var table gains the
two new entries. ARCHITECTURE.md "Master + proxy/cache/mirror"
section now documents the walker scheduler / walk algorithm / state
file in a "Mirror walker (access-triggered)" subsection.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-08 08:11:30 -05:00

1789 lines
90 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# zddc-server
A purpose-built HTTPS file server for ZDDC document archives. Designed to replace
`caddy file-server --browse` with features specific to ZDDC workflows.
## Features
- **High-performance static file serving** — ETag, conditional GET, Cache-Control
- **ETag on embedded tool HTMLs** — sha256 of the embedded bytes; repeat loads return 304 Not Modified instead of re-shipping 50–920 KB
- **gzip compression middleware** — wraps the entire mux; ~75% size reduction on tool HTMLs and JSON listings (skips bodies under 1 KB)
- **Public landing page** — root `/` is reachable by anyone, including anonymous; per-project ACL filtering still hides projects the caller can't reach
- **Cascading `.zddc` ACL** — email-based allow/deny lists evaluated bottom-up from requested directory to root
- **Caddy-compatible JSON listings** — the Archive Browser works without modification
- **Virtual `.archive` index** — resolve the earliest revision of any tracked document by URL
- **Filesystem watcher** — archive index updates automatically when files change
- **File-based audit log** — JSON-line access log tee'd to `<ZDDC_ROOT>/.zddc.d/logs/access-<host>.log` by default, rotated by lumberjack (100 MB / 10 backups / 90 days, gzipped)
- **Conservative HTTP timeouts** — slowloris-resistant; 10 s read-header, 60 s read+write, 120 s idle
- **Flexible TLS modes** — self-signed, real certificates, or plain HTTP
- **Single static binary** — CGO-free, no runtime dependencies; cross-compiled to Linux/macOS/Windows
## Quick Start
zddc-server ships as a cross-compiled binary distributed via Codeberg release assets.
```sh
# Pick a tag from https://codeberg.org/VARASYS/ZDDC/releases (filter by zddc-server-v*)
curl -L -o zddc-server \
https://codeberg.org/VARASYS/ZDDC/releases/download/zddc-server-vX.Y.Z/zddc-server-linux-amd64
chmod +x zddc-server
# Run against your archive root (HTTPS on :8443 with an in-memory self-signed cert)
ZDDC_ROOT=/srv/archive ./zddc-server
```
Or build from source (requires Go 1.24+):
```sh
git clone https://codeberg.org/VARASYS/ZDDC.git
cd ZDDC/zddc
go build -o zddc-server ./cmd/zddc-server
ZDDC_ROOT=/srv/archive ./zddc-server
```
For plain HTTP behind a reverse proxy, set `ZDDC_TLS_CERT=none` and `ZDDC_INSECURE_DIRECT=1` — see "TLS" below.
There is no Containerfile / Dockerfile / compose file in this repo. Two ways to run zddc-server in Kubernetes / containers:
- The example Helm charts under [`helm/`](../helm/) (`zddc-server-prod/` for stable / `zddc-server-dev/` for tracking main HEAD) compile zddc-server from source via init container — no image registry needed.
- Roll your own image: copy the static binary into a `FROM scratch` or `FROM alpine` base in a few lines.
## Environment Variables
| Variable | Default | Description |
|---|---|---|
| `ZDDC_ROOT` | *(required)* | Absolute path to the served file tree |
| `ZDDC_ADDR` | `:8443` | Bind address (host:port) |
| `ZDDC_TLS_CERT` | *(empty)* | Path to PEM certificate file. `none` = plain HTTP (no TLS); empty = generate self-signed |
| `ZDDC_TLS_KEY` | *(empty)* | Path to PEM private key file. Required when `ZDDC_TLS_CERT` is a file path; ignored otherwise |
| `ZDDC_INSECURE_DIRECT` | *(empty)* | Must be `1` when `ZDDC_TLS_CERT=none` and the bind address is non-loopback. Acknowledges that an authenticating reverse proxy is in front of zddc-server; without it, plain-HTTP non-loopback startup is refused |
| `ZDDC_INSECURE` | *(empty)* | Must be `1` to allow startup when `<ZDDC_ROOT>/.zddc` is missing. Without it, the server refuses to start because no `.zddc` files anywhere → public-by-default access. Set only for deliberately-public deployments |
| `ZDDC_OPA_URL` | `internal` | Policy decider endpoint. `internal` = built-in Go evaluator (default). `http(s)://...` or `unix:///...` = external OPA-compatible server (federal deployments using their own audited Rego). See "External policy decider" below. |
| `ZDDC_OPA_FAIL_OPEN` | *(empty)* | External OPA only. `1` = on transport error, allow the request (availability over correctness). Default = fail closed (deny). Never set to `1` in federal contexts. |
| `ZDDC_OPA_CACHE_TTL` | `1s` | External OPA only. Per-decision cache TTL — bursts of identical queries (a single `.archive` listing can hit the same `(email, dir)` tuple many times) collapse to one OPA round-trip. Set `0` to disable. Format is Go's `time.ParseDuration` (`500ms`, `2s`, `1m`). |
| `ZDDC_APPS_PUBKEY` | *(empty)* | Path to a PEM-encoded Ed25519 public key used to verify signatures on URL-fetched `apps:` artifacts. Empty (default) = URL-fetched apps refused; only embedded + local-path apps work. Operators using zddc.varasys.io's canonical channels download `pubkey.pem` from there and pass the local path here. Operators with their own signing infrastructure pass their own public key. Same posture as `ZDDC_TLS_CERT` — zddc-server bakes nothing in. **Alternative inline form:** the same key can be pasted as `apps_pubkey:` in the root `<ZDDC_ROOT>/.zddc` (root-only, like `admins:`). The env/flag wins when both are set. |
| `ZDDC_LOG_LEVEL` | `info` | Log level: `debug`, `info`, `warn`, `error` |
| `ZDDC_INDEX_PATH` | `.archive` | URL path segment name for the virtual archive index |
| `ZDDC_EMAIL_HEADER` | `X-Auth-Request-Email` | HTTP request header containing the authenticated user's email (the oauth2-proxy / nginx auth-request convention) |
| `ZDDC_CORS_ORIGIN` | *(empty)* | Comma-separated allowlist of origins permitted to make cross-origin requests. Empty (default) disables CORS — appropriate when zddc-server's embedded tools serve same-origin. Set explicitly only if browser-loaded pages from a different origin call back into this server (e.g. `https://tools.acme.com` for self-hosted tools, or `https://zddc.varasys.io` for the CDN-bootstrap pattern) |
| `ZDDC_ACCESS_LOG` | `<ZDDC_ROOT>/.zddc.d/logs/access-<host>.log` | Tee'd structured access log. Auto-mkdir on first run. Empty value (set explicitly with `--access-log=`) disables file logging; stderr stream stays. Per-host filenames let multiple replicas write to the same `.zddc.d/` directory without collision; every record carries a `host` field for downstream aggregation. |
`ZDDC_TLS_CERT=none` disables TLS entirely (plain HTTP). Both cert and key must be set together when using real certificates.
### CORS
CORS is **disabled by default** — `ZDDC_CORS_ORIGIN` defaults to empty.
The embedded-tools install path serves tools and data from the same origin
(both come from zddc-server itself), so no cross-origin allowlist is needed
and there is no implicit cross-origin trust to a third-party host.
Set the value explicitly only when browser-loaded pages from a *different*
origin need to call back into this server. Two scenarios:
```sh
# Self-hosted tools on a separate domain
ZDDC_CORS_ORIGIN=https://tools.acme.com
# CDN-bootstrap pattern (loading tools from the canonical upstream and
# pointing them at your server) — opt in to this trust explicitly
ZDDC_CORS_ORIGIN=https://zddc.varasys.io
```
Multiple origins are comma-separated. The middleware echoes the matched
origin back per-request and sets `Access-Control-Allow-Credentials: true`
so the upstream-set `X-Auth-Request-Email` header crosses the boundary.
> Why empty by default? Earlier releases defaulted this to
> `https://zddc.varasys.io` for the CDN-bootstrap convenience, but every
> deployment then implicitly trusted that origin to make authenticated
> cross-origin XHRs on behalf of any logged-in user. That's an unusual
> trust assumption to bake into a default. Now you opt in explicitly when
> you actually need it.
## TLS
### Plain HTTP (no TLS)
Set `ZDDC_TLS_CERT=none` to run without TLS. Recommended when an upstream reverse proxy
(nginx, Caddy, Traefik) terminates external TLS and talks to zddc-server over plain HTTP
on a private network. zddc-server requires `ZDDC_INSECURE_DIRECT=1` for any non-loopback
bind in this mode — an explicit acknowledgement that an authenticating proxy sits in front:
```sh
ZDDC_ROOT=/srv/archive \
ZDDC_TLS_CERT=none \
ZDDC_ADDR=:8080 \
ZDDC_INSECURE_DIRECT=1 \
./zddc-server
```
When `ZDDC_TLS_CERT` / `ZDDC_TLS_KEY` are empty (or when using real certificates), zddc-server generates an ECDSA P-256
self-signed certificate in memory at startup. The certificate changes on every restart —
this is intentional and acceptable when an upstream reverse proxy terminates external TLS
and uses this server only for encrypted in-datacenter transport.
To use a real certificate (e.g. from Let's Encrypt or an internal CA):
```sh
ZDDC_ROOT=/srv/archive \
ZDDC_TLS_CERT=/etc/ssl/zddc/server.crt \
ZDDC_TLS_KEY=/etc/ssl/zddc/server.key \
./zddc-server
```
## Authentication
zddc-server reads the user's email from one of two sources, in order:
1. **`Authorization: Bearer <token>`** — a server-issued bearer token (see
"Bearer tokens" below). Used by CLI clients, scripts, and any
non-browser caller. Validated against `<ZDDC_ROOT>/.zddc.d/tokens/`;
on success the request runs as the email recorded in the token file.
On any failure (unknown token, expired token, malformed header) the
request is rejected with `401` — there is no silent fallback.
2. **`X-Auth-Request-Email`** (header name configurable) — set by an
upstream reverse proxy (nginx, Caddy, Traefik, oauth2-proxy, Azure
Application Gateway, etc.) after authenticating a browser user.
This is the path browser sessions use. **The header is trusted from
any client when no auth proxy is in front** — bind to loopback or
require TLS + proxy in any deployment that exposes the port.
If neither source yields an email, the user is treated as anonymous
(empty email). A request is then allowed only if (a) **no `.zddc` file
exists anywhere in the chain from `ZDDC_ROOT` to the requested
directory** (a fresh tree with zero `.zddc` files defaults to public
access — see warning at the top of the next section), or (b) some level
in the chain explicitly allows the empty email / anonymous callers.
Pass `--no-auth` (or `ZDDC_NO_AUTH=1`) to skip ACL enforcement entirely
on this instance. Anyone hitting the port reads everything in scope, no
matter what `.zddc` files say. Distinct from `--insecure`, which only
permits startup with no root `.zddc`. Use cases: dev, trusted-LAN read-
only deployments, and the "client trusts upstream's filtering" mode of
zddc-server when running as a downstream cache or mirror (see the
"master + proxy/cache/mirror" architecture in ARCHITECTURE.md).
### Bearer tokens
Bearer tokens are issued by zddc-server itself — no external IDP, no
JWKS rotation. Users self-serve via the browser:
1. Sign in to the master via the browser as usual (whatever the upstream
auth proxy expects).
2. Visit `https://<master>/.tokens`. The page lists existing tokens and
offers a "Create token" form (free-form description, optional
expiry).
3. The newly-created token is shown **once** — copy it into a file with
mode 0600 (e.g. `~/.config/zddc/token`). It is never re-derivable
from the server's storage.
4. Pass the token to a CLI/client: `--bearer-file ~/.config/zddc/token`
(or send `Authorization: Bearer <token>` directly via curl/scripts).
Storage lives at `<ZDDC_ROOT>/.zddc.d/tokens/<sha256-hex-of-token>`,
file mode 0600, directory mode 0700. The file is YAML with `email`,
`created`, `expires`, and `description` fields. The plaintext token is
never stored at rest — only its SHA256 hash. A token leak via backup
tools, FS-level audit logs, or an `ls` of the directory would expose
hashes, not credentials.
The directory is shielded from public serving by the standard
dot-prefix guard (any `.foo` segment in a request URL returns 404, and
listings filter dot-prefix entries) — but verify your deployment by
attempting `GET /.zddc.d/tokens/<anything>` and confirming 404.
JSON API for automation (same auth as the page):
| Method | Path | Description |
|---|---|---|
| `GET` | `/.api/tokens` | list current user's tokens (metadata only — never the plaintext) |
| `POST` | `/.api/tokens` | create a token. Body: `{"description":"…","expires":"<RFC3339>"}` (both optional). Returns the plaintext **once**. |
| `DELETE` | `/.api/tokens/<id>` | revoke. `<id>` is the 8-char ID shown in `GET` (or the full 64-char hash). |
A user can only see and revoke their own tokens. Revoking another
user's token returns 404 to avoid leaking ownership.
## Client mode (proxy / cache / mirror)
The same `zddc-server` binary can run as a downstream client of another
zddc-server. Set `--upstream <url>` and the master-side machinery
(archive index, apps server, watcher, OPA decider, ACL middleware,
token store) is replaced by a thin caching HTTP layer that forwards to
the master and (optionally) persists responses under `--root`.
Three modes via `--mode`:
| Mode | Persists responses? | Subtree warmer? | Use case |
|---|---|---|---|
| `proxy` | no | no | thin pass-through; nothing on local disk |
| `cache` (default) | yes | no (only what you visit) | field engineer — what you've viewed is available offline |
| `mirror` | yes | yes (access-triggered) | vendor mirrors, admin backups, complete offline working set |
The cache directory layout is a normal ZDDC root: `<master>/foo/bar.txt`
is stored at `<root>/foo/bar.txt`. No sidecar metadata. Running
`zddc-server --root <cache-dir>` (without `--upstream`) serves the
cached files as a plain master — useful for portable offline snapshots.
A small marker file `.zddc-upstream` is written to the cache root on
first persist, recording the upstream URL and first-cached-at timestamp.
Prevents accidentally pointing master mode at a cache directory and
provides ops provenance.
### Flags
| Flag / env | Purpose |
|---|---|
| `--upstream <url>` / `ZDDC_UPSTREAM` | Master URL (e.g. `https://master.example.com`). Setting this enables client mode. |
| `--mode <proxy\|cache\|mirror>` / `ZDDC_MODE` | Default `cache`. Ignored when `--upstream` is empty. |
| `--bearer-file <path>` / `ZDDC_BEARER_FILE` | Path to a 0600 file with a master-issued token (see `/.tokens` on the master). Forwarded as `Authorization: Bearer …` on every upstream request. |
| `--skip-tls-verify` / `ZDDC_SKIP_TLS_VERIFY` | Accept self-signed / untrusted upstream certs. Distinct from `--no-auth`. Dev / internal-CA scenarios only. |
| `--no-auth` / `ZDDC_NO_AUTH` | Skip ACL enforcement on incoming requests to the local instance. The common case for personal field-engineer / cache deployments where the laptop is single-user-trust and the master already filtered. |
| `--mirror-subtree <csv>` / `ZDDC_MIRROR_SUBTREE` | Mirror-mode only. Comma-separated URL subtrees the access-triggered walker keeps current. Empty + `--mode=mirror` = full mirror (`/`). |
| `--mirror-min-interval <duration>` / `ZDDC_MIRROR_MIN_INTERVAL` | Mirror-mode only. Minimum gap between walks of the same subtree. Default `1h`. Idle subtrees generate zero upstream traffic until next access. |
### Pipeline
For each incoming `GET` (writes are not yet supported in client mode):
1. **Directory request** (URL ends in `/`): always proxied live. No listing cache yet (phase 3 / mirror mode).
2. **File request, cache hit**: serve cached bytes immediately with `X-ZDDC-Cache: hit`. Kick off a background `If-Modified-Since` revalidate; on `304` no-op, on `200` overwrite the cache, on `403`/`404` purge.
3. **File request, cache miss**: forward to upstream with the configured bearer. On `200` stream simultaneously to the client and a tmp-file that's atomically renamed into the cache. Header `X-ZDDC-Cache: miss`.
4. **Network error and a cached version exists**: serve cached + `X-ZDDC-Cache: offline`.
5. **Network error and no cached version**: `503 Service Unavailable` with `X-ZDDC-Cache: offline`.
Range requests (`Range: bytes=...`) work end-to-end: forwarded to upstream on miss, served via `http.ServeContent` from disk on hit (which handles `Range` natively).
Responses with `Cache-Control: no-store` or `Cache-Control: private` are forwarded but not persisted.
### Two-instance dev recipe
```sh
# Master (your normal zddc-server). Pick any --root with a .zddc.
zddc-server --root /srv/zddc --addr :8443
# Client (any port; doesn't need TLS for local dev).
mkdir -p /tmp/zddc-mirror
zddc-server \
--upstream http://master.example.com:8443 \
--root /tmp/zddc-mirror \
--mode cache \
--bearer-file ~/.config/zddc/token \
--addr 127.0.0.1:8444 \
--tls-cert=none \
--no-auth
```
Browse `http://localhost:8444/`. Files you visit appear under `/tmp/zddc-mirror/` mirroring the master's path layout. Disconnect, refresh — previously-visited files keep working. Reconnect — background revalidates run on every cache hit, picking up master-side changes the next time you reload.
### Mirror mode (access-triggered subtree walker)
`--mode mirror` adds an access-triggered walker on top of the cache pipeline. When a request arrives at a path under one of the configured `--mirror-subtree`s, the scheduler kicks off a recursive walk of that subtree if `now - last_walk_at >= --mirror-min-interval` and no walk is already in flight for that subtree. The walk:
1. Recursively fetches JSON listings under the subtree (each listing also persisted as the JSON sidecar `.zddc-listing.json`, so directory browsing works offline).
2. For each file in the listing, fires a conditional `If-Modified-Since` GET — `304` is a no-op, `200` writes new bytes, `403`/`404` purges the local cache entry.
3. After enumeration, purges local files in each walked directory that no longer appear in the master's filtered listing (handles upstream deletes + ACL revocations).
State (`{subtree → last_walk_at}`) persists in `<root>/.zddc-mirror-state.json` so restarts honor the last-walked timestamp. In-flight tracking is in-memory only — a crash mid-walk lets the next access retry immediately.
Properties:
- **Idle mirrors are quiet.** No requests means no walks means no traffic to the master. A vendor who hasn't logged in for three months triggers exactly one walk on next access.
- **Active mirrors stay current as a side effect of use.** Browsing always touches the access-trigger; walks fire at most once per `--mirror-min-interval` per subtree.
- **Revocation latency** is bounded by access frequency: a revoked file in an idle mirror remains until the next walk fires.
- **Bounded concurrency** (4 parallel fetches per walk) so the walker doesn't starve the user's interactive requests sharing the same connection pool.
### What client mode is NOT, yet
- **No write path**: `PUT`/`POST`/`DELETE` return `405`. The offline write outbox lands in a later phase.
- **No multi-tenancy**: the local instance forwards a single bearer to upstream regardless of who's calling locally. For multi-user deployments, run multiple instances or front the local server with your own auth proxy.
## Access control: the `.zddc` cascade
> ⚠️ **zddc-server refuses to start without a root `.zddc`.** A `ZDDC_ROOT` containing
> no `.zddc` files anywhere would default to allow-all (anonymous callers included),
> so the server fails fast at startup with a clear error. Pass `--insecure` (or
> `ZDDC_INSECURE=1`) to acknowledge a deliberately-public deployment, otherwise
> drop a starter `<ZDDC_ROOT>/.zddc` per "Step 1" below.
`zddc-server` enforces access via cascading `.zddc` YAML files: drop one in any
directory, and its rules apply to that directory and everything beneath it that doesn't
override. The model is small enough to hold in your head, but the cascade has one
asymmetry that bites operators on first contact — read "When the cascade helps and
when it fights you" below before designing a layout.
```yaml
# Example .zddc — modern schema with verbs and roles
roles:
_company:
members: ["*@mycompany.com"]
_doc_controller:
members: [dc@mycompany.com]
acl:
permissions:
_company: r # everyone at mycompany.com gets read
_doc_controller: rwcda # doc controller gets full control
"contractor@partner.com": rw # specific external — read + overwrite
"intern@mycompany.com": "" # explicit deny (empty verb set)
# Legacy form below still works — equivalent to the new form, with
# allow → "rwcd" and deny → "" entries auto-merged into permissions.
# acl:
# allow: ["*@mycompany.com", "contractor@partner.com"]
# deny: ["intern@mycompany.com"]
```
### Permission verbs
Every access decision resolves to a verb set drawn from `r/w/c/d/a`:
| Verb | Allows |
|---|---|
| `r` | read file bytes; list directory |
| `w` | overwrite an existing file; rename existing file |
| `c` | create a new file or directory |
| `d` | delete a file |
| `a` | modify the ACL of this subtree (write `.zddc`) |
The verb set is written as concatenated lowercase letters in canonical order — `""` (none / explicit deny), `r`, `cr`, `rwcd`, `rwcda`. Common archetypes:
- **`r`** — read-only (typical company default).
- **`cr`** — append-only / drop-box (the doc controller in `Issued`/`Received`: can file new documents, cannot overwrite or delete).
- **`rwcd`** — full content control without the right to change the ACL (vendor inside their working subtree).
- **`rwcda`** — full control including the ability to grant access to others (subtree creator; project owner).
### Roles
`roles:` defines named principal groups, available at the level they're declared and all descendants:
```yaml
roles:
vendor_acme:
members: ["*@acme.com"]
_doc_controller:
members: [dc@mycompany.com, alice@mycompany.com]
```
Members are email patterns using the same glob syntax as legacy `acl.allow`. Underscore-prefixed names are conventional (`_doc_controller`, `_company`) but not magic. A role redefined closer to the leaf shadows the ancestor's definition. Permission-map keys without `@` are treated as role references first; if no role of that name exists in the visible cascade, they fall back to legacy email-pattern matching (so `*@example.com` and bare `*` continue to work).
### Step 1: starter `.zddc`
Every install should write a root `.zddc` before exposing the bind address. The
minimum is an `admins:` line so the admin debug page works (see "Admin Debug Page"
below) — adding `acl:` is optional at this step:
```yaml
# <ZDDC_ROOT>/.zddc — bare minimum
admins:
- you@yourcompany.com
```
With this single file in place, `HasAnyFile` becomes `true` for every directory in
the tree and the default switches from "allow-all-anonymous" to "deny-anything-not-
explicitly-allowed." From here you grant access by adding `acl:` rules at the levels
that need them. (See worked examples below.)
### How a request is evaluated
Each request carries an **action verb** (`r` for `GET`, `w` for `PUT` to an
existing file, `c` for `PUT` to a new file or `mkdir`, `d` for `DELETE`, `a`
for writes to `.zddc`). zddc-server reads every `.zddc` along the chain from
`ZDDC_ROOT` down to the request directory, then walks **leaf → root** looking
for a level whose `acl.permissions` map matches the user.
1. **Admin bypass.** If the email is in the root `admins:` list (root admin) or
any subtree-level `admins:` list on the chain (subtree admin), grant
`rwcda` and skip the cascade entirely.
2. **At each level**, find every `permissions:` entry whose principal matches
the user (direct email pattern, or role membership via `roles:` lookup).
- If any matching entry has the empty verb set `""` → **403 Forbidden**, stop.
- Otherwise, take the **union** of matching verb sets at this level. If the
union is non-empty, the level "wins" — the requested verb is allowed iff
it's present in the union. Stop walking.
3. **No match at this level** → walk up to the parent directory's `.zddc`.
4. **No level matched anywhere in the chain:**
- No `.zddc` anywhere (`HasAnyFile=false`) → **allow** (the empty-tree default).
- At least one `.zddc` existed (`HasAnyFile=true`) → **403 Forbidden** (default-deny).
The walk respects an **inherit fence** (see "The `inherit:` directive" below).
A level whose `acl.inherit: false` flag is set acts as a fence: ancestors above
it are invisible to descendants at-and-below the fence, both for grants and for
role lookups. In strict cascade mode the fence is ignored (NIST AC-6 invariant).
Implementation: `GrantedVerbsAtLevel` (`zddc/internal/zddc/acl.go`) computes the
per-level grant; `EffectiveVerbs` / `AllowedAction` walk the chain; the chain
itself is built by `EffectivePolicy` (`zddc/internal/zddc/cascade.go`); the
fence is computed by `PolicyChain.VisibleStart`.
#### Cascade mode
The leaf-overrides-ancestor behavior above is the default — it's the historical
commercial-tenant model where a subtree owner can grant access without
root-admin involvement. Federal deployments needing absolute parent denies
(NIST AC-6) start the server with `--cascade-mode=strict` (or
`ZDDC_CASCADE_MODE=strict`):
- **`delegated`** (default) — leaf grant overrides ancestor explicit-deny.
- **`strict`** — two-pass evaluation. First pass walks **root → leaf** for any
matching explicit-deny; if found, denied (subject to root-admin bypass).
Second pass is the leaf→root grant walk above. An ancestor explicit-deny
cannot be overridden by any leaf grant.
The mode is logged at startup and surfaced on `/.profile/config`. Subtree
`.zddc` files cannot change the mode — it's a deployment-wide policy.
#### The `inherit:` directive
A `.zddc` may declare `acl.inherit: false` to fence off all ancestor grants
and roles from the descendant subtree. Useful for the "complete reset, then
add back specific principals" pattern — vendor folders, regulated subtrees,
anywhere a permissive ancestor rule is too broad and the operator wants
explicit local control.
```yaml
# <ZDDC_ROOT>/Vendor/.zddc — vendor folder, fully reset
acl:
inherit: false # ancestors above this level become invisible
permissions:
"*@vendor.com": rwcd # the vendor and the doc controller are
_doc_controller: rwcda # the only principals with access here
```
Behaviour:
- **Grants:** the cascade walker (leaf → root) stops at the deepest
`inherit: false` level when looking for a matching grant. Ancestor
`permissions` entries don't contribute. The default-deny rule still
applies if no descendant grant matches.
- **Roles:** role definitions in ancestors above the fence are also
invisible. To use a role inside a fenced subtree, redefine it locally
(a redefinition at-or-below the fence is fine).
- **Admins:** the root `admins:` list is unaffected. Root admins still
bypass all ACL evaluation, fence or no fence — that's the deliberate
escape hatch for misfiled documents.
- **WORM:** the `archive/<party>/issued|received/` mask is path-based,
not cascade-based. `inherit:` does not change WORM behaviour.
**Strict cascade mode IGNORES `inherit: false`.** NIST AC-6 requires
ancestor explicit-denies to be absolute, and the inherit directive
would let a leaf widen access an ancestor refused. Under
`--cascade-mode=strict` the directive has no effect (and the bundled
federal Rego at `--print-rego=federal` mirrors that rule). Operators
who need fence-style "reset" semantics in a federal-track deployment
should not use the directive — instead, restructure the tree so the
permissive ancestor rule never appears.
The cascade tracer (`/.profile/effective-policy`) surfaces every
level's `inherit` flag and the `chain.visible_start` index so a
reviewer can see exactly where the fence sits and whether it's in
effect under the active mode.
Implementation: parser (`zddc/internal/zddc/file.go`),
`PolicyChain.VisibleStart` (`zddc/internal/zddc/cascade.go`), and the
fence-aware role walk (`zddc/internal/zddc/roles.go`).
#### Special folders
Five folder names trigger built-in behaviors regardless of cascade mode (canonical list in `zddc/internal/zddc/special.go`):
- **`Incoming`, `Working`, `Staging`** — *auto-ownership*. When the file API processes `POST /<parent>/<new>/ X-ZDDC-Op: mkdir` and the parent is one of these three, the server writes a `.zddc` into the new folder containing `created_by: <email>` and `permissions: { <email>: rwcda }`. The grant uses the same direct email-pattern form an operator would write by hand; the creator can edit the `.zddc` later to add collaborators. `created_by` is an audit field — the cascade evaluator does not consult it.
- **`Issued`, `Received`** — *write-once / immutable archive*. When a request path crosses an `Issued` or `Received` segment, the server applies a **WORM split**: cascade grants inherited from ancestors above the WORM folder are masked to `r` only; grants at-or-below the WORM folder retain `r,c`. Anyone with `w`/`d`/`a` from inheritance loses those verbs once they enter the archive. To grant write-once (`cr`) to the doc controller, the operator places an explicit `.zddc` at the `Issued` or `Received` folder:
```yaml
# /<vendor>/Issued/.zddc
acl:
permissions:
_doc_controller: cr
```
The mask preserves the `c` from this same-level grant, so the doc controller can file new documents — but they still cannot overwrite, delete, or change the ACL. **Only admins (root or subtree) can mutate filed documents.** The mask is server-enforced and not configurable in v1; operators who want a non-WORM directory must avoid the names `Issued` and `Received`.
### Glob patterns
`*` matches any sequence of characters within one side of the `@` boundary:
| Pattern | Matches |
|---|---|
| `*@mycompany.com` | Any user at mycompany.com |
| `alice@*` | alice at any domain |
| `*` | Any non-empty email |
| `alice@example.com` | Exact match only |
The `*` does **not** cross the `@`. Implementation at `zddc/internal/zddc/acl.go:52`.
### When the cascade helps and when it fights you
The cascade is well-suited to one shape and clumsy at another. Internalize the
asymmetry before designing your layout:
- **Adding a new email at a leaf is easy** (third parties, occasional contractors).
The new email doesn't match any rule at higher levels, so the cascade just grants
the leaf-level allow and silently default-denies them everywhere else. No extra
`deny:` rules needed.
- **Excluding insiders from a leaf is harder** (commercially sensitive subset of
an otherwise company-wide tree). If a parent has `allow: ["*@company.com"]`, a
leaf-level `allow: [alice@company.com]` *adds* alice on top of everyone — it
does not subtract everyone-else. Subtracting requires either (a) the two-level
gate-and-reallow pattern in the worked example below, or (b) not having the
permissive parent rule in the first place.
This shape is intentional: the cascade is designed for **delegation** (subtree
owners can grant access without coordinating with central admin), not for
fine-grained subtractive policy. If your model is "everyone has access by default,
specific dirs are restricted," push the wildcards downward off the root rather than
fighting the cascade.
### Pick your layout
| Your shape | Recommended pattern |
|---|---|
| Solo / single-user archive | One `.zddc` at root with `admins: [you]`, no `acl:` block — root-only restriction |
| Single small team, full sharing | Root `.zddc` with `acl: {allow: ["*@team.com"]}`. No subdir overrides needed |
| Multi-tenant: each tenant in own subdir | Empty root `.zddc` (just `admins:`), per-tenant `<tenant>/.zddc` with `acl: {allow: [<tenant emails>]}`. Tenants don't see each other's listings |
| Mixed: half open within company, half locked-down to a subset, plus third-party folders | The worked example below — careful: do **not** put `*@company.com` at root |
### Worked example: paired open/closed projects + third-party archive
This is the deployment shape most operators end up with: technical projects are
shared company-wide; their commercial siblings are restricted to a subset; a
separate `/Archive/` tree holds per-vendor folders where each vendor sees only
their own subdir. The cascade handles all three with no `deny:` rules — but only
if you keep the `*@company.com` wildcard *off the root*.
```yaml
# <ZDDC_ROOT>/.zddc — admins only, no broad ACL
admins:
- admin@mycompany.com
```
```yaml
# <ZDDC_ROOT>/Acme-tech/.zddc — open employee project (technical)
acl:
allow: ["*@mycompany.com"]
```
```yaml
# <ZDDC_ROOT>/Acme-comm/.zddc — closed sibling (commercially sensitive)
acl:
allow:
- alice@mycompany.com
- bob@mycompany.com
```
```yaml
# <ZDDC_ROOT>/Archive/.zddc — employees can browse the vendor list
acl:
allow: ["*@mycompany.com"]
```
```yaml
# <ZDDC_ROOT>/Archive/Acme/.zddc — vendor's only window
acl:
allow:
- acme-rep@acme.com
```
Trace for an insider (`alice@mycompany.com`) and a vendor (`acme-rep@acme.com`)
hitting representative paths:
| Path | alice@mycompany.com | acme-rep@acme.com |
|---|---|---|
| `/Acme-tech/` | ✅ matches `*@mycompany.com` at this level | ❌ no match anywhere; `HasAnyFile=true` → 403 |
| `/Acme-comm/` | ✅ matches `alice` at this level | ❌ default-deny |
| `/<other-closed-project>/` | ❌ no match anywhere → 403 | ❌ default-deny |
| `/Archive/` | ✅ matches `*@mycompany.com` | ❌ no match; default-deny |
| `/Archive/Acme/` | ✅ falls up to `/Archive/.zddc`, matches | ✅ matches `acme-rep@acme.com` at this level |
| `/Archive/Acme/Incoming/` | ✅ inherits from `/Archive/.zddc` | ✅ inherits from `/Archive/Acme/.zddc` |
| `/Archive/<other-vendor>/` | ✅ inherits from `/Archive/` | ❌ no match; default-deny |
| `/` (project picker) | Lands; sees the projects she has access to | Lands; project picker filtering hides everything she can't reach |
The vendor reaches `/Archive/Acme/...` only via direct URL (a bookmark or a
transmittal email link). Trying to navigate up to `/Archive/` returns 403; the
archive tool itself treats `/Archive/Acme/` as the root of *its* world (see
"Tool-rooted view" in `ARCHITECTURE.md` § Server security model), so there is
no breadcrumb leading anywhere they can't see.
### Patterns that look secure but aren't
These are the traps. Each is plausible at first glance and doesn't behave as
naive intuition suggests.
1. **Same-level `allow + deny "*@company.com"` does NOT lock the level down to the
allow's targets.** Deny is checked before allow within a single `.zddc`, so the
allowed user's email matches the deny first and is blocked. To exclude insiders
from a leaf, use the `inherit: false` directive (preferred — see "The
`inherit:` directive" above), or the two-level gate-and-reallow pattern, or
avoid putting `*@company.com` at any ancestor in the first place.
```yaml
# /Closed/.zddc — DOES NOT WORK as intended
acl:
allow: [alice@company.com]
deny: ["*@company.com"] # blocks alice too — deny is checked first at same level
# /Closed/.zddc — works correctly
acl:
inherit: false # ancestor "*@company.com" rule is invisible here
permissions:
alice@company.com: rwcd
```
2. **A leaf-level `allow: [subset]` does NOT restrict when a parent has
`allow: ["*@domain"]`.** Non-subset users hit the leaf with no match, walk up,
match the parent wildcard, and are allowed. Adding a leaf allow is *additive*,
never *subtractive*. (See the asymmetry section above.)
3. **`admins:` in any `.zddc` other than the root is silently ignored.** The check
at `zddc/internal/zddc/file.go:17-20` (and `IsAdmin`) only reads root. This is
the only upward-escalation gate; subtree write access never grants admin.
4. **An `apps:` URL override is a full UI mount, not just a tool version pin.**
Any `.zddc` writer in a subtree can pin `archive: https://attacker.example/...`
and serve arbitrary HTML to every viewer below that level. Subtree write
authority on `.zddc` should be treated as full UI-mounting authority. The
`_app/` cache is fetch-once-and-keep — operators clear it by deleting
`<ZDDC_ROOT>/_app/<scheme>/<host>[:<port>]/<path>`. (See "Apps: virtual tool HTMLs" below for
the resolver order; SHA-256 pinning is on the federal-readiness list, not
currently implemented.)
5. **Relying on `/Archive/` being unbrowsable to "hide" sibling vendor folders'
existence.** Sibling-vendor names are hidden because directories the caller
can't access are omitted from listings (see "Directory visibility" below) —
that's the actual mechanism. Don't rely on the parent dir being denied; rely
on the listing filter.
### Trust model and invariants
These are the invariants security reviewers should expect to find, stated plainly
and tied to the code that enforces them:
- **Auth boundary.** zddc-server does not authenticate. The user's email is read
from the configured request header (default `X-Auth-Request-Email`) set by an
upstream reverse proxy. If the proxy is misconfigured or the bind address is
reachable without traversing the proxy, every request is anonymous — and any
caller able to set the header can claim any email. Network isolation between
the proxy and zddc-server is required (see "Trust boundary" below).
- **Subtree authority.** Whoever can write a `.zddc` controls that subtree's
ACL — including overriding a parent *deny* with a leaf *allow* (test:
`zddc/internal/zddc/acl_test.go:212` "leaf allows user that parent denies →
leaf wins"). This is intentional delegation, not a bug. If you grant write
access to `/Project-A/.zddc`, you've granted full ACL authority over the
Project-A subtree.
- **Root-only escalation gate.** `admins:` is honored *only* at
`<ZDDC_ROOT>/.zddc`. Subtree `admins:` entries are silently ignored
(`zddc/internal/zddc/file.go:17-20`). This is the *only* upward escalation
block; without it, anyone with subtree write authority could promote
themselves to admin.
### Trust boundary
What zddc-server enforces stops at the network boundary. The deployment must
guarantee these for the model above to hold:
1. **The bind address must be reachable only via the authenticating proxy.** The
email-header trust assumes the proxy is the only path to the server. In
Kubernetes: a `NetworkPolicy` restricting ingress to the proxy pod. On a
single host: bind to loopback (`ZDDC_ADDR=127.0.0.1:8080`) and run the proxy
on the same host. **Without this**, anyone reaching the bind address can
forge any email by setting `X-Auth-Request-Email` themselves. zddc-server
refuses to start with `ZDDC_TLS_CERT=none` on a non-loopback bind unless
`ZDDC_INSECURE_DIRECT=1` is set as an explicit acknowledgement that an
authenticating proxy is enforcing this.
2. **Anonymous information disclosure on `/` is by design.** The public landing
page returns a project picker filtered by ACL — anonymous callers see only
projects with no `.zddc` rules along their chain (in a properly-configured
deployment, none). The *existence* of the server and the names of any
ACL-free projects are disclosed without authentication. For deployments
where this disclosure is unacceptable, gate `/` itself behind the proxy's
auth-required path; zddc-server's public-landing logic does not need to be
disabled, it simply never receives an anonymous request. `/.profile` is also
reachable anonymously by design — same caveat applies.
3. **Audit log integrity is filesystem-level, not application-level.** The
tee'd JSON access log lives at `<ZDDC_ROOT>/.zddc.d/logs/access-<host>.log`
on the served volume. While `.`-prefixed paths return 404 over HTTP (so the
log is not readable through zddc-server), anything with **filesystem write
authority on the served volume** (a sidecar, a backup-restore process, an
admin shell) can modify log entries after the fact. For tamper-evident
logging, ship the JSON-line file to an external append-only sink (syslog,
SIEM) via a sidecar; do not treat the local rotation as the system of
record.
4. **`apps:` URL fetches have no integrity check.** Fetched once on first
miss, cached at `<ZDDC_ROOT>/_app/<scheme>/<host>[:<port>]/<path>` forever — no SHA-256 pin,
no signature. Use only URLs you control, treat the apps cache as a trust
boundary, and audit who has `.zddc` write authority where.
### Debugging permissions
When a user reports "I can't see /Project-X/" and you need to figure out why,
the fastest path is the built-in cascade tracer:
```
GET /.profile/effective-policy?path=/Project-X/sub/&email=alice@mycompany.com
```
(Admin-only — 404 to non-admins. Same gate as `/.profile/whoami`,
`/.profile/config`, `/.profile/logs`.)
Returns JSON with the resolved policy chain (every level along the
walk from `ZDDC_ROOT` to the requested directory), the decision the
active decider produces, the per-level email-match breakdown
(`decision_at_level: "allow" | "deny" | "no_match"`), and which
decider is wired in (`*policy.InternalDecider`,
`*policy.cachingDecider`, etc.). With `ZDDC_OPA_URL` pointing at an
external OPA, the decision goes through that OPA — so this endpoint
also doubles as a smoke test for the OPA wiring.
Manual procedure (if the endpoint isn't reachable for some reason):
1. **Confirm the resolved email** — hit `/.profile/whoami` as the
user. Shows every header on the request and the `email` field
zddc-server resolved.
2. **List the chain.** From `<ZDDC_ROOT>` down to the requested
directory, inspect each `.zddc` (most directories have none). For
`/Project-X/sub/sub/`, that's `/.zddc`, `/Project-X/.zddc`,
`/Project-X/sub/.zddc`, `/Project-X/sub/sub/.zddc` — read whatever
exists.
3. **Walk bottom-up.** At each level, mentally run `AllowedAtLevel`:
deny patterns first (any match → blocked), then allow (any match
→ allowed). First explicit match in the bottom-up walk is the
answer. Default-deny if `HasAnyFile=true` and nothing matches.
### Directory visibility
Directories for which the user lacks access are **omitted** from JSON listings entirely —
they are neither listed nor queryable. A direct request to a denied path returns `403`.
This is the mechanism that hides sibling subtrees from a caller. Vendor `acme-rep`
sees an empty-looking `/Archive/` (in fact returns 403 since they're not allowed
there at all in the worked example), and no other vendor's name leaks via listing.
### Reserved hidden segments
Two prefixes are filtered from listings under `ZDDC_ROOT`:
- **`.`-prefixed** (e.g. `/.devshell/`, `/Project-A/.internal/notes.md`) — excluded
from listings **and** 404 on direct HTTP access. The recognized virtual prefixes
(`.archive`, `.admin`) are explicitly permitted through. This lets operators store
side-state (caches, dev-shell home dirs, snapshot staging) on the same volume
that's served, without exposing it.
- **`_`-prefixed** (e.g. `/_template/`) — excluded from listings only. Direct URL
access still works. Use this for operator-managed scaffolding the user shouldn't
browse to but might link to (e.g. a `_template/` directory of stub-HTML examples
to copy into project subdirs).
### How to verify in 5 minutes
This recipe stands up the worked-example layout in a tmpdir, hits each
(email, path) cell with `curl`, and asserts the documented behavior. Run it on
your own deployment to confirm the cascade is doing what you think:
```sh
ROOT=$(mktemp -d)
mkdir -p "$ROOT/Acme-tech" "$ROOT/Acme-comm" "$ROOT/Archive/Acme"
cat > "$ROOT/.zddc" <<'YAML'
admins: [admin@mycompany.com]
YAML
cat > "$ROOT/Acme-tech/.zddc" <<'YAML'
acl: {allow: ["*@mycompany.com"]}
YAML
cat > "$ROOT/Acme-comm/.zddc" <<'YAML'
acl: {allow: [alice@mycompany.com]}
YAML
cat > "$ROOT/Archive/.zddc" <<'YAML'
acl: {allow: ["*@mycompany.com"]}
YAML
cat > "$ROOT/Archive/Acme/.zddc" <<'YAML'
acl: {allow: [acme-rep@acme.com]}
YAML
# Plain HTTP on loopback so curl doesn't need TLS
ZDDC_ROOT="$ROOT" ZDDC_TLS_CERT=none ZDDC_ADDR=127.0.0.1:8090 \
./zddc-server &
SERVER_PID=$!
sleep 1
probe() { # email path expected_status
got=$(curl -s -o /dev/null -w '%{http_code}' \
-H "X-Auth-Request-Email: $1" \
"http://127.0.0.1:8090$2")
printf '%-40s %-30s expected=%s got=%s %s\n' \
"$1" "$2" "$3" "$got" \
"$([ "$got" = "$3" ] && echo OK || echo FAIL)"
}
# Insider — alice should see all the technical + her closed project + Archive tree
probe alice@mycompany.com /Acme-tech/ 200
probe alice@mycompany.com /Acme-comm/ 200
probe alice@mycompany.com /Archive/ 200
probe alice@mycompany.com /Archive/Acme/ 200
# Insider not on the closed project — bob should hit the technical and Archive,
# NOT the closed sibling
probe bob@mycompany.com /Acme-tech/ 200
probe bob@mycompany.com /Acme-comm/ 403
probe bob@mycompany.com /Archive/ 200
probe bob@mycompany.com /Archive/Acme/ 200
# Vendor — acme-rep should ONLY see /Archive/Acme/, blocked everywhere else
probe acme-rep@acme.com /Acme-tech/ 403
probe acme-rep@acme.com /Acme-comm/ 403
probe acme-rep@acme.com /Archive/ 403
probe acme-rep@acme.com /Archive/Acme/ 200
# Anonymous — root .zddc exists, so HasAnyFile=true → default-deny everywhere
# (the root / itself is the public-landing exception; subdirs are gated)
probe '' /Acme-tech/ 403
probe '' /Archive/Acme/ 403
# Anti-pattern: same-level allow + deny *@company.com does NOT lock alice in
mkdir -p "$ROOT/Trap"
cat > "$ROOT/Trap/.zddc" <<'YAML'
acl:
allow: [alice@mycompany.com]
deny: ["*@mycompany.com"] # deny is checked first → blocks alice too
YAML
probe alice@mycompany.com /Trap/ 403 # the trap docs warn about
kill $SERVER_PID
```
Every line should print `OK`. If any prints `FAIL`, the cascade isn't behaving
as documented — file an issue with the failing line. **After every `.zddc`
change in production**, retest at minimum as the editing user (to confirm you
haven't locked yourself out).
### Federal-readiness gap analysis
The current model is well-suited for commercial-tenant ACL with delegated
authentication. To clear federal hurdles (FedRAMP Moderate, NIST 800-53 Rev. 5
baseline, FIPS 140-3, DoD STIG), these gaps would need to be closed. None are
implemented today — this list is informational so security reviewers don't
have to redo the gap analysis from scratch.
- **FIPS 140-3 cryptography** (NIST SC-13) — current build uses Go stdlib
crypto. *Required:* parallel `zddc-server-fips` build target using the
`microsoft/go` toolchain on a RHEL/UBI base with the validated OpenSSL
FIPS module. See `federal.html` for the deployment-shape narrative and
the §"FIPS-validated cryptography" subsection below for the engineering
detail.
- **TLS hardening** (NIST SC-8(1)) — *partially complete.* Server now
sets `MinVersion: tls.VersionTLS12`, the NIST SP 800-52 Rev. 2
AEAD-only cipher allowlist (ECDHE+AES-GCM and ECDHE+ChaCha20Poly1305
variants), curve preferences (X25519, P-256, P-384), and emits HSTS
(`max-age=31536000; includeSubDomains`) when zddc-server itself
terminates TLS. *Still required for full DoD STIG conformance:*
OCSP stapling, certificate-transparency-log inclusion, and an
audit-grade documentation pack mapping the cipher list to FIPS
140-3 validated implementations.
- **Authenticated proxy↔server channel** (NIST IA-3) — current trust is
network-level isolation only. *Required:* mTLS or a signed forwarding
token (JWT). See §"Authenticated proxy↔server channel" subsection
below for the design trade-off and recommended path.
- **Multi-factor authentication** (NIST IA-2(1)) — delegated to upstream
proxy. Required: documented reference deployment with PIV/CAC via
oauth2-proxy or equivalent.
- ~~**Role-based access control** (NIST AC-3(7))~~ — *closed.* Roles are
first-class entities defined under `roles:` in any `.zddc`, available
at the level they're declared and all descendants. `acl.permissions`
grants verb sets (`r`/`w`/`c`/`d`/`a`) per role or per email pattern.
Identity-source-driven role assignment plumbs through unchanged
(the upstream proxy still asserts the email; role membership is
evaluated server-side against the cascade).
- ~~**Least-privilege bounding** (NIST AC-6)~~ — *closed.* Operators
set `--cascade-mode=strict` (or `ZDDC_CASCADE_MODE=strict`) to
switch the in-process Go evaluator into the federal posture: any
ancestor explicit-deny is absolute and cannot be overridden by a
leaf grant. The mode is logged at startup and surfaced on
`/.profile/config`. The legacy commercial behavior is preserved as
the default `delegated` mode. External OPA (`ZDDC_OPA_URL`) remains
available for org-specific Rego on top of this.
- **Account lifecycle** (NIST AC-2) — emails as identifiers must tie to
authoritative sources (PIV cert subject, IdP-managed identity). Required:
documented integration with at least one IdP supporting federal identity
attestation.
- **Audit log integrity & retention** (NIST AU-9, AU-11) — current 90-day
local rotation is a starting point. Required: tamper-evident logs (signed
log chain or external append-only sink) with 1-year minimum online and
3-year archive.
- **Continuous monitoring hooks** (NIST CA-7) — automated alerting on
`.zddc` changes, admin endpoint use, repeated 403s from one identity.
Required: structured event emission to syslog/SIEM beyond the local file.
- **Configuration baseline export** (NIST CM-3) — change-control review
needs a way to enumerate every `.zddc` file's resolved effective ACL
for diff against a baseline. *Required:* `zddc-server policy export`
subcommand. See §"Policy export for change control" below.
- **Supply-chain integrity** (NIST SI-7) — vendored libs (jszip,
docx-preview, xlsx) need SBOM, CVE tracking, automated update pipeline.
`apps:` URL fetches need code signing (operator trusts a published
public key once; no per-artifact hash management). See §"Code-signed
apps: URL fetches" below.
- **Data-at-rest encryption** (NIST SC-28) — delegated to the deployment
platform. Required: documented baseline (cloud KMS, LUKS, dm-crypt) with
key-rotation procedures.
- **Vulnerability disclosure process** (NIST SI-5) — repo lacks
`SECURITY.md`. Required: documented disclosure procedure, embargoed-fix
workflow, CVE-assignment policy.
A full SSP / control-by-control mapping consumes this list as input; it is
not a substitute for one.
The four bullets most likely to need engineering depth — FIPS, the
authenticated proxy channel, policy export, and signed `apps:` URL
fetches — have their own subsections below capturing the design
considerations and effort estimates so a future implementor doesn't
restart from zero.
#### FIPS-validated cryptography (NIST SC-13)
**Why the current build doesn't qualify.** Go's stdlib crypto is
correct and well-reviewed but has not been submitted to NIST's CMVP
(Cryptographic Module Validation Program). Federal evaluators want a
specific module identifier ("the cryptography is performed by validated
module #4282" or whatever certificate number applies); Go stdlib has
none.
**Why "swap in microsoft/go" isn't free.** The microsoft/go fork
redirects every `crypto/*` call into the host's OpenSSL via cgo. The
moment we adopt it as the default toolchain:
- The "single static binary, CGO-free" property in the Features list
(line 19) goes away. The binary dynamically links `libssl`/`libcrypto`
and needs OpenSSL installed at the target ABI.
- Cross-compile becomes harder: cgo cross-compilation needs the target
platform's OpenSSL during the build. Today `./build` cross-compiles
all four release platforms from one container with no extra tooling.
- macOS/Windows builds either bundle a non-validated OpenSSL (defeats
the FIPS purpose) or fall back to Go stdlib for those platforms
(validation not transitive). The four-platform release matrix
collapses to one (linux-amd64).
- Binary size grows ~5-10 MB; deployment gains a runtime dependency.
- Commercial customers gain literally nothing — Go stdlib crypto is
cryptographically correct; FIPS is a procurement-checkbox property,
not a strength property.
**Recommended path: parallel `zddc-server-fips` build target.** Don't
swap the default toolchain. Instead, ship a second binary built with
microsoft/go on a RHEL/UBI base with the validated OpenSSL FIPS package
installed. linux-amd64 only. Distributed at
`releases/zddc-server-fips_<channel>_linux-amd64`. Federal customers
download the fips variant; everyone else continues to download the lean
pure-Go binary.
The validation belongs to OpenSSL, not to microsoft/go — the cert
number is for the OpenSSL FIPS provider. The microsoft/go toolchain is
the bridge that lets Go-built apps consume that validated module. So
the deployment commitment is "RHEL/UBI image, kernel-side
`update-crypto-policies --set FIPS`, validated OpenSSL on the host." A
helm chart `helm/zddc-server-fips/` would bundle the matching base
image and `securityContext`.
**Effort estimate:**
- `Containerfile.fips` and `helm/zddc-server-fips/` chart: ~50 lines
each.
- `./build` updates to add the new target: ~30 lines (one
`podman build -f Containerfile.fips` invocation, one
`promote_zddc_server_fips` helper, integration into the existing
release flow).
- Verification: a smoke test that runs the fips binary, hits a TLS
endpoint, and confirms the negotiated cipher came from the FIPS
provider (visible via `SSL_CIPHER` exposed from OpenSSL).
- Documentation: ~50 lines updating `federal.html` and this README to
describe the supported deployment shape concretely.
Total: ~250 lines net once the customer engages.
**What stays unchanged.** Application code is identical between tracks.
The TLS hardening already shipped (cipher allowlist, curve preferences,
HSTS) reuses verbatim — every cipher we ship is on the OpenSSL FIPS
allowlist, so the same `tls.Config` works in both tracks. The
difference is purely the toolchain and the host OS configuration.
#### Authenticated proxy↔server channel (NIST IA-3)
**The current trust assumption.** zddc-server reads
`X-Auth-Request-Email` from any client that can connect to the bind
address. Network isolation (loopback bind + same-host proxy, or k8s
NetworkPolicy gating ingress) is the only thing preventing email
forgery. NIST IA-3 (Device Identification and Authentication) requires
**cryptographic** binding between system components for higher assurance
levels.
**Two design options.**
**Option A — mTLS.**
The proxy presents a client certificate during the TLS handshake to
zddc-server. zddc-server verifies the cert against a trusted CA and
extracts the email from the cert's subject alternative names (or
matches the cert against an allowlist).
- *Pros:* Works at the connection layer; transparent to handler code
(the email comes from `r.TLS.PeerCertificates[0]` instead of a
header). Reuses existing TLS infrastructure. Zero per-request
overhead beyond the standard TLS handshake.
- *Cons:* Cert distribution. Every proxy needs its own client cert;
every cert eventually expires. Operator runs a small private CA
(cert-manager + k8s, smallstep, Vault PKI) or rotates manually.
Operationally heavier than the alternative.
**Option B — JWT (signed forwarding token).**
The proxy signs a JWT for each forwarded request using its private key.
zddc-server verifies the signature with the proxy's public key. The JWT
carries the email, an issuer claim, an audience claim, and a short
expiry (replay protection). Sent in `Authorization: Bearer <jwt>` or a
custom `X-Auth-Forwarding-Token` header.
- *Pros:* Lightweight. No PKI — just one public key to distribute to
zddc-server. Easy to debug (JWT is base64 JSON; `jwt.io` exists).
Easy to rotate (run two trusted public keys during rotation).
Integrates with existing oauth2-proxy / nginx pipelines that have
"sign and forward" modes built in.
- *Cons:* Adds a JWT library dep (~50KB). Requires a freshness model
(short expiry, e.g. 30s — so an exfiltrated token is useless after
half a minute). Requires key distribution + rotation, but at much
smaller scale than mTLS.
**Recommended path: JWT first; mTLS as a sidecar option.**
JWT is the better starting point. Lighter operationally, doesn't
compete with the existing TLS termination story (an stunnel-FIPS or
nginx-FIPS sidecar can still do mTLS at the network layer if needed),
and integrates with future role-based access (item AC-3(7) above) —
`roles` claims in the JWT can flow into the ABAC roles work.
**Implementation sketch.**
- New env var `ZDDC_JWT_PUBKEY=/etc/zddc/proxy-pub.pem` or
`ZDDC_JWT_PUBKEY_INLINE=...PEM bytes...`.
- New middleware (peer to `ACLMiddleware` in
`zddc/internal/handler/middleware.go`) that, when configured, reads
`Authorization: Bearer <jwt>` from the request, verifies the
signature, validates `aud == zddc-server`, `exp` in the future,
`nbf` in the past, and extracts `email` (and `roles`) from the
verified claims.
- Refusal mode: when `ZDDC_JWT_PUBKEY` is set and a request lacks a
valid JWT, return 401. No silent fallback — the operator opted in
to cryptographic binding.
- Library choice: `github.com/golang-jwt/jwt/v5` (pure Go, well
maintained) or `github.com/lestrrat-go/jwx` (more features, larger).
Lean toward the simpler `golang-jwt/jwt`.
**Effort estimate:** ~150 lines of Go (middleware + config wiring +
key parsing) + the JWT library dependency + ~30 lines of deployment
docs showing oauth2-proxy and nginx sign-and-forward configurations.
#### Policy export for change control (NIST CM-3)
**Why this is needed.** Federal change-control review wants a baseline
file that can be diffed against current state. For zddc-server, "the
configuration" is the union of every `.zddc` file in the served tree.
Today an operator wanting "what's the effective access at /Project-A?"
has to walk the cascade by hand. The `/.profile/effective-policy`
endpoint shortcuts a single `(path, email)` query, but it doesn't
enumerate the whole policy.
**What the export produces.** For every directory containing a `.zddc`,
the command emits:
- The directory path
- The raw `.zddc` contents
- The resolved effective ACL (the chain back to root, flattened into
this directory's effective allow/deny lists)
- Equivalence-class summary: `(email-class, decision)` pairs covering
groups like `*@mycompany.com → allow`, `*@external.com → deny`,
specific exceptions
Output formats:
- **JSON** — machine-readable for `diff` against a baseline file in a
Git repo or a change-control tool
- **Markdown** — human-readable summary suitable for ATO documentation
- **CSV** — pivot-ready for spreadsheet review
**Use cases:**
- *Baseline + diff workflow.* Operator commits the export to a Git repo.
Every `.zddc` change auto-runs the export in CI; the diff is the
change-control artifact. Reviewers see "alice was added to
/Project-X/.zddc, which means alice can now access /Project-X/,
/Project-X/sub/, … because the cascade propagates."
- *Drift detection.* Export at deploy time vs export every day; any
unexpected change triggers an alert.
- *ATO documentation.* Plug the export into the SSP's "Configuration
Inventory" section.
**Implementation sketch.** Mostly mechanical reuse of existing pieces:
- `zddc.ScanZddcFiles(root)` already walks the tree (used by the
profile page's admin-subtrees listing).
- `zddc.EffectivePolicy(root, dir)` already builds the chain (used by
every ACL check).
- `zddc.MatchesPattern` already covers the email matching.
What's missing is the CLI subcommand wiring (today the binary takes
flags only; needs subcommand parsing for `policy export`), the
equivalence-class computation (group emails by which patterns they
match — small graph problem), and the format renderers.
**Effort estimate:** ~250 lines of Go (CLI subcommand + equivalence-
class computation + JSON/Markdown/CSV renderers) + ~100 lines of tests.
#### Code-signed `apps:` URL fetches (NIST SI-7)
**The supply-chain risk today.** The `apps:` mechanism fetches a URL
once on first request, caches the bytes forever, and serves them to
every viewer below that level. There's no integrity check. If the
fetched URL is ever compromised — DNS hijack, CDN account takeover,
malicious upstream commit, MITM during the one fetch window — every
customer caches the bad bytes. The blast radius is "every user who
visits an archive page in a subtree where this `.zddc` applies."
**Why code signing instead of SHA-256 pinning.** SHA-256 pinning would
require operators to track-and-update a hash in `.zddc` every time an
artifact changes. Wrong workflow for this product. Code signing
sidesteps the operator entirely:
- Release pipeline signs each artifact once at publish time with a
long-lived private key.
- Operator trusts the published public key once and never deals with
hashes.
- zddc-server fetches the URL, downloads the detached signature
(e.g. `<artifact>.sig`), verifies against the configured public key,
caches if valid.
**Implementation has three parts** that interlock:
1. **Signing in the build pipeline.** `./build alpha|beta|release` runs
`sign_release_artifacts` (in `./build`) after promote: walks
`dist/release-output/` and produces a detached Ed25519 signature
(`<artifact>.sig`) alongside every real file. Private key path comes
from `ZDDC_SIGNING_KEY`; absent or unreadable → release fails.
Symlinks (channel mirrors, partial-version pins) skip — the .sig
at the symlink target is what counts.
2. **Public key on the website.** `pubkey.pem` is a real file in
`~/src/zddc-website/`, deployed to `zddc.varasys.io/pubkey.pem`.
The releases-page index includes a "Verify your downloads"
section with a download link, the SHA-256 fingerprint shown in
plain text, and a `curl + openssl pkeyutl -verify` example.
3. **Verifier in zddc-server (`apps/fetch.go`).** When fetching a
URL-pinned `apps:` artifact, also fetch `<url>.sig`, then call
`VerifyEd25519` against `Fetcher.VerifyKey`. The key is loaded
at startup with this resolution order:
1. `--apps-pubkey` / `ZDDC_APPS_PUBKEY` (path to PEM file)
2. `apps_pubkey:` inline PEM in the root `.zddc` file (root-only,
same trust-anchor treatment as `admins:`)
3. nothing → URL fetches refused
Failure cases at fetch time — sig 404, transport error, wrong key,
tampered body — all reject; the body is dropped and the apps
resolver falls back to the embedded copy. No baked-in default
public key; same posture as TLS certificates.
**Trust model.** The operator decides which signing infrastructure to
trust by configuring `--apps-pubkey`. The website publishes the
canonical-channel public key; operators who use `apps: archive: stable`
download it once, save the file on their server, and configure the
path. Operators running their own signing point at their own pubkey
instead. zddc-server has no opinion.
**Future direction.** A `signed_by:` field per-`apps:` entry would let
a single deployment trust multiple signing keys (one for canonical
channels, one for an in-house mirror). Sigstore integration
(transparency-log-backed signing via `github.com/sigstore/sigstore`)
is the federally-acceptable evolution. Both are additive on top of
the current single-key-per-server model.
**What's currently in place.** All three parts. The scaffolding
matches the design above one-to-one; future enhancements are
extensions, not refactors.
### External policy decider (OPA-compatible)
For deployments that need policy decisions made by an external,
independently-audited engine — typically federal customers using
[Open Policy Agent](https://www.openpolicyagent.org/) — zddc-server can
delegate every access decision to an HTTP/Unix-socket endpoint that
speaks OPA's `/v1/data/...` JSON wire protocol.
Set one of:
```sh
ZDDC_OPA_URL=internal # built-in Go evaluator (default)
ZDDC_OPA_URL=http://127.0.0.1:8181 # OPA via HTTP
ZDDC_OPA_URL=https://opa.internal:8181 # OPA via HTTPS
ZDDC_OPA_URL=unix:///run/opa/opa.sock # OPA via Unix socket
```
Internal mode uses zddc-server's in-process evaluator — same Go code that
backs the cascade rules above, no network round-trip, no external dependency.
This is the default.
External mode POSTs each access decision to
`<ZDDC_OPA_URL>/v1/data/zddc/access/allow` with body:
```json
{
"input": {
"user": {"email": "alice@mycompany.com"},
"path": "/Project-A/sub/",
"policy_chain": {
"levels": [
{"acl": {}, "admins": ["admin@mycompany.com"]},
{"acl": {"allow": ["*@mycompany.com"]}}
],
"has_any_file": true
}
}
}
```
OPA evaluates the deployment's Rego policy against this input and returns:
```json
{"result": true}
```
Rego policy authors can implement either:
- **Same semantics as our internal evaluator** — walk
`input.policy_chain.levels` bottom-up, deny-first within a level,
default-deny when `input.policy_chain.has_any_file` is true. The
`.zddc` files in the served tree continue to drive policy unchanged.
- **Federal-mode tightening** — same chain, but parent denies are
absolute (no leaf-allow override of an ancestor's deny). NIST AC-6
least-privilege posture.
- **RBAC-from-IdP** — read additional fields from `input.user`
(e.g. `roles` populated by the upstream proxy from SAML/OIDC claims)
and decide based on those, treating `.zddc` as a file-tree map of
required-roles instead of explicit emails.
- **Anything else** — Rego is general-purpose policy; once you're
running real OPA, the constraints are whatever you write.
### OPA failure modes
External mode adds a network call to the request hot path. zddc-server
treats unreachable / non-2xx / malformed-response cases as **deny**
(fail-closed) by default, with a `WARN` log. Operators who prefer
availability over correctness — typically not federal — can flip this:
```sh
ZDDC_OPA_FAIL_OPEN=1 # allow on transport error
```
Always-WARN logging means a healthy run is silent and a sick OPA is
loud regardless of which mode you pick.
### OPA decision cache
External OPA mode wraps the HTTP/socket client in a small per-decision
cache (default 1 s TTL). A single `.archive` listing or directory
enumeration walks the cascade for every entry, which would otherwise
issue one OPA round-trip per entry; the cache collapses identical
`(email, decision-input)` tuples down to one call per TTL window.
The 1-second default is short enough that a `.zddc` edit is reflected
in the next listing (it's the same window as the fsnotify watcher's
debounce). Operators who want zero staleness — or who are running
their own caching layer in front of OPA — can disable:
```sh
ZDDC_OPA_CACHE_TTL=0 # no caching, every request → OPA
ZDDC_OPA_CACHE_TTL=5s # longer window for batchy workloads
```
The cache is per-process, in-memory, and capped at ~4096 entries with
opportunistic eviction of expired entries. Internal mode (the default)
gets no cache — the in-process Go evaluator is already cheaper than a
cache lookup would be.
### Reference Rego policy
The `--print-rego` flag emits the bundled reference Rego policies. Two
variants ship:
```sh
zddc-server --print-rego # standard cascade (commercial)
zddc-server --print-rego=standard # same
zddc-server --print-rego=federal # parent-deny-is-absolute (NIST AC-6)
```
The standard variant mirrors internal-mode semantics exactly — leaf-
level allows can override an ancestor's deny (the cascade's intentional
delegation property). The federal variant is the strict-least-privilege
posture: any deny anywhere in the chain is absolute, no leaf-level
override possible. Federal customers running their own OPA can drop
the federal Rego in unchanged, or use either as a starting point for
further customization.
Parity is enforced at build time. `zddc/internal/policy/parity_test.go`
imports the OPA Go module **as a test-only dependency**, evaluates both
bundled Regos against fixture sets and asserts:
- The standard Rego matches the internal Go evaluator on every documented
cascade scenario (`TestRegoParity_AllInternalCases`).
- The federal Rego agrees with the standard policy on every case where
no ancestor-deny intersects a leaf-allow, AND **disagrees** on the
cases where the AC-6 rule differs (`TestFederalRego_DivergencesFromStandard`).
This way both policies are guaranteed to behave as documented.
The test-only import means the production binary stays OPA-free (still
13 MB) — the OPA library is in `go.mod` but not in `go build`'s output.
This gives you both ends of the spectrum: a single OPA-aware codebase
where the production decider is pure Go (no library bloat, no extra
process), the wire format is OPA-canonical (just point an external OPA
at it and decisions delegate seamlessly), and the bundled reference
Rego is a parity-tested artifact you can ship alongside or extend.
Typical federal customizations on top of the bundled Rego:
- **Parent-deny-is-absolute** — flip the leaf-allow-overrides-parent-deny
rule for NIST AC-6 least-privilege posture.
- **Role-based access** — read additional input fields like
`input.user.roles` populated by the upstream proxy from SAML/OIDC
claims, and decide based on those instead of (or alongside) email.
- **Time-of-day or IP-range constraints** — Rego can read
`input.context.now` and request metadata for context-aware
decisions.
- **SIEM-shipped decision logs** — OPA's logging plugins emit every
decision in a structured format ready for Splunk Government, Elastic
Federal, etc.
### Reference deployment shapes
**Commercial / default**: nothing to set. `ZDDC_OPA_URL=internal` is
the implicit default; the in-process evaluator handles every decision.
No sidecar, no extra port, no extra binary.
**Federal sidecar**: deploy OPA alongside zddc-server (k8s sidecar,
nomad task, or systemd service on the same host), bind it to
`127.0.0.1:8181` (or a Unix socket), point `ZDDC_OPA_URL` at it. OPA
loads the deployment's bundled Rego policy from a configured source
(filesystem, signed bundle from S3, OPAL, etc.) and is patched
independently of zddc-server.
**Per-tenant policy variants**: run multiple OPA instances each loaded
with a different bundle, point each zddc-server replica at the
appropriate one. The `.zddc` files in the served tree stay the same;
the *interpretation* of those files differs per tenant.
### Future work
Items the conversation flagged as friction in operator setup or as documented
gaps that warrant code, in addition to the federal-readiness items above:
- `.zddc.form.yaml` ACL editor (built on the form-data system) once
file-as-truth round-trip preserves comments — turn the manual YAML edit
into a self-service UI for project owners.
- Save-time validation that warns when a `.zddc` change would lock the
editing user out (or have a measurable effect they didn't anticipate).
- `zddc-server policy export` command emitting every `.zddc` file's resolved
effect, suitable for change-control review (and a prerequisite for the
CM-3 federal control above).
- Per-decision caching for external OPA mode (small TTL on (email, path)
to amortize the .archive listing's per-entry round-trip).
- A reference Rego bundle shipped alongside the binary that exactly
reproduces internal mode, plus a "federal-mode" variant that flips
the parent-deny-is-absolute toggle. Useful as a starting point for
customers who want to extend rather than write from scratch.
## Admin Debug Page
`zddc-server` exposes a built-in debug page at `/.profile/` for operators who can
push code/images but cannot `kubectl exec` into the running container. It surfaces:
- **`/.profile/whoami`** — every header on the current request, the configured email
header name, the value observed at that name, and the resolved email. This is the
first thing to look at when access logs show `email=anonymous` — it tells you
exactly which (if any) header the upstream proxy is sending.
- **`/.profile/config`** — the resolved `Config` (env vars). Equivalent to
`kubectl exec -- env | grep ^ZDDC_` for diagnosing chart / deployment overrides.
- **`/.profile/logs`** — recent log entries (last 500) from an in-memory ring buffer.
Optional `?level=info|warn|error|debug` and `?since=<RFC3339>` query params.
At `ZDDC_LOG_LEVEL=debug` every request also logs its full header map under
`msg=request headers` — useful for diagnosing proxy / SSO header passthrough
(e.g. confirming which header carries the email). Note: that dump includes
auth tokens and cookies; only enable debug in trusted environments.
- **`/.profile/effective-policy?path=...&email=...`** — cascade tracer.
Returns the resolved policy chain (every level along the walk from
`ZDDC_ROOT` to the requested path), the active decider's allow/deny
verdict, the per-level email-match breakdown, and the decider kind
(`*policy.InternalDecider` / `*policy.cachingDecider`). When
`ZDDC_OPA_URL` points at an external OPA, the decision goes through
that OPA — also a useful smoke test for OPA wiring. See "Debugging
permissions" above.
- **`/.profile/`** — HTML dashboard that fetches the JSON endpoints client-side.
### Authorization
Authorization is via an `admins:` list in the **root** `.zddc` file (`<ZDDC_ROOT>/.zddc`).
Patterns use the same glob syntax as `acl.allow` / `acl.deny`:
```yaml
admins:
- alice@mycompany.com
- "*@admin.mycompany.com"
acl:
allow:
- "*@mycompany.com"
```
The root-only invariant (subdirectory `admins:` entries are silently ignored, so
subtree write authority cannot be self-promoted to admin) is documented under
"Trust model and invariants" in the access-control section above — that's the
canonical home; this section just shows the syntax.
If the root `.zddc` has no `admins` list (or no `.zddc` exists), every admin
endpoint returns **404** to every caller. Non-admin requests also receive 404
(not 403) so the existence of the admin page is invisible to unauthorized
callers.
### Forward-auth target for upstream proxies
`zddc-server` also exposes `GET /.auth/admin` — a machine-only endpoint that
returns **200** if the caller's resolved email is in the root `.zddc` `admins:`
list, **403** otherwise. No body, no redirect, no UI; it is a pure
authorization decision intended to be polled by an upstream proxy's
forward-auth directive (Caddy `forward_auth`, nginx `auth_request`, Traefik
`ForwardAuth`, etc.).
The intended use case is gating *adjacent* services on the same pod / host
that don't have their own ACL. Concretely: the dev-shell deployment runs
both `zddc-server` and `code-server` behind one Caddy listener; Caddy uses
`forward_auth` to ask `/.auth/admin` whether the caller is allowed to reach
`/devshell/*` (the IDE) before forwarding. zddc-server's own routes (`/`,
`/<project>/`, `/.archive/`, etc.) keep their existing `.zddc`-cascade ACL
and don't go through this endpoint.
```caddy
# example: protect /devshell/* with forward_auth on /.auth/admin
handle_path /devshell/* {
forward_auth 127.0.0.1:9090 {
uri /.auth/admin
copy_headers X-Auth-Request-Email
}
reverse_proxy 127.0.0.1:8443 # code-server
}
```
The check is cheap (one map lookup against the cached `PolicyChain`); calling
it on every request is fine. Edits to `/srv/.zddc` propagate within the
fsnotify watcher's debounce window (~2 s) — no service restart needed.
### Caveats
- Logs are in-memory and lost on restart. The buffer holds the most recent 500
records; for long-term audit, parse the stderr stream the way you already do.
- The page reads only configuration and request state — it does not modify anything.
- An interactive terminal is not yet available; that's planned as a follow-up
behind a separate `ZDDC_ADMIN_TERM=1` env-var gate so it stays opt-in.
## Apps: virtual tool HTMLs
`zddc-server` virtually serves the five tool HTMLs (archive, transmittal,
classifier, mdedit, landing) at the appropriate paths. The current-stable
build of each tool is **baked into the binary at compile time** via
`//go:embed`; that's the default. No fetch happens out of the box.
### Where each tool is served
| App | Available at |
|---------------|-------------------------------------------------------------------------|
| `archive` | every directory (multi-project, project, archive, vendor) |
| `classifier` | any `Incoming`, `Working`, or `Staging` directory and its subtree |
| `mdedit` | any `Working` directory and its subtree |
| `transmittal` | any `Staging` directory and its subtree |
| `landing` | only at the deployment root (the project picker) |
Outside these locations, the corresponding `<app>.html` URL returns 404.
### Override and version-pin
For any path, the resolution order is:
1. **Real file at the path** — operator drops `archive.html` (or any other)
into a directory; the static handler serves it. Beats everything below.
2. **Closer-to-leaf `.zddc apps:` entry** — walks `.zddc` files leaf→root
for an `apps.<app>` entry. The first match wins. Spec is one of:
- `stable` / `beta` / `alpha` (canonical upstream channel)
- `v0.0.4` / `v0.0` / `v0` (canonical upstream version pin)
- `https://...` (full URL to a custom mirror)
- `./local.html` / `/abs/path.html` (local file)
3. **Embedded** — the build-time HTML compiled into the binary.
URL sources are fetched once on first request and cached forever in
`<ZDDC_ROOT>/_app/<scheme>/<host>[:<port>]/<path>`. There is no background refresh and no
hash verification — to pull a new build, delete the cache file. Concurrent
misses for the same URL share one outbound fetch (singleflight). Direct
URL access to `/_app/...` is blocked at dispatch; cached HTMLs are served
only via the apps resolver.
If a configured URL fetch fails (network down, 5xx), the server falls back
to the embedded copy and emits a one-time WARN log per source. The
`X-ZDDC-Source` response header always reports what was served:
`fetch:URL`, `cache:URL`, `path:/abs`, or `embedded:<app>@<build>`.
### Example
```yaml
# <ZDDC_ROOT>/Project-A/.zddc
apps:
classifier: alpha # track alpha for this project
archive: https://my-mirror.internal/zddc/archive_v0.0.4.html # custom mirror, pinned
mdedit: ./our-mdedit.html # local fork
```
### Env vars
| Variable | Default | Purpose |
|----------------------|---------|----------------------------------------------------------|
| `ZDDC_BUILD_VERSION` | `dev` | String stamped into `X-ZDDC-Source: embedded:<app>@<v>` |
The landing page fetches `GET /` (with `Accept: application/json`) to retrieve the list
of top-level project directories the requesting user has access to. It renders checkboxes
for each project and opens `archive.html?projects=Proj-A,Proj-B` when the user clicks
"Open Archive".
**Presets** (named project selections) are stored in the browser's `localStorage` — no
server-side state required.
**Shared URLs**: the `?projects=` parameter is preserved in the archive browser URL so
users can email direct links to a pre-filtered view. If the recipient does not have
access to a project listed in the URL, a warning banner is shown.
## Caching and ETags
zddc-server uses content-hash ETags wherever a re-fetch of identical
content is plausible — directory listings, the project list, the
embedded HTML tools.
| Endpoint | ETag source | Notes |
|---|---|---|
| `GET /` (project list, `Accept: application/json`) | SHA-256 prefix of the response body | Refetched on every request; the JSON is rebuilt from current FS state. The hash is the actual response, so 304s are always trustworthy regardless of filesystem-watcher reliability. |
| `GET /<dir>/` (directory listing, `Accept: application/json`) | SHA-256 prefix of the response body | Same. |
| `GET /<dir>/` (HTML browse) | Hash of the embedded `browse.html` template | Computed once at startup, memoized. Changes only on binary redeploy. |
| `GET /<app>.html` (embedded tools) | Hash of the embedded bytes | Same — memoized at startup. |
**Why content-hash and not server-side caching?** The cascade walks
`.zddc` files on every directory access; an `os.ReadDir` runs to build
the listing; the ACL filter applies. A cache keyed on directory mtime
would save that server work but depends on reliable filesystem
watching — and Azure Files SMB mounts (a common deployment substrate)
do not support `inotify`/`fsnotify` reliably. Content-hash ETags
deliver only the bandwidth savings, not the server-work savings, but
they cannot lie about staleness regardless of watcher behavior. A
future enhancement could add an mtime-keyed cache for environments
with reliable watchers, behind a feature flag.
The response headers are:
```
Cache-Control: private, max-age=0, must-revalidate
ETag: "<16-hex-char hash>"
```
`must-revalidate` ensures every refresh round-trips the server (which
re-runs the cascade and ACL filter); `max-age=0` means no client-side
freshness window; `private` prevents intermediary caches from sharing
responses across users (each user has their own ACL-filtered view).
## Access Logging
Every HTTP request is logged as a structured `slog` entry at `INFO` level:
| Field | Description |
|---|---|
| `ts` | Request arrival timestamp (RFC3339) |
| `email` | User email from the configured header, or `anonymous` |
| `method` | HTTP method |
| `path` | URL path |
| `status` | HTTP response status code |
| `bytes` | Response body bytes written |
| `duration_ms` | Request duration in milliseconds |
Log output goes to `stderr`. Use `ZDDC_LOG_LEVEL=warn` to suppress access logs if needed,
or pipe `stderr` to a log aggregator.
## Virtual Archive Index (`.archive`)
Any URL path segment named `.archive` (configurable via `ZDDC_INDEX_PATH`) is intercepted
by the server and treated as a virtual document index.
The index is built at startup by scanning all transmittal folders under `ZDDC_ROOT`. It
maps each `(project, trackingNumber, revision, modifier)` tuple to the file from the
**chronologically earliest** transmittal folder within that project that contains it.
### Project scoping
The `.archive` index is **scoped to the project** — i.e. the first slash-separated
segment of the request's `.archive` context path. The same tracking number issued
under two different projects does NOT collide; each project's `.archive/` surfaces
only that project's documents.
A request to `/.archive/...` at the very root has no project segment to scope by
and returns **404 Not Found**. Stable references must always be project-rooted
(e.g. `/ProjectA/.archive/TRK-001.html`).
Within one project, two different files claiming to be the same `(tracking, rev)`
are an authoring mistake. The chronological winner still wins, but a `WARN`
log is emitted with both paths so the conflict can be diagnosed and corrected.
### URL patterns
| URL | Resolves to |
|---|---|
| `GET /Project/.archive/TRK-001.html` | Latest base revision of TRK-001 within Project |
| `GET /Project/.archive/TRK-001_A.html` | Base revision A of TRK-001 within Project |
| `GET /Project/.archive/TRK-001_A+C1.html` | Modifier C1 of revision A of TRK-001 within Project |
| `GET /Project/.archive/` | JSON listing of Project's resolvable entries |
| `GET /Project/sub/sub/.archive/TRK-001.html` | Same as the top-level Project listing — depth within a project doesn't change scope |
| `GET /.archive/...` | **404** — root has no project segment |
Successful `.html` responses **serve the resolved file's bytes inline** at the
`.archive/` URL — no `Location` redirect. The per-transmittal URL is hidden on
purpose: external links of the form `.archive/<tracking>.html#section` keep
tracking the latest revision. A redirect would expose the snapshot URL and any
forwarded link would pin to that snapshot instead of "latest." Cache-Control is
`no-cache` so each load revalidates against the on-disk file's
`Last-Modified`/`ETag`; when a new revision lands the resolver picks it and the
browser refetches. ACL is enforced on both the `.archive` context directory and
the resolved target file (per-target denial returns 404, not 403, to avoid
disclosing that the tracking number exists in a hidden subtree).
### Why "earliest" transmittal?
Within one project, any file claiming to be `TRK-001_A (IFC)` should be identical
across transmittals (same content, same SHA-256). If the same tracking number and
revision appears in multiple transmittals, the first one received chronologically is
treated as the authoritative copy. A later arrival with a different file path is an
error condition; the server logs a `WARN` with both paths but does not change the
winner.
### Index refresh
The index refreshes automatically via an `fsnotify` filesystem watcher. Changes are
debounced by 2 seconds before the relevant transmittal folder is re-indexed.
> **Note for Azure Files**: Azure SMB mounts do not support `inotify`/`fsnotify` reliably.
> The watcher will log a warning and the index will only be updated by restarting the server.
## ZDDC Filename Convention
The server parses filenames following the ZDDC convention:
```
trackingNumber_revision (status) - title.extension
```
| Part | Format | Example |
|---|---|---|
| `trackingNumber` | No spaces or underscores | `123456-EL-SPC-2623` |
| `revision` | `~?[A-Z0-9]+(\+[CBNQ][0-9]+)?` | `A`, `~B`, `C+C1` |
| `status` | One of the valid status codes | `IFC`, `REC`, `---` |
| `title` | Free text | `Electrical Specification` |
Valid status codes: `IFA IFB IFC IFD IFI IFP IFR IFU REC RSA RSB RSC RSD RSI ---`
Transmittal folder format: `YYYY-MM-DD_trackingNumber (STATUS) - title`
## Integration with Archive Browser
The Archive Browser (`archive.html`) can connect to zddc-server in HTTP mode. The server
returns JSON directory listings in exactly the same format as Caddy's `file-server --browse`
— no changes to `archive/js/source.js` are needed.
To use: install `archive.html` at `ZDDC_ROOT/archive.html` (or any subdirectory) — either
the actual built tool downloaded by the self-contained install snippet, or one of the
six-line stubs from the project-subdir / track-upstream snippets that fetches it. Then
open it via the zddc-server URL; the app will auto-connect and scan the directory tree.
## Distribution
Each stable release is a Codeberg git tag (`zddc-server-vX.Y.Z`) with four pre-built binaries attached as release assets:
| File | Platform |
|---|---|
| `zddc-server-linux-amd64` | Linux (x86-64) |
| `zddc-server-darwin-amd64` | macOS (Intel) |
| `zddc-server-darwin-arm64` | macOS (Apple Silicon) |
| `zddc-server-windows-amd64.exe` | Windows (x86-64) |
All binaries are statically linked (CGO disabled), built with `-trimpath -ldflags="-s -w -X main.version=<ver>"`. No runtime dependencies.
Download URLs from Codeberg directly:
```
https://codeberg.org/VARASYS/ZDDC/releases/download/zddc-server-vX.Y.Z/zddc-server-linux-amd64
```
Browse all releases at <https://codeberg.org/VARASYS/ZDDC/releases>.
There is no alpha/beta channel for binary distribution. Active dev/soak happens via the [`helm/zddc-server-dev/`](../helm/zddc-server-dev/) chart, which builds zddc-server from source on every pod restart against any commit you point it at. There is no container image; if you want your own, copy the static binary into a `FROM scratch` or `FROM alpine` base in a few lines, or use one of the helm charts which compile from source via init container.
### Env-var contract (for chart consumers)
Downstream Helm charts and Compose files should set these explicitly:
| Variable | Typical value (behind ingress + SSO) | Purpose |
|---|---|---|
| `ZDDC_ROOT` | `/srv` | Path of the served archive (volume mount) |
| `ZDDC_TLS_CERT` | `none` | TLS terminated upstream |
| `ZDDC_INSECURE_DIRECT` | `1` | Acknowledge plain HTTP behind a trusted proxy |
| `ZDDC_ADDR` | `:8080` | Match service / probe port |
| `ZDDC_EMAIL_HEADER` | `X-Auth-Request-Email` | Header your auth proxy sets |
| `ZDDC_CORS_ORIGIN` | *(unset)* | Leave unset for embedded-tools deployments (same-origin); set to your tool host (`https://tools.acme.com`) only for self-hosted-tools or CDN-bootstrap layouts |
See "Environment Variables" above for the full list.
## Building from source
Requires Go 1.24+.
```sh
# Single binary for the host platform
(cd zddc && go build -o zddc-server ./cmd/zddc-server)
# All four release platforms (cross-compiled, statically linked)
./build # at the repo root — silently skips if Go isn't on PATH
# → outputs to zddc/dist/zddc-server-{linux,darwin,windows}-*
```
To run unit tests:
```sh
(cd zddc && go test ./...)
```
## Release tagging
zddc-server has no separate release script. The repo's top-level `./build alpha|beta|release [version]` is the canonical path: it cross-compiles the four binaries inside the containerized Go toolchain, copies them into `dist/release-output/` with the lockstep symlink chain (one set per platform), regenerates the per-version + per-channel stub pages, refreshes the index, and (on stable cuts) tags `zddc-server-v<X.Y.Z>` alongside the five HTML-tool tags.
```sh
./build release # lockstep stable, coordinated next version
./build release 1.2.0 # lockstep stable, explicit version
./build alpha # lockstep alpha cut
./build beta # lockstep beta cut
./deploy --releases # publish dist/release-output/ to /srv/zddc/releases/
```
The script tags every tool but does NOT push — finish with `git push origin main && git push origin --tags` (and run `./deploy` to put the artifacts on the live site).
Prerequisites:
- Go 1.24+ available inside the build container (downloaded automatically — `docker.io/golang:1.24-alpine`).
- `podman` (preferred) or `docker` on PATH.
Single-developer / solo-release flow by design — no CI babysitting, no separate dashboard to debug. The script fails loudly and visibly on the developer's terminal if anything goes wrong.
### Versioning
Clean semver, lockstep across all six tools (5 HTML + zddc-server). Stable cuts get `<tool>-vX.Y.Z` tags for every tool, all six sharing the same X.Y.Z. There are no alpha/beta tags — channel URLs are stable URLs by design (counters defeat that). Active dev runs via `helm/zddc-server-dev/`, which builds from source on each rollout.
The two existing `zddc-server-v0.0.8-alpha.1` and `zddc-server-v0.0.8-alpha.2` tags from a previous experiment stay as historical artifacts; no new alpha/beta tags are created going forward.
---
**Notes:**
- The `.archive` virtual path resolves ZDDC tracking numbers to their earliest-received revision
- ACL is enforced via bottom-up `.zddc` file evaluation
Reference in a new issue View git blame Copy permalink