VARASYS/ZDDC
1
1
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
ZDDC/zddc/README.md
ZDDC d3a9ea7ad9 feat(server): federal-mode reference Rego (parent-deny-is-absolute) 
Ship a second parity-tested Rego policy that flips the cascade's
leaf-allow-overrides-parent-deny rule for NIST AC-6 conformance.

Standard cascade (existing access.rego, mirrors internal Go evaluator):
  Bottom-up walk; first explicit match wins; deny-first within a level.
  A leaf-level allow CAN override an ancestor's deny. This is the
  cascade's intentional delegation property — a project-owner who
  re-allows a previously-denied collaborator works as expected.

Federal mode (new access_federal.rego):
  Any deny anywhere along the chain is absolute. An allow only matters
  if no level (any depth) has denied the same email. Required by
  NIST AC-6 default expectations: a central admin's deny at the root
  must be unbypassable by a tenant who controls a subtree's .zddc.

Operators run real OPA with this Rego and point ZDDC_OPA_URL at it;
the internal Go evaluator stays on the commercial cascade. The
toggle is "which policy does your OPA evaluate," not a knob inside
zddc-server.

Surfaced via --print-rego flag:

  zddc-server --print-rego               # standard (default)
  zddc-server --print-rego=standard      # same
  zddc-server --print-rego=federal       # AC-6 strict variant

Parity test (federal_parity_test.go) compiles both Regos and asserts:
  * They AGREE on every cascade scenario where no ancestor-deny
    intersects a leaf-allow (most cases).
  * They DISAGREE — by design — on the three scenarios where the
    AC-6 rule differs:
      - "leaf allows what parent denied" → standard allows, federal denies
      - "deep leaf re-allows after middle deny" → same
      - "glob deny at root + specific allow at leaf" → same

Cross-checks the divergence flag explicitly so any future change that
accidentally collapses the two policies fails the test.

Closes the AC-6 row of the federal-readiness gap analysis (now marked
"partially complete" in zddc/README.md — the full bullet would be a
built-in --policy-mode=federal toggle that also flips the in-process
Go evaluator).

Production binary unchanged at 13.1 MB (Rego files embedded as bytes;
OPA library remains test-only).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-04 18:05:44 -05:00

1203 lines
58 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# zddc-server
A purpose-built HTTPS file server for ZDDC document archives. Designed to replace
`caddy file-server --browse` with features specific to ZDDC workflows.
## Features
- **High-performance static file serving** — ETag, conditional GET, Cache-Control
- **ETag on embedded tool HTMLs** — sha256 of the embedded bytes; repeat loads return 304 Not Modified instead of re-shipping 50–920 KB
- **gzip compression middleware** — wraps the entire mux; ~75% size reduction on tool HTMLs and JSON listings (skips bodies under 1 KB)
- **Public landing page** — root `/` is reachable by anyone, including anonymous; per-project ACL filtering still hides projects the caller can't reach
- **Cascading `.zddc` ACL** — email-based allow/deny lists evaluated bottom-up from requested directory to root
- **Caddy-compatible JSON listings** — the Archive Browser works without modification
- **Virtual `.archive` index** — resolve the earliest revision of any tracked document by URL
- **Filesystem watcher** — archive index updates automatically when files change
- **File-based audit log** — JSON-line access log tee'd to `<ZDDC_ROOT>/.zddc.d/logs/access-<host>.log` by default, rotated by lumberjack (100 MB / 10 backups / 90 days, gzipped)
- **Conservative HTTP timeouts** — slowloris-resistant; 10 s read-header, 60 s read+write, 120 s idle
- **Flexible TLS modes** — self-signed, real certificates, or plain HTTP
- **Single static binary** — CGO-free, no runtime dependencies; cross-compiled to Linux/macOS/Windows
## Quick Start
zddc-server ships as a cross-compiled binary distributed via Codeberg release assets.
```sh
# Pick a tag from https://codeberg.org/VARASYS/ZDDC/releases (filter by zddc-server-v*)
curl -L -o zddc-server \
https://codeberg.org/VARASYS/ZDDC/releases/download/zddc-server-vX.Y.Z/zddc-server-linux-amd64
chmod +x zddc-server
# Run against your archive root (HTTPS on :8443 with an in-memory self-signed cert)
ZDDC_ROOT=/srv/archive ./zddc-server
```
Or build from source (requires Go 1.24+):
```sh
git clone https://codeberg.org/VARASYS/ZDDC.git
cd ZDDC/zddc
go build -o zddc-server ./cmd/zddc-server
ZDDC_ROOT=/srv/archive ./zddc-server
```
For plain HTTP behind a reverse proxy, set `ZDDC_TLS_CERT=none` and `ZDDC_INSECURE_DIRECT=1` — see "TLS" below.
There is no Containerfile / Dockerfile / compose file in this repo. Two ways to run zddc-server in Kubernetes / containers:
- The example Helm charts under [`helm/`](../helm/) (`zddc-server-prod/` for stable / `zddc-server-dev/` for tracking main HEAD) compile zddc-server from source via init container — no image registry needed.
- Roll your own image: copy the static binary into a `FROM scratch` or `FROM alpine` base in a few lines.
## Environment Variables
| Variable | Default | Description |
|---|---|---|
| `ZDDC_ROOT` | *(required)* | Absolute path to the served file tree |
| `ZDDC_ADDR` | `:8443` | Bind address (host:port) |
| `ZDDC_TLS_CERT` | *(empty)* | Path to PEM certificate file. `none` = plain HTTP (no TLS); empty = generate self-signed |
| `ZDDC_TLS_KEY` | *(empty)* | Path to PEM private key file. Required when `ZDDC_TLS_CERT` is a file path; ignored otherwise |
| `ZDDC_INSECURE_DIRECT` | *(empty)* | Must be `1` when `ZDDC_TLS_CERT=none` and the bind address is non-loopback. Acknowledges that an authenticating reverse proxy is in front of zddc-server; without it, plain-HTTP non-loopback startup is refused |
| `ZDDC_INSECURE` | *(empty)* | Must be `1` to allow startup when `<ZDDC_ROOT>/.zddc` is missing. Without it, the server refuses to start because no `.zddc` files anywhere → public-by-default access. Set only for deliberately-public deployments |
| `ZDDC_OPA_URL` | `internal` | Policy decider endpoint. `internal` = built-in Go evaluator (default). `http(s)://...` or `unix:///...` = external OPA-compatible server (federal deployments using their own audited Rego). See "External policy decider" below. |
| `ZDDC_OPA_FAIL_OPEN` | *(empty)* | External OPA only. `1` = on transport error, allow the request (availability over correctness). Default = fail closed (deny). Never set to `1` in federal contexts. |
| `ZDDC_OPA_CACHE_TTL` | `1s` | External OPA only. Per-decision cache TTL — bursts of identical queries (a single `.archive` listing can hit the same `(email, dir)` tuple many times) collapse to one OPA round-trip. Set `0` to disable. Format is Go's `time.ParseDuration` (`500ms`, `2s`, `1m`). |
| `ZDDC_LOG_LEVEL` | `info` | Log level: `debug`, `info`, `warn`, `error` |
| `ZDDC_INDEX_PATH` | `.archive` | URL path segment name for the virtual archive index |
| `ZDDC_EMAIL_HEADER` | `X-Auth-Request-Email` | HTTP request header containing the authenticated user's email (the oauth2-proxy / nginx auth-request convention) |
| `ZDDC_CORS_ORIGIN` | *(empty)* | Comma-separated allowlist of origins permitted to make cross-origin requests. Empty (default) disables CORS — appropriate when zddc-server's embedded tools serve same-origin. Set explicitly only if browser-loaded pages from a different origin call back into this server (e.g. `https://tools.acme.com` for self-hosted tools, or `https://zddc.varasys.io` for the CDN-bootstrap pattern) |
| `ZDDC_ACCESS_LOG` | `<ZDDC_ROOT>/.zddc.d/logs/access-<host>.log` | Tee'd structured access log. Auto-mkdir on first run. Empty value (set explicitly with `--access-log=`) disables file logging; stderr stream stays. Per-host filenames let multiple replicas write to the same `.zddc.d/` directory without collision; every record carries a `host` field for downstream aggregation. |
`ZDDC_TLS_CERT=none` disables TLS entirely (plain HTTP). Both cert and key must be set together when using real certificates.
### CORS
CORS is **disabled by default** — `ZDDC_CORS_ORIGIN` defaults to empty.
The embedded-tools install path serves tools and data from the same origin
(both come from zddc-server itself), so no cross-origin allowlist is needed
and there is no implicit cross-origin trust to a third-party host.
Set the value explicitly only when browser-loaded pages from a *different*
origin need to call back into this server. Two scenarios:
```sh
# Self-hosted tools on a separate domain
ZDDC_CORS_ORIGIN=https://tools.acme.com
# CDN-bootstrap pattern (loading tools from the canonical upstream and
# pointing them at your server) — opt in to this trust explicitly
ZDDC_CORS_ORIGIN=https://zddc.varasys.io
```
Multiple origins are comma-separated. The middleware echoes the matched
origin back per-request and sets `Access-Control-Allow-Credentials: true`
so the upstream-set `X-Auth-Request-Email` header crosses the boundary.
> Why empty by default? Earlier releases defaulted this to
> `https://zddc.varasys.io` for the CDN-bootstrap convenience, but every
> deployment then implicitly trusted that origin to make authenticated
> cross-origin XHRs on behalf of any logged-in user. That's an unusual
> trust assumption to bake into a default. Now you opt in explicitly when
> you actually need it.
## TLS
### Plain HTTP (no TLS)
Set `ZDDC_TLS_CERT=none` to run without TLS. Recommended when an upstream reverse proxy
(nginx, Caddy, Traefik) terminates external TLS and talks to zddc-server over plain HTTP
on a private network. zddc-server requires `ZDDC_INSECURE_DIRECT=1` for any non-loopback
bind in this mode — an explicit acknowledgement that an authenticating proxy sits in front:
```sh
ZDDC_ROOT=/srv/archive \
ZDDC_TLS_CERT=none \
ZDDC_ADDR=:8080 \
ZDDC_INSECURE_DIRECT=1 \
./zddc-server
```
When `ZDDC_TLS_CERT` / `ZDDC_TLS_KEY` are empty (or when using real certificates), zddc-server generates an ECDSA P-256
self-signed certificate in memory at startup. The certificate changes on every restart —
this is intentional and acceptable when an upstream reverse proxy terminates external TLS
and uses this server only for encrypted in-datacenter transport.
To use a real certificate (e.g. from Let's Encrypt or an internal CA):
```sh
ZDDC_ROOT=/srv/archive \
ZDDC_TLS_CERT=/etc/ssl/zddc/server.crt \
ZDDC_TLS_KEY=/etc/ssl/zddc/server.key \
./zddc-server
```
## Authentication
zddc-server does **not** perform authentication itself. It reads the user's email address
from a request header (default: `X-Auth-Request-Email`) that must be set by an upstream reverse proxy
(nginx, Caddy, Traefik, Azure Application Gateway, etc.) after authenticating the user.
If the header is absent, the user is treated as anonymous (empty email). A request is
allowed only if (a) **no `.zddc` file exists anywhere in the chain from `ZDDC_ROOT` to
the requested directory** (a fresh tree with zero `.zddc` files defaults to public
access — see warning at the top of the next section), or (b) some level in the chain
explicitly allows the caller's email. See "Access control: the `.zddc` cascade" below
for the full evaluation order.
## Access control: the `.zddc` cascade
> ⚠️ **zddc-server refuses to start without a root `.zddc`.** A `ZDDC_ROOT` containing
> no `.zddc` files anywhere would default to allow-all (anonymous callers included),
> so the server fails fast at startup with a clear error. Pass `--insecure` (or
> `ZDDC_INSECURE=1`) to acknowledge a deliberately-public deployment, otherwise
> drop a starter `<ZDDC_ROOT>/.zddc` per "Step 1" below.
`zddc-server` enforces access via cascading `.zddc` YAML files: drop one in any
directory, and its rules apply to that directory and everything beneath it that doesn't
override. The model is small enough to hold in your head, but the cascade has one
asymmetry that bites operators on first contact — read "When the cascade helps and
when it fights you" below before designing a layout.
```yaml
# Example .zddc file
acl:
allow:
- "*@mycompany.com" # everyone at mycompany.com
- "contractor@partner.com" # specific external user
deny:
- "intern@mycompany.com" # override: block this specific user
```
### Step 1: starter `.zddc`
Every install should write a root `.zddc` before exposing the bind address. The
minimum is an `admins:` line so the admin debug page works (see "Admin Debug Page"
below) — adding `acl:` is optional at this step:
```yaml
# <ZDDC_ROOT>/.zddc — bare minimum
admins:
- you@yourcompany.com
```
With this single file in place, `HasAnyFile` becomes `true` for every directory in
the tree and the default switches from "allow-all-anonymous" to "deny-anything-not-
explicitly-allowed." From here you grant access by adding `acl:` rules at the levels
that need them. (See worked examples below.)
### How a request is evaluated
When a request arrives for `/A/B/C/`, zddc-server reads every `.zddc` file along
the chain from `ZDDC_ROOT` down to `/A/B/C/`, then walks **bottom-up** (deepest
level first) looking for a match. The first explicit match wins — either an allow
or a deny.
1. **At the current level**, check deny patterns first. If the email matches any
deny → **403 Forbidden**, stop walking. *(Important: at the same level, deny
beats allow — see anti-patterns below.)*
2. **Same level**, check allow patterns. If the email matches → **allow**, stop
walking.
3. **No match at this level** → walk up to the parent directory's `.zddc` and
repeat.
4. **No level matched anywhere in the chain:**
- If no `.zddc` file existed anywhere in the chain (`HasAnyFile=false`) → **allow** (the empty-tree default; see warning above).
- If at least one `.zddc` file existed somewhere in the chain (`HasAnyFile=true`) → **403 Forbidden** (default-deny).
The two functions implementing this are `AllowedAtLevel` (within-level: deny first,
then allow) at `zddc/internal/zddc/acl.go:10` and `AllowedWithChain` (deepest-first
walk + default-deny rule) at `zddc/internal/zddc/acl.go:29`. The chain itself is
built by `EffectivePolicy` at `zddc/internal/zddc/cascade.go:25`.
### Glob patterns
`*` matches any sequence of characters within one side of the `@` boundary:
| Pattern | Matches |
|---|---|
| `*@mycompany.com` | Any user at mycompany.com |
| `alice@*` | alice at any domain |
| `*` | Any non-empty email |
| `alice@example.com` | Exact match only |
The `*` does **not** cross the `@`. Implementation at `zddc/internal/zddc/acl.go:52`.
### When the cascade helps and when it fights you
The cascade is well-suited to one shape and clumsy at another. Internalize the
asymmetry before designing your layout:
- **Adding a new email at a leaf is easy** (third parties, occasional contractors).
The new email doesn't match any rule at higher levels, so the cascade just grants
the leaf-level allow and silently default-denies them everywhere else. No extra
`deny:` rules needed.
- **Excluding insiders from a leaf is harder** (commercially sensitive subset of
an otherwise company-wide tree). If a parent has `allow: ["*@company.com"]`, a
leaf-level `allow: [alice@company.com]` *adds* alice on top of everyone — it
does not subtract everyone-else. Subtracting requires either (a) the two-level
gate-and-reallow pattern in the worked example below, or (b) not having the
permissive parent rule in the first place.
This shape is intentional: the cascade is designed for **delegation** (subtree
owners can grant access without coordinating with central admin), not for
fine-grained subtractive policy. If your model is "everyone has access by default,
specific dirs are restricted," push the wildcards downward off the root rather than
fighting the cascade.
### Pick your layout
| Your shape | Recommended pattern |
|---|---|
| Solo / single-user archive | One `.zddc` at root with `admins: [you]`, no `acl:` block — root-only restriction |
| Single small team, full sharing | Root `.zddc` with `acl: {allow: ["*@team.com"]}`. No subdir overrides needed |
| Multi-tenant: each tenant in own subdir | Empty root `.zddc` (just `admins:`), per-tenant `<tenant>/.zddc` with `acl: {allow: [<tenant emails>]}`. Tenants don't see each other's listings |
| Mixed: half open within company, half locked-down to a subset, plus third-party folders | The worked example below — careful: do **not** put `*@company.com` at root |
### Worked example: paired open/closed projects + third-party archive
This is the deployment shape most operators end up with: technical projects are
shared company-wide; their commercial siblings are restricted to a subset; a
separate `/Archive/` tree holds per-vendor folders where each vendor sees only
their own subdir. The cascade handles all three with no `deny:` rules — but only
if you keep the `*@company.com` wildcard *off the root*.
```yaml
# <ZDDC_ROOT>/.zddc — admins only, no broad ACL
admins:
- admin@mycompany.com
```
```yaml
# <ZDDC_ROOT>/Acme-tech/.zddc — open employee project (technical)
acl:
allow: ["*@mycompany.com"]
```
```yaml
# <ZDDC_ROOT>/Acme-comm/.zddc — closed sibling (commercially sensitive)
acl:
allow:
- alice@mycompany.com
- bob@mycompany.com
```
```yaml
# <ZDDC_ROOT>/Archive/.zddc — employees can browse the vendor list
acl:
allow: ["*@mycompany.com"]
```
```yaml
# <ZDDC_ROOT>/Archive/Acme/.zddc — vendor's only window
acl:
allow:
- acme-rep@acme.com
```
Trace for an insider (`alice@mycompany.com`) and a vendor (`acme-rep@acme.com`)
hitting representative paths:
| Path | alice@mycompany.com | acme-rep@acme.com |
|---|---|---|
| `/Acme-tech/` | ✅ matches `*@mycompany.com` at this level | ❌ no match anywhere; `HasAnyFile=true` → 403 |
| `/Acme-comm/` | ✅ matches `alice` at this level | ❌ default-deny |
| `/<other-closed-project>/` | ❌ no match anywhere → 403 | ❌ default-deny |
| `/Archive/` | ✅ matches `*@mycompany.com` | ❌ no match; default-deny |
| `/Archive/Acme/` | ✅ falls up to `/Archive/.zddc`, matches | ✅ matches `acme-rep@acme.com` at this level |
| `/Archive/Acme/Incoming/` | ✅ inherits from `/Archive/.zddc` | ✅ inherits from `/Archive/Acme/.zddc` |
| `/Archive/<other-vendor>/` | ✅ inherits from `/Archive/` | ❌ no match; default-deny |
| `/` (project picker) | Lands; sees the projects she has access to | Lands; project picker filtering hides everything she can't reach |
The vendor reaches `/Archive/Acme/...` only via direct URL (a bookmark or a
transmittal email link). Trying to navigate up to `/Archive/` returns 403; the
archive tool itself treats `/Archive/Acme/` as the root of *its* world (see
"Tool-rooted view" in `ARCHITECTURE.md` § Server security model), so there is
no breadcrumb leading anywhere they can't see.
### Patterns that look secure but aren't
These are the traps. Each is plausible at first glance and doesn't behave as
naive intuition suggests.
1. **Same-level `allow + deny "*@company.com"` does NOT lock the level down to the
allow's targets.** Deny is checked before allow within a single `.zddc`, so the
allowed user's email matches the deny first and is blocked. To exclude insiders
from a leaf, use the two-level gate-and-reallow (parent denies, deeper level
re-allows) — *or* avoid putting `*@company.com` at any ancestor.
```yaml
# /Closed/.zddc — DOES NOT WORK as intended
acl:
allow: [alice@company.com]
deny: ["*@company.com"] # blocks alice too — deny is checked first at same level
```
2. **A leaf-level `allow: [subset]` does NOT restrict when a parent has
`allow: ["*@domain"]`.** Non-subset users hit the leaf with no match, walk up,
match the parent wildcard, and are allowed. Adding a leaf allow is *additive*,
never *subtractive*. (See the asymmetry section above.)
3. **`admins:` in any `.zddc` other than the root is silently ignored.** The check
at `zddc/internal/zddc/file.go:17-20` (and `IsAdmin`) only reads root. This is
the only upward-escalation gate; subtree write access never grants admin.
4. **An `apps:` URL override is a full UI mount, not just a tool version pin.**
Any `.zddc` writer in a subtree can pin `archive: https://attacker.example/...`
and serve arbitrary HTML to every viewer below that level. Subtree write
authority on `.zddc` should be treated as full UI-mounting authority. The
`_app/` cache is fetch-once-and-keep — operators clear it by deleting
`<ZDDC_ROOT>/_app/<scheme>/<host>[:<port>]/<path>`. (See "Apps: virtual tool HTMLs" below for
the resolver order; SHA-256 pinning is on the federal-readiness list, not
currently implemented.)
5. **Relying on `/Archive/` being unbrowsable to "hide" sibling vendor folders'
existence.** Sibling-vendor names are hidden because directories the caller
can't access are omitted from listings (see "Directory visibility" below) —
that's the actual mechanism. Don't rely on the parent dir being denied; rely
on the listing filter.
### Trust model and invariants
These are the invariants security reviewers should expect to find, stated plainly
and tied to the code that enforces them:
- **Auth boundary.** zddc-server does not authenticate. The user's email is read
from the configured request header (default `X-Auth-Request-Email`) set by an
upstream reverse proxy. If the proxy is misconfigured or the bind address is
reachable without traversing the proxy, every request is anonymous — and any
caller able to set the header can claim any email. Network isolation between
the proxy and zddc-server is required (see "Trust boundary" below).
- **Subtree authority.** Whoever can write a `.zddc` controls that subtree's
ACL — including overriding a parent *deny* with a leaf *allow* (test:
`zddc/internal/zddc/acl_test.go:212` "leaf allows user that parent denies →
leaf wins"). This is intentional delegation, not a bug. If you grant write
access to `/Project-A/.zddc`, you've granted full ACL authority over the
Project-A subtree.
- **Root-only escalation gate.** `admins:` is honored *only* at
`<ZDDC_ROOT>/.zddc`. Subtree `admins:` entries are silently ignored
(`zddc/internal/zddc/file.go:17-20`). This is the *only* upward escalation
block; without it, anyone with subtree write authority could promote
themselves to admin.
### Trust boundary
What zddc-server enforces stops at the network boundary. The deployment must
guarantee these for the model above to hold:
1. **The bind address must be reachable only via the authenticating proxy.** The
email-header trust assumes the proxy is the only path to the server. In
Kubernetes: a `NetworkPolicy` restricting ingress to the proxy pod. On a
single host: bind to loopback (`ZDDC_ADDR=127.0.0.1:8080`) and run the proxy
on the same host. **Without this**, anyone reaching the bind address can
forge any email by setting `X-Auth-Request-Email` themselves. zddc-server
refuses to start with `ZDDC_TLS_CERT=none` on a non-loopback bind unless
`ZDDC_INSECURE_DIRECT=1` is set as an explicit acknowledgement that an
authenticating proxy is enforcing this.
2. **Anonymous information disclosure on `/` is by design.** The public landing
page returns a project picker filtered by ACL — anonymous callers see only
projects with no `.zddc` rules along their chain (in a properly-configured
deployment, none). The *existence* of the server and the names of any
ACL-free projects are disclosed without authentication. For deployments
where this disclosure is unacceptable, gate `/` itself behind the proxy's
auth-required path; zddc-server's public-landing logic does not need to be
disabled, it simply never receives an anonymous request. `/.profile` is also
reachable anonymously by design — same caveat applies.
3. **Audit log integrity is filesystem-level, not application-level.** The
tee'd JSON access log lives at `<ZDDC_ROOT>/.zddc.d/logs/access-<host>.log`
on the served volume. While `.`-prefixed paths return 404 over HTTP (so the
log is not readable through zddc-server), anything with **filesystem write
authority on the served volume** (a sidecar, a backup-restore process, an
admin shell) can modify log entries after the fact. For tamper-evident
logging, ship the JSON-line file to an external append-only sink (syslog,
SIEM) via a sidecar; do not treat the local rotation as the system of
record.
4. **`apps:` URL fetches have no integrity check.** Fetched once on first
miss, cached at `<ZDDC_ROOT>/_app/<scheme>/<host>[:<port>]/<path>` forever — no SHA-256 pin,
no signature. Use only URLs you control, treat the apps cache as a trust
boundary, and audit who has `.zddc` write authority where.
### Debugging permissions
When a user reports "I can't see /Project-X/" and you need to figure out why,
the fastest path is the built-in cascade tracer:
```
GET /.profile/effective-policy?path=/Project-X/sub/&email=alice@mycompany.com
```
(Admin-only — 404 to non-admins. Same gate as `/.profile/whoami`,
`/.profile/config`, `/.profile/logs`.)
Returns JSON with the resolved policy chain (every level along the
walk from `ZDDC_ROOT` to the requested directory), the decision the
active decider produces, the per-level email-match breakdown
(`decision_at_level: "allow" | "deny" | "no_match"`), and which
decider is wired in (`*policy.InternalDecider`,
`*policy.cachingDecider`, etc.). With `ZDDC_OPA_URL` pointing at an
external OPA, the decision goes through that OPA — so this endpoint
also doubles as a smoke test for the OPA wiring.
Manual procedure (if the endpoint isn't reachable for some reason):
1. **Confirm the resolved email** — hit `/.profile/whoami` as the
user. Shows every header on the request and the `email` field
zddc-server resolved.
2. **List the chain.** From `<ZDDC_ROOT>` down to the requested
directory, inspect each `.zddc` (most directories have none). For
`/Project-X/sub/sub/`, that's `/.zddc`, `/Project-X/.zddc`,
`/Project-X/sub/.zddc`, `/Project-X/sub/sub/.zddc` — read whatever
exists.
3. **Walk bottom-up.** At each level, mentally run `AllowedAtLevel`:
deny patterns first (any match → blocked), then allow (any match
→ allowed). First explicit match in the bottom-up walk is the
answer. Default-deny if `HasAnyFile=true` and nothing matches.
### Directory visibility
Directories for which the user lacks access are **omitted** from JSON listings entirely —
they are neither listed nor queryable. A direct request to a denied path returns `403`.
This is the mechanism that hides sibling subtrees from a caller. Vendor `acme-rep`
sees an empty-looking `/Archive/` (in fact returns 403 since they're not allowed
there at all in the worked example), and no other vendor's name leaks via listing.
### Reserved hidden segments
Two prefixes are filtered from listings under `ZDDC_ROOT`:
- **`.`-prefixed** (e.g. `/.devshell/`, `/Project-A/.internal/notes.md`) — excluded
from listings **and** 404 on direct HTTP access. The recognized virtual prefixes
(`.archive`, `.admin`) are explicitly permitted through. This lets operators store
side-state (caches, dev-shell home dirs, snapshot staging) on the same volume
that's served, without exposing it.
- **`_`-prefixed** (e.g. `/_template/`) — excluded from listings only. Direct URL
access still works. Use this for operator-managed scaffolding the user shouldn't
browse to but might link to (e.g. a `_template/` directory of stub-HTML examples
to copy into project subdirs).
### How to verify in 5 minutes
This recipe stands up the worked-example layout in a tmpdir, hits each
(email, path) cell with `curl`, and asserts the documented behavior. Run it on
your own deployment to confirm the cascade is doing what you think:
```sh
ROOT=$(mktemp -d)
mkdir -p "$ROOT/Acme-tech" "$ROOT/Acme-comm" "$ROOT/Archive/Acme"
cat > "$ROOT/.zddc" <<'YAML'
admins: [admin@mycompany.com]
YAML
cat > "$ROOT/Acme-tech/.zddc" <<'YAML'
acl: {allow: ["*@mycompany.com"]}
YAML
cat > "$ROOT/Acme-comm/.zddc" <<'YAML'
acl: {allow: [alice@mycompany.com]}
YAML
cat > "$ROOT/Archive/.zddc" <<'YAML'
acl: {allow: ["*@mycompany.com"]}
YAML
cat > "$ROOT/Archive/Acme/.zddc" <<'YAML'
acl: {allow: [acme-rep@acme.com]}
YAML
# Plain HTTP on loopback so curl doesn't need TLS
ZDDC_ROOT="$ROOT" ZDDC_TLS_CERT=none ZDDC_ADDR=127.0.0.1:8090 \
./zddc-server &
SERVER_PID=$!
sleep 1
probe() { # email path expected_status
got=$(curl -s -o /dev/null -w '%{http_code}' \
-H "X-Auth-Request-Email: $1" \
"http://127.0.0.1:8090$2")
printf '%-40s %-30s expected=%s got=%s %s\n' \
"$1" "$2" "$3" "$got" \
"$([ "$got" = "$3" ] && echo OK || echo FAIL)"
}
# Insider — alice should see all the technical + her closed project + Archive tree
probe alice@mycompany.com /Acme-tech/ 200
probe alice@mycompany.com /Acme-comm/ 200
probe alice@mycompany.com /Archive/ 200
probe alice@mycompany.com /Archive/Acme/ 200
# Insider not on the closed project — bob should hit the technical and Archive,
# NOT the closed sibling
probe bob@mycompany.com /Acme-tech/ 200
probe bob@mycompany.com /Acme-comm/ 403
probe bob@mycompany.com /Archive/ 200
probe bob@mycompany.com /Archive/Acme/ 200
# Vendor — acme-rep should ONLY see /Archive/Acme/, blocked everywhere else
probe acme-rep@acme.com /Acme-tech/ 403
probe acme-rep@acme.com /Acme-comm/ 403
probe acme-rep@acme.com /Archive/ 403
probe acme-rep@acme.com /Archive/Acme/ 200
# Anonymous — root .zddc exists, so HasAnyFile=true → default-deny everywhere
# (the root / itself is the public-landing exception; subdirs are gated)
probe '' /Acme-tech/ 403
probe '' /Archive/Acme/ 403
# Anti-pattern: same-level allow + deny *@company.com does NOT lock alice in
mkdir -p "$ROOT/Trap"
cat > "$ROOT/Trap/.zddc" <<'YAML'
acl:
allow: [alice@mycompany.com]
deny: ["*@mycompany.com"] # deny is checked first → blocks alice too
YAML
probe alice@mycompany.com /Trap/ 403 # the trap docs warn about
kill $SERVER_PID
```
Every line should print `OK`. If any prints `FAIL`, the cascade isn't behaving
as documented — file an issue with the failing line. **After every `.zddc`
change in production**, retest at minimum as the editing user (to confirm you
haven't locked yourself out).
### Federal-readiness gap analysis
The current model is well-suited for commercial-tenant ACL with delegated
authentication. To clear federal hurdles (FedRAMP Moderate, NIST 800-53 Rev. 5
baseline, FIPS 140-3, DoD STIG), these gaps would need to be closed. None are
implemented today — this list is informational so security reviewers don't
have to redo the gap analysis from scratch.
- **FIPS 140-3 cryptography** (NIST SC-13) — current build uses Go stdlib
crypto. Required: build with `GOEXPERIMENT=systemcrypto` + RHEL FIPS
userspace, or use the `microsoft/go` (formerly goboring) toolchain.
- **TLS hardening** (NIST SC-8(1)) — *partially complete.* Server now
sets `MinVersion: tls.VersionTLS12`, the NIST SP 800-52 Rev. 2
AEAD-only cipher allowlist (ECDHE+AES-GCM and ECDHE+ChaCha20Poly1305
variants), curve preferences (X25519, P-256, P-384), and emits HSTS
(`max-age=31536000; includeSubDomains`) when zddc-server itself
terminates TLS. *Still required for full DoD STIG conformance:*
OCSP stapling, certificate-transparency-log inclusion, and an
audit-grade documentation pack mapping the cipher list to FIPS
140-3 validated implementations.
- **Authenticated proxy↔server channel** (NIST IA-3) — current trust is
network-level isolation only. Required: mTLS or signed forwarding token
(e.g. JWT signed by the proxy with a key zddc-server validates).
- **Multi-factor authentication** (NIST IA-2(1)) — delegated to upstream
proxy. Required: documented reference deployment with PIV/CAC via
oauth2-proxy or equivalent.
- **Role-based access control** (NIST AC-3(7)) — current model is per-email
allow/deny + a single root-admin role. Required: roles as first-class
entities, `.zddc` syntax for role grants, identity-source-driven role
assignment.
- **Least-privilege bounding** (NIST AC-6) — *partially complete.*
Leaf-allow-overrides-parent-deny is the cascade's intentional
delegation behavior in commercial mode and is preserved in the
internal Go evaluator. For federal deployments, `--print-rego=federal`
emits a parity-tested Rego policy where parent denies are absolute;
drop it into an external OPA and point `ZDDC_OPA_URL` at it. *Still
required for full coverage:* a built-in toggle (e.g. `ZDDC_POLICY_MODE=federal`)
that switches the in-process Go evaluator's semantics without
requiring an OPA sidecar — currently federal-mode is reachable only
via the external-OPA path.
- **Account lifecycle** (NIST AC-2) — emails as identifiers must tie to
authoritative sources (PIV cert subject, IdP-managed identity). Required:
documented integration with at least one IdP supporting federal identity
attestation.
- **Audit log integrity & retention** (NIST AU-9, AU-11) — current 90-day
local rotation is a starting point. Required: tamper-evident logs (signed
log chain or external append-only sink) with 1-year minimum online and
3-year archive.
- **Continuous monitoring hooks** (NIST CA-7) — automated alerting on
`.zddc` changes, admin endpoint use, repeated 403s from one identity.
Required: structured event emission to syslog/SIEM beyond the local file.
- **Supply-chain integrity** (NIST SI-7) — vendored libs (jszip,
docx-preview, xlsx) need SBOM, CVE tracking, automated update pipeline.
`apps:` URL fetches need SHA-256 pinning and signature verification.
- **Data-at-rest encryption** (NIST SC-28) — delegated to the deployment
platform. Required: documented baseline (cloud KMS, LUKS, dm-crypt) with
key-rotation procedures.
- **Vulnerability disclosure process** (NIST SI-5) — repo lacks
`SECURITY.md`. Required: documented disclosure procedure, embargoed-fix
workflow, CVE-assignment policy.
A full SSP / control-by-control mapping consumes this list as input; it is
not a substitute for one.
### External policy decider (OPA-compatible)
For deployments that need policy decisions made by an external,
independently-audited engine — typically federal customers using
[Open Policy Agent](https://www.openpolicyagent.org/) — zddc-server can
delegate every access decision to an HTTP/Unix-socket endpoint that
speaks OPA's `/v1/data/...` JSON wire protocol.
Set one of:
```sh
ZDDC_OPA_URL=internal # built-in Go evaluator (default)
ZDDC_OPA_URL=http://127.0.0.1:8181 # OPA via HTTP
ZDDC_OPA_URL=https://opa.internal:8181 # OPA via HTTPS
ZDDC_OPA_URL=unix:///run/opa/opa.sock # OPA via Unix socket
```
Internal mode uses zddc-server's in-process evaluator — same Go code that
backs the cascade rules above, no network round-trip, no external dependency.
This is the default.
External mode POSTs each access decision to
`<ZDDC_OPA_URL>/v1/data/zddc/access/allow` with body:
```json
{
"input": {
"user": {"email": "alice@mycompany.com"},
"path": "/Project-A/sub/",
"policy_chain": {
"levels": [
{"acl": {}, "admins": ["admin@mycompany.com"]},
{"acl": {"allow": ["*@mycompany.com"]}}
],
"has_any_file": true
}
}
}
```
OPA evaluates the deployment's Rego policy against this input and returns:
```json
{"result": true}
```
Rego policy authors can implement either:
- **Same semantics as our internal evaluator** — walk
`input.policy_chain.levels` bottom-up, deny-first within a level,
default-deny when `input.policy_chain.has_any_file` is true. The
`.zddc` files in the served tree continue to drive policy unchanged.
- **Federal-mode tightening** — same chain, but parent denies are
absolute (no leaf-allow override of an ancestor's deny). NIST AC-6
least-privilege posture.
- **RBAC-from-IdP** — read additional fields from `input.user`
(e.g. `roles` populated by the upstream proxy from SAML/OIDC claims)
and decide based on those, treating `.zddc` as a file-tree map of
required-roles instead of explicit emails.
- **Anything else** — Rego is general-purpose policy; once you're
running real OPA, the constraints are whatever you write.
### OPA failure modes
External mode adds a network call to the request hot path. zddc-server
treats unreachable / non-2xx / malformed-response cases as **deny**
(fail-closed) by default, with a `WARN` log. Operators who prefer
availability over correctness — typically not federal — can flip this:
```sh
ZDDC_OPA_FAIL_OPEN=1 # allow on transport error
```
Always-WARN logging means a healthy run is silent and a sick OPA is
loud regardless of which mode you pick.
### OPA decision cache
External OPA mode wraps the HTTP/socket client in a small per-decision
cache (default 1 s TTL). A single `.archive` listing or directory
enumeration walks the cascade for every entry, which would otherwise
issue one OPA round-trip per entry; the cache collapses identical
`(email, decision-input)` tuples down to one call per TTL window.
The 1-second default is short enough that a `.zddc` edit is reflected
in the next listing (it's the same window as the fsnotify watcher's
debounce). Operators who want zero staleness — or who are running
their own caching layer in front of OPA — can disable:
```sh
ZDDC_OPA_CACHE_TTL=0 # no caching, every request → OPA
ZDDC_OPA_CACHE_TTL=5s # longer window for batchy workloads
```
The cache is per-process, in-memory, and capped at ~4096 entries with
opportunistic eviction of expired entries. Internal mode (the default)
gets no cache — the in-process Go evaluator is already cheaper than a
cache lookup would be.
### Reference Rego policy
The `--print-rego` flag emits the bundled reference Rego policies. Two
variants ship:
```sh
zddc-server --print-rego # standard cascade (commercial)
zddc-server --print-rego=standard # same
zddc-server --print-rego=federal # parent-deny-is-absolute (NIST AC-6)
```
The standard variant mirrors internal-mode semantics exactly — leaf-
level allows can override an ancestor's deny (the cascade's intentional
delegation property). The federal variant is the strict-least-privilege
posture: any deny anywhere in the chain is absolute, no leaf-level
override possible. Federal customers running their own OPA can drop
the federal Rego in unchanged, or use either as a starting point for
further customization.
Parity is enforced at build time. `zddc/internal/policy/parity_test.go`
imports the OPA Go module **as a test-only dependency**, evaluates both
bundled Regos against fixture sets and asserts:
- The standard Rego matches the internal Go evaluator on every documented
cascade scenario (`TestRegoParity_AllInternalCases`).
- The federal Rego agrees with the standard policy on every case where
no ancestor-deny intersects a leaf-allow, AND **disagrees** on the
cases where the AC-6 rule differs (`TestFederalRego_DivergencesFromStandard`).
This way both policies are guaranteed to behave as documented.
The test-only import means the production binary stays OPA-free (still
13 MB) — the OPA library is in `go.mod` but not in `go build`'s output.
This gives you both ends of the spectrum: a single OPA-aware codebase
where the production decider is pure Go (no library bloat, no extra
process), the wire format is OPA-canonical (just point an external OPA
at it and decisions delegate seamlessly), and the bundled reference
Rego is a parity-tested artifact you can ship alongside or extend.
Typical federal customizations on top of the bundled Rego:
- **Parent-deny-is-absolute** — flip the leaf-allow-overrides-parent-deny
rule for NIST AC-6 least-privilege posture.
- **Role-based access** — read additional input fields like
`input.user.roles` populated by the upstream proxy from SAML/OIDC
claims, and decide based on those instead of (or alongside) email.
- **Time-of-day or IP-range constraints** — Rego can read
`input.context.now` and request metadata for context-aware
decisions.
- **SIEM-shipped decision logs** — OPA's logging plugins emit every
decision in a structured format ready for Splunk Government, Elastic
Federal, etc.
### Reference deployment shapes
**Commercial / default**: nothing to set. `ZDDC_OPA_URL=internal` is
the implicit default; the in-process evaluator handles every decision.
No sidecar, no extra port, no extra binary.
**Federal sidecar**: deploy OPA alongside zddc-server (k8s sidecar,
nomad task, or systemd service on the same host), bind it to
`127.0.0.1:8181` (or a Unix socket), point `ZDDC_OPA_URL` at it. OPA
loads the deployment's bundled Rego policy from a configured source
(filesystem, signed bundle from S3, OPAL, etc.) and is patched
independently of zddc-server.
**Per-tenant policy variants**: run multiple OPA instances each loaded
with a different bundle, point each zddc-server replica at the
appropriate one. The `.zddc` files in the served tree stay the same;
the *interpretation* of those files differs per tenant.
### Future work
Items the conversation flagged as friction in operator setup or as documented
gaps that warrant code, in addition to the federal-readiness items above:
- `.zddc.form.yaml` ACL editor (built on the form-data system) once
file-as-truth round-trip preserves comments — turn the manual YAML edit
into a self-service UI for project owners.
- Save-time validation that warns when a `.zddc` change would lock the
editing user out (or have a measurable effect they didn't anticipate).
- `zddc-server policy export` command emitting every `.zddc` file's resolved
effect, suitable for change-control review (and a prerequisite for the
CM-3 federal control above).
- Per-decision caching for external OPA mode (small TTL on (email, path)
to amortize the .archive listing's per-entry round-trip).
- A reference Rego bundle shipped alongside the binary that exactly
reproduces internal mode, plus a "federal-mode" variant that flips
the parent-deny-is-absolute toggle. Useful as a starting point for
customers who want to extend rather than write from scratch.
## Admin Debug Page
`zddc-server` exposes a built-in debug page at `/.profile/` for operators who can
push code/images but cannot `kubectl exec` into the running container. It surfaces:
- **`/.profile/whoami`** — every header on the current request, the configured email
header name, the value observed at that name, and the resolved email. This is the
first thing to look at when access logs show `email=anonymous` — it tells you
exactly which (if any) header the upstream proxy is sending.
- **`/.profile/config`** — the resolved `Config` (env vars). Equivalent to
`kubectl exec -- env | grep ^ZDDC_` for diagnosing chart / deployment overrides.
- **`/.profile/logs`** — recent log entries (last 500) from an in-memory ring buffer.
Optional `?level=info|warn|error|debug` and `?since=<RFC3339>` query params.
At `ZDDC_LOG_LEVEL=debug` every request also logs its full header map under
`msg=request headers` — useful for diagnosing proxy / SSO header passthrough
(e.g. confirming which header carries the email). Note: that dump includes
auth tokens and cookies; only enable debug in trusted environments.
- **`/.profile/effective-policy?path=...&email=...`** — cascade tracer.
Returns the resolved policy chain (every level along the walk from
`ZDDC_ROOT` to the requested path), the active decider's allow/deny
verdict, the per-level email-match breakdown, and the decider kind
(`*policy.InternalDecider` / `*policy.cachingDecider`). When
`ZDDC_OPA_URL` points at an external OPA, the decision goes through
that OPA — also a useful smoke test for OPA wiring. See "Debugging
permissions" above.
- **`/.profile/`** — HTML dashboard that fetches the JSON endpoints client-side.
### Authorization
Authorization is via an `admins:` list in the **root** `.zddc` file (`<ZDDC_ROOT>/.zddc`).
Patterns use the same glob syntax as `acl.allow` / `acl.deny`:
```yaml
admins:
- alice@mycompany.com
- "*@admin.mycompany.com"
acl:
allow:
- "*@mycompany.com"
```
The root-only invariant (subdirectory `admins:` entries are silently ignored, so
subtree write authority cannot be self-promoted to admin) is documented under
"Trust model and invariants" in the access-control section above — that's the
canonical home; this section just shows the syntax.
If the root `.zddc` has no `admins` list (or no `.zddc` exists), every admin
endpoint returns **404** to every caller. Non-admin requests also receive 404
(not 403) so the existence of the admin page is invisible to unauthorized
callers.
### Forward-auth target for upstream proxies
`zddc-server` also exposes `GET /.auth/admin` — a machine-only endpoint that
returns **200** if the caller's resolved email is in the root `.zddc` `admins:`
list, **403** otherwise. No body, no redirect, no UI; it is a pure
authorization decision intended to be polled by an upstream proxy's
forward-auth directive (Caddy `forward_auth`, nginx `auth_request`, Traefik
`ForwardAuth`, etc.).
The intended use case is gating *adjacent* services on the same pod / host
that don't have their own ACL. Concretely: the dev-shell deployment runs
both `zddc-server` and `code-server` behind one Caddy listener; Caddy uses
`forward_auth` to ask `/.auth/admin` whether the caller is allowed to reach
`/devshell/*` (the IDE) before forwarding. zddc-server's own routes (`/`,
`/<project>/`, `/.archive/`, etc.) keep their existing `.zddc`-cascade ACL
and don't go through this endpoint.
```caddy
# example: protect /devshell/* with forward_auth on /.auth/admin
handle_path /devshell/* {
forward_auth 127.0.0.1:9090 {
uri /.auth/admin
copy_headers X-Auth-Request-Email
}
reverse_proxy 127.0.0.1:8443 # code-server
}
```
The check is cheap (one map lookup against the cached `PolicyChain`); calling
it on every request is fine. Edits to `/srv/.zddc` propagate within the
fsnotify watcher's debounce window (~2 s) — no service restart needed.
### Caveats
- Logs are in-memory and lost on restart. The buffer holds the most recent 500
records; for long-term audit, parse the stderr stream the way you already do.
- The page reads only configuration and request state — it does not modify anything.
- An interactive terminal is not yet available; that's planned as a follow-up
behind a separate `ZDDC_ADMIN_TERM=1` env-var gate so it stays opt-in.
## Apps: virtual tool HTMLs
`zddc-server` virtually serves the five tool HTMLs (archive, transmittal,
classifier, mdedit, landing) at the appropriate paths. The current-stable
build of each tool is **baked into the binary at compile time** via
`//go:embed`; that's the default. No fetch happens out of the box.
### Where each tool is served
| App | Available at |
|---------------|-------------------------------------------------------------------------|
| `archive` | every directory (multi-project, project, archive, vendor) |
| `classifier` | any `Incoming`, `Working`, or `Staging` directory and its subtree |
| `mdedit` | any `Working` directory and its subtree |
| `transmittal` | any `Staging` directory and its subtree |
| `landing` | only at the deployment root (the project picker) |
Outside these locations, the corresponding `<app>.html` URL returns 404.
### Override and version-pin
For any path, the resolution order is:
1. **Real file at the path** — operator drops `archive.html` (or any other)
into a directory; the static handler serves it. Beats everything below.
2. **Closer-to-leaf `.zddc apps:` entry** — walks `.zddc` files leaf→root
for an `apps.<app>` entry. The first match wins. Spec is one of:
- `stable` / `beta` / `alpha` (canonical upstream channel)
- `v0.0.4` / `v0.0` / `v0` (canonical upstream version pin)
- `https://...` (full URL to a custom mirror)
- `./local.html` / `/abs/path.html` (local file)
3. **Embedded** — the build-time HTML compiled into the binary.
URL sources are fetched once on first request and cached forever in
`<ZDDC_ROOT>/_app/<scheme>/<host>[:<port>]/<path>`. There is no background refresh and no
hash verification — to pull a new build, delete the cache file. Concurrent
misses for the same URL share one outbound fetch (singleflight). Direct
URL access to `/_app/...` is blocked at dispatch; cached HTMLs are served
only via the apps resolver.
If a configured URL fetch fails (network down, 5xx), the server falls back
to the embedded copy and emits a one-time WARN log per source. The
`X-ZDDC-Source` response header always reports what was served:
`fetch:URL`, `cache:URL`, `path:/abs`, or `embedded:<app>@<build>`.
### Example
```yaml
# <ZDDC_ROOT>/Project-A/.zddc
apps:
classifier: alpha # track alpha for this project
archive: https://my-mirror.internal/zddc/archive_v0.0.4.html # custom mirror, pinned
mdedit: ./our-mdedit.html # local fork
```
### Env vars
| Variable | Default | Purpose |
|----------------------|---------|----------------------------------------------------------|
| `ZDDC_BUILD_VERSION` | `dev` | String stamped into `X-ZDDC-Source: embedded:<app>@<v>` |
The landing page fetches `GET /` (with `Accept: application/json`) to retrieve the list
of top-level project directories the requesting user has access to. It renders checkboxes
for each project and opens `archive.html?projects=Proj-A,Proj-B` when the user clicks
"Open Archive".
**Presets** (named project selections) are stored in the browser's `localStorage` — no
server-side state required.
**Shared URLs**: the `?projects=` parameter is preserved in the archive browser URL so
users can email direct links to a pre-filtered view. If the recipient does not have
access to a project listed in the URL, a warning banner is shown.
## Caching and ETags
zddc-server uses content-hash ETags wherever a re-fetch of identical
content is plausible — directory listings, the project list, the
embedded HTML tools.
| Endpoint | ETag source | Notes |
|---|---|---|
| `GET /` (project list, `Accept: application/json`) | SHA-256 prefix of the response body | Refetched on every request; the JSON is rebuilt from current FS state. The hash is the actual response, so 304s are always trustworthy regardless of filesystem-watcher reliability. |
| `GET /<dir>/` (directory listing, `Accept: application/json`) | SHA-256 prefix of the response body | Same. |
| `GET /<dir>/` (HTML browse) | Hash of the embedded `browse.html` template | Computed once at startup, memoized. Changes only on binary redeploy. |
| `GET /<app>.html` (embedded tools) | Hash of the embedded bytes | Same — memoized at startup. |
**Why content-hash and not server-side caching?** The cascade walks
`.zddc` files on every directory access; an `os.ReadDir` runs to build
the listing; the ACL filter applies. A cache keyed on directory mtime
would save that server work but depends on reliable filesystem
watching — and Azure Files SMB mounts (a common deployment substrate)
do not support `inotify`/`fsnotify` reliably. Content-hash ETags
deliver only the bandwidth savings, not the server-work savings, but
they cannot lie about staleness regardless of watcher behavior. A
future enhancement could add an mtime-keyed cache for environments
with reliable watchers, behind a feature flag.
The response headers are:
```
Cache-Control: private, max-age=0, must-revalidate
ETag: "<16-hex-char hash>"
```
`must-revalidate` ensures every refresh round-trips the server (which
re-runs the cascade and ACL filter); `max-age=0` means no client-side
freshness window; `private` prevents intermediary caches from sharing
responses across users (each user has their own ACL-filtered view).
## Access Logging
Every HTTP request is logged as a structured `slog` entry at `INFO` level:
| Field | Description |
|---|---|
| `ts` | Request arrival timestamp (RFC3339) |
| `email` | User email from the configured header, or `anonymous` |
| `method` | HTTP method |
| `path` | URL path |
| `status` | HTTP response status code |
| `bytes` | Response body bytes written |
| `duration_ms` | Request duration in milliseconds |
Log output goes to `stderr`. Use `ZDDC_LOG_LEVEL=warn` to suppress access logs if needed,
or pipe `stderr` to a log aggregator.
## Virtual Archive Index (`.archive`)
Any URL path segment named `.archive` (configurable via `ZDDC_INDEX_PATH`) is intercepted
by the server and treated as a virtual document index.
The index is built at startup by scanning all transmittal folders under `ZDDC_ROOT`. It
maps each `(project, trackingNumber, revision, modifier)` tuple to the file from the
**chronologically earliest** transmittal folder within that project that contains it.
### Project scoping
The `.archive` index is **scoped to the project** — i.e. the first slash-separated
segment of the request's `.archive` context path. The same tracking number issued
under two different projects does NOT collide; each project's `.archive/` surfaces
only that project's documents.
A request to `/.archive/...` at the very root has no project segment to scope by
and returns **404 Not Found**. Stable references must always be project-rooted
(e.g. `/ProjectA/.archive/TRK-001.html`).
Within one project, two different files claiming to be the same `(tracking, rev)`
are an authoring mistake. The chronological winner still wins, but a `WARN`
log is emitted with both paths so the conflict can be diagnosed and corrected.
### URL patterns
| URL | Resolves to |
|---|---|
| `GET /Project/.archive/TRK-001.html` | Latest base revision of TRK-001 within Project |
| `GET /Project/.archive/TRK-001_A.html` | Base revision A of TRK-001 within Project |
| `GET /Project/.archive/TRK-001_A+C1.html` | Modifier C1 of revision A of TRK-001 within Project |
| `GET /Project/.archive/` | JSON listing of Project's resolvable entries |
| `GET /Project/sub/sub/.archive/TRK-001.html` | Same as the top-level Project listing — depth within a project doesn't change scope |
| `GET /.archive/...` | **404** — root has no project segment |
All successful responses are `302 Found` redirects to the actual file URL. ACL
is enforced on both the `.archive` context directory and the resolved target file.
### Why "earliest" transmittal?
Within one project, any file claiming to be `TRK-001_A (IFC)` should be identical
across transmittals (same content, same SHA-256). If the same tracking number and
revision appears in multiple transmittals, the first one received chronologically is
treated as the authoritative copy. A later arrival with a different file path is an
error condition; the server logs a `WARN` with both paths but does not change the
winner.
### Index refresh
The index refreshes automatically via an `fsnotify` filesystem watcher. Changes are
debounced by 2 seconds before the relevant transmittal folder is re-indexed.
> **Note for Azure Files**: Azure SMB mounts do not support `inotify`/`fsnotify` reliably.
> The watcher will log a warning and the index will only be updated by restarting the server.
## ZDDC Filename Convention
The server parses filenames following the ZDDC convention:
```
trackingNumber_revision (status) - title.extension
```
| Part | Format | Example |
|---|---|---|
| `trackingNumber` | No spaces or underscores | `123456-EL-SPC-2623` |
| `revision` | `~?[A-Z0-9]+(\+[CBNQ][0-9]+)?` | `A`, `~B`, `C+C1` |
| `status` | One of the valid status codes | `IFC`, `REC`, `---` |
| `title` | Free text | `Electrical Specification` |
Valid status codes: `IFA IFB IFC IFD IFI IFP IFR IFU REC RSA RSB RSC RSD RSI ---`
Transmittal folder format: `YYYY-MM-DD_trackingNumber (STATUS) - title`
## Integration with Archive Browser
The Archive Browser (`archive.html`) can connect to zddc-server in HTTP mode. The server
returns JSON directory listings in exactly the same format as Caddy's `file-server --browse`
— no changes to `archive/js/source.js` are needed.
To use: install `archive.html` at `ZDDC_ROOT/archive.html` (or any subdirectory) — either
the actual built tool downloaded by the self-contained install snippet, or one of the
six-line stubs from the project-subdir / track-upstream snippets that fetches it. Then
open it via the zddc-server URL; the app will auto-connect and scan the directory tree.
## Distribution
Each stable release is a Codeberg git tag (`zddc-server-vX.Y.Z`) with four pre-built binaries attached as release assets:
| File | Platform |
|---|---|
| `zddc-server-linux-amd64` | Linux (x86-64) |
| `zddc-server-darwin-amd64` | macOS (Intel) |
| `zddc-server-darwin-arm64` | macOS (Apple Silicon) |
| `zddc-server-windows-amd64.exe` | Windows (x86-64) |
All binaries are statically linked (CGO disabled), built with `-trimpath -ldflags="-s -w -X main.version=<ver>"`. No runtime dependencies.
Download URLs from Codeberg directly:
```
https://codeberg.org/VARASYS/ZDDC/releases/download/zddc-server-vX.Y.Z/zddc-server-linux-amd64
```
Browse all releases at <https://codeberg.org/VARASYS/ZDDC/releases>.
There is no alpha/beta channel for binary distribution. Active dev/soak happens via the [`helm/zddc-server-dev/`](../helm/zddc-server-dev/) chart, which builds zddc-server from source on every pod restart against any commit you point it at. There is no container image; if you want your own, copy the static binary into a `FROM scratch` or `FROM alpine` base in a few lines, or use one of the helm charts which compile from source via init container.
### Env-var contract (for chart consumers)
Downstream Helm charts and Compose files should set these explicitly:
| Variable | Typical value (behind ingress + SSO) | Purpose |
|---|---|---|
| `ZDDC_ROOT` | `/srv` | Path of the served archive (volume mount) |
| `ZDDC_TLS_CERT` | `none` | TLS terminated upstream |
| `ZDDC_INSECURE_DIRECT` | `1` | Acknowledge plain HTTP behind a trusted proxy |
| `ZDDC_ADDR` | `:8080` | Match service / probe port |
| `ZDDC_EMAIL_HEADER` | `X-Auth-Request-Email` | Header your auth proxy sets |
| `ZDDC_CORS_ORIGIN` | *(unset)* | Leave unset for embedded-tools deployments (same-origin); set to your tool host (`https://tools.acme.com`) only for self-hosted-tools or CDN-bootstrap layouts |
See "Environment Variables" above for the full list.
## Building from source
Requires Go 1.24+.
```sh
# Single binary for the host platform
(cd zddc && go build -o zddc-server ./cmd/zddc-server)
# All four release platforms (cross-compiled, statically linked)
./build # at the repo root — silently skips if Go isn't on PATH
# → outputs to zddc/dist/zddc-server-{linux,darwin,windows}-*
```
To run unit tests:
```sh
(cd zddc && go test ./...)
```
## Release tagging
zddc-server has no separate release script. The repo's top-level `./build alpha|beta|release [version]` is the canonical path: it cross-compiles the four binaries inside the containerized Go toolchain, copies them into `dist/release-output/` with the lockstep symlink chain (one set per platform), regenerates the per-version + per-channel stub pages, refreshes the index, and (on stable cuts) tags `zddc-server-v<X.Y.Z>` alongside the five HTML-tool tags.
```sh
./build release # lockstep stable, coordinated next version
./build release 1.2.0 # lockstep stable, explicit version
./build alpha # lockstep alpha cut
./build beta # lockstep beta cut
./deploy --releases # publish dist/release-output/ to /srv/zddc/releases/
```
The script tags every tool but does NOT push — finish with `git push origin main && git push origin --tags` (and run `./deploy` to put the artifacts on the live site).
Prerequisites:
- Go 1.24+ available inside the build container (downloaded automatically — `docker.io/golang:1.24-alpine`).
- `podman` (preferred) or `docker` on PATH.
Single-developer / solo-release flow by design — no CI babysitting, no separate dashboard to debug. The script fails loudly and visibly on the developer's terminal if anything goes wrong.
### Versioning
Clean semver, lockstep across all six tools (5 HTML + zddc-server). Stable cuts get `<tool>-vX.Y.Z` tags for every tool, all six sharing the same X.Y.Z. There are no alpha/beta tags — channel URLs are stable URLs by design (counters defeat that). Active dev runs via `helm/zddc-server-dev/`, which builds from source on each rollout.
The two existing `zddc-server-v0.0.8-alpha.1` and `zddc-server-v0.0.8-alpha.2` tags from a previous experiment stay as historical artifacts; no new alpha/beta tags are created going forward.
---
**Notes:**
- The `.archive` virtual path resolves ZDDC tracking numbers to their earliest-received revision
- ACL is enforced via bottom-up `.zddc` file evaluation
Reference in a new issue View git blame Copy permalink