Resolved `.archive/<tracking>.html` URLs now serve the target file's bytes inline via http.ServeFile with Cache-Control: no-cache, replacing the previous 302 redirect to the per-transmittal URL. Why: external links like `.archive/<tracking>.html#section` are meant to track the latest revision. A redirect exposes the snapshot URL — any forwarded link then pins to that snapshot instead of "latest." Serving in-place keeps the `.archive/` URL stable as the resolver's "current" target moves over time. Cache-Control: no-cache is intentional. Each load revalidates against the on-disk file's Last-Modified/ETag, so when a new revision lands the resolver picks it and the browser refetches transparently. ACL is unchanged: enforced on both the `.archive` context directory and the resolved target file (per-target denial returns 404, not 403, to avoid disclosing that a tracking number exists in a hidden subtree). archivehandler_test.go status expectations updated 302 → 200; fixture bodies adjusted for body-content verification of the in-place serve. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1575 lines
77 KiB
Markdown
1575 lines
77 KiB
Markdown
# zddc-server
|
||
|
||
A purpose-built HTTPS file server for ZDDC document archives. Designed to replace
|
||
`caddy file-server --browse` with features specific to ZDDC workflows.
|
||
|
||
## Features
|
||
|
||
- **High-performance static file serving** — ETag, conditional GET, Cache-Control
|
||
- **ETag on embedded tool HTMLs** — sha256 of the embedded bytes; repeat loads return 304 Not Modified instead of re-shipping 50–920 KB
|
||
- **gzip compression middleware** — wraps the entire mux; ~75% size reduction on tool HTMLs and JSON listings (skips bodies under 1 KB)
|
||
- **Public landing page** — root `/` is reachable by anyone, including anonymous; per-project ACL filtering still hides projects the caller can't reach
|
||
- **Cascading `.zddc` ACL** — email-based allow/deny lists evaluated bottom-up from requested directory to root
|
||
- **Caddy-compatible JSON listings** — the Archive Browser works without modification
|
||
- **Virtual `.archive` index** — resolve the earliest revision of any tracked document by URL
|
||
- **Filesystem watcher** — archive index updates automatically when files change
|
||
- **File-based audit log** — JSON-line access log tee'd to `<ZDDC_ROOT>/.zddc.d/logs/access-<host>.log` by default, rotated by lumberjack (100 MB / 10 backups / 90 days, gzipped)
|
||
- **Conservative HTTP timeouts** — slowloris-resistant; 10 s read-header, 60 s read+write, 120 s idle
|
||
- **Flexible TLS modes** — self-signed, real certificates, or plain HTTP
|
||
- **Single static binary** — CGO-free, no runtime dependencies; cross-compiled to Linux/macOS/Windows
|
||
|
||
## Quick Start
|
||
|
||
zddc-server ships as a cross-compiled binary distributed via Codeberg release assets.
|
||
|
||
```sh
|
||
# Pick a tag from https://codeberg.org/VARASYS/ZDDC/releases (filter by zddc-server-v*)
|
||
curl -L -o zddc-server \
|
||
https://codeberg.org/VARASYS/ZDDC/releases/download/zddc-server-vX.Y.Z/zddc-server-linux-amd64
|
||
chmod +x zddc-server
|
||
|
||
# Run against your archive root (HTTPS on :8443 with an in-memory self-signed cert)
|
||
ZDDC_ROOT=/srv/archive ./zddc-server
|
||
```
|
||
|
||
Or build from source (requires Go 1.24+):
|
||
|
||
```sh
|
||
git clone https://codeberg.org/VARASYS/ZDDC.git
|
||
cd ZDDC/zddc
|
||
go build -o zddc-server ./cmd/zddc-server
|
||
ZDDC_ROOT=/srv/archive ./zddc-server
|
||
```
|
||
|
||
For plain HTTP behind a reverse proxy, set `ZDDC_TLS_CERT=none` and `ZDDC_INSECURE_DIRECT=1` — see "TLS" below.
|
||
|
||
There is no Containerfile / Dockerfile / compose file in this repo. Two ways to run zddc-server in Kubernetes / containers:
|
||
|
||
- The example Helm charts under [`helm/`](../helm/) (`zddc-server-prod/` for stable / `zddc-server-dev/` for tracking main HEAD) compile zddc-server from source via init container — no image registry needed.
|
||
- Roll your own image: copy the static binary into a `FROM scratch` or `FROM alpine` base in a few lines.
|
||
|
||
## Environment Variables
|
||
|
||
| Variable | Default | Description |
|
||
|---|---|---|
|
||
| `ZDDC_ROOT` | *(required)* | Absolute path to the served file tree |
|
||
| `ZDDC_ADDR` | `:8443` | Bind address (host:port) |
|
||
| `ZDDC_TLS_CERT` | *(empty)* | Path to PEM certificate file. `none` = plain HTTP (no TLS); empty = generate self-signed |
|
||
| `ZDDC_TLS_KEY` | *(empty)* | Path to PEM private key file. Required when `ZDDC_TLS_CERT` is a file path; ignored otherwise |
|
||
| `ZDDC_INSECURE_DIRECT` | *(empty)* | Must be `1` when `ZDDC_TLS_CERT=none` and the bind address is non-loopback. Acknowledges that an authenticating reverse proxy is in front of zddc-server; without it, plain-HTTP non-loopback startup is refused |
|
||
| `ZDDC_INSECURE` | *(empty)* | Must be `1` to allow startup when `<ZDDC_ROOT>/.zddc` is missing. Without it, the server refuses to start because no `.zddc` files anywhere → public-by-default access. Set only for deliberately-public deployments |
|
||
| `ZDDC_OPA_URL` | `internal` | Policy decider endpoint. `internal` = built-in Go evaluator (default). `http(s)://...` or `unix:///...` = external OPA-compatible server (federal deployments using their own audited Rego). See "External policy decider" below. |
|
||
| `ZDDC_OPA_FAIL_OPEN` | *(empty)* | External OPA only. `1` = on transport error, allow the request (availability over correctness). Default = fail closed (deny). Never set to `1` in federal contexts. |
|
||
| `ZDDC_OPA_CACHE_TTL` | `1s` | External OPA only. Per-decision cache TTL — bursts of identical queries (a single `.archive` listing can hit the same `(email, dir)` tuple many times) collapse to one OPA round-trip. Set `0` to disable. Format is Go's `time.ParseDuration` (`500ms`, `2s`, `1m`). |
|
||
| `ZDDC_APPS_PUBKEY` | *(empty)* | Path to a PEM-encoded Ed25519 public key used to verify signatures on URL-fetched `apps:` artifacts. Empty (default) = URL-fetched apps refused; only embedded + local-path apps work. Operators using zddc.varasys.io's canonical channels download `pubkey.pem` from there and pass the local path here. Operators with their own signing infrastructure pass their own public key. Same posture as `ZDDC_TLS_CERT` — zddc-server bakes nothing in. **Alternative inline form:** the same key can be pasted as `apps_pubkey:` in the root `<ZDDC_ROOT>/.zddc` (root-only, like `admins:`). The env/flag wins when both are set. |
|
||
| `ZDDC_LOG_LEVEL` | `info` | Log level: `debug`, `info`, `warn`, `error` |
|
||
| `ZDDC_INDEX_PATH` | `.archive` | URL path segment name for the virtual archive index |
|
||
| `ZDDC_EMAIL_HEADER` | `X-Auth-Request-Email` | HTTP request header containing the authenticated user's email (the oauth2-proxy / nginx auth-request convention) |
|
||
| `ZDDC_CORS_ORIGIN` | *(empty)* | Comma-separated allowlist of origins permitted to make cross-origin requests. Empty (default) disables CORS — appropriate when zddc-server's embedded tools serve same-origin. Set explicitly only if browser-loaded pages from a different origin call back into this server (e.g. `https://tools.acme.com` for self-hosted tools, or `https://zddc.varasys.io` for the CDN-bootstrap pattern) |
|
||
| `ZDDC_ACCESS_LOG` | `<ZDDC_ROOT>/.zddc.d/logs/access-<host>.log` | Tee'd structured access log. Auto-mkdir on first run. Empty value (set explicitly with `--access-log=`) disables file logging; stderr stream stays. Per-host filenames let multiple replicas write to the same `.zddc.d/` directory without collision; every record carries a `host` field for downstream aggregation. |
|
||
|
||
`ZDDC_TLS_CERT=none` disables TLS entirely (plain HTTP). Both cert and key must be set together when using real certificates.
|
||
|
||
### CORS
|
||
|
||
CORS is **disabled by default** — `ZDDC_CORS_ORIGIN` defaults to empty.
|
||
The embedded-tools install path serves tools and data from the same origin
|
||
(both come from zddc-server itself), so no cross-origin allowlist is needed
|
||
and there is no implicit cross-origin trust to a third-party host.
|
||
|
||
Set the value explicitly only when browser-loaded pages from a *different*
|
||
origin need to call back into this server. Two scenarios:
|
||
|
||
```sh
|
||
# Self-hosted tools on a separate domain
|
||
ZDDC_CORS_ORIGIN=https://tools.acme.com
|
||
|
||
# CDN-bootstrap pattern (loading tools from the canonical upstream and
|
||
# pointing them at your server) — opt in to this trust explicitly
|
||
ZDDC_CORS_ORIGIN=https://zddc.varasys.io
|
||
```
|
||
|
||
Multiple origins are comma-separated. The middleware echoes the matched
|
||
origin back per-request and sets `Access-Control-Allow-Credentials: true`
|
||
so the upstream-set `X-Auth-Request-Email` header crosses the boundary.
|
||
|
||
> Why empty by default? Earlier releases defaulted this to
|
||
> `https://zddc.varasys.io` for the CDN-bootstrap convenience, but every
|
||
> deployment then implicitly trusted that origin to make authenticated
|
||
> cross-origin XHRs on behalf of any logged-in user. That's an unusual
|
||
> trust assumption to bake into a default. Now you opt in explicitly when
|
||
> you actually need it.
|
||
|
||
## TLS
|
||
|
||
### Plain HTTP (no TLS)
|
||
|
||
Set `ZDDC_TLS_CERT=none` to run without TLS. Recommended when an upstream reverse proxy
|
||
(nginx, Caddy, Traefik) terminates external TLS and talks to zddc-server over plain HTTP
|
||
on a private network. zddc-server requires `ZDDC_INSECURE_DIRECT=1` for any non-loopback
|
||
bind in this mode — an explicit acknowledgement that an authenticating proxy sits in front:
|
||
|
||
```sh
|
||
ZDDC_ROOT=/srv/archive \
|
||
ZDDC_TLS_CERT=none \
|
||
ZDDC_ADDR=:8080 \
|
||
ZDDC_INSECURE_DIRECT=1 \
|
||
./zddc-server
|
||
```
|
||
|
||
When `ZDDC_TLS_CERT` / `ZDDC_TLS_KEY` are empty (or when using real certificates), zddc-server generates an ECDSA P-256
|
||
self-signed certificate in memory at startup. The certificate changes on every restart —
|
||
this is intentional and acceptable when an upstream reverse proxy terminates external TLS
|
||
and uses this server only for encrypted in-datacenter transport.
|
||
|
||
To use a real certificate (e.g. from Let's Encrypt or an internal CA):
|
||
|
||
```sh
|
||
ZDDC_ROOT=/srv/archive \
|
||
ZDDC_TLS_CERT=/etc/ssl/zddc/server.crt \
|
||
ZDDC_TLS_KEY=/etc/ssl/zddc/server.key \
|
||
./zddc-server
|
||
```
|
||
|
||
## Authentication
|
||
|
||
zddc-server does **not** perform authentication itself. It reads the user's email address
|
||
from a request header (default: `X-Auth-Request-Email`) that must be set by an upstream reverse proxy
|
||
(nginx, Caddy, Traefik, Azure Application Gateway, etc.) after authenticating the user.
|
||
|
||
If the header is absent, the user is treated as anonymous (empty email). A request is
|
||
allowed only if (a) **no `.zddc` file exists anywhere in the chain from `ZDDC_ROOT` to
|
||
the requested directory** (a fresh tree with zero `.zddc` files defaults to public
|
||
access — see warning at the top of the next section), or (b) some level in the chain
|
||
explicitly allows the caller's email. See "Access control: the `.zddc` cascade" below
|
||
for the full evaluation order.
|
||
|
||
## Access control: the `.zddc` cascade
|
||
|
||
> ⚠️ **zddc-server refuses to start without a root `.zddc`.** A `ZDDC_ROOT` containing
|
||
> no `.zddc` files anywhere would default to allow-all (anonymous callers included),
|
||
> so the server fails fast at startup with a clear error. Pass `--insecure` (or
|
||
> `ZDDC_INSECURE=1`) to acknowledge a deliberately-public deployment, otherwise
|
||
> drop a starter `<ZDDC_ROOT>/.zddc` per "Step 1" below.
|
||
|
||
`zddc-server` enforces access via cascading `.zddc` YAML files: drop one in any
|
||
directory, and its rules apply to that directory and everything beneath it that doesn't
|
||
override. The model is small enough to hold in your head, but the cascade has one
|
||
asymmetry that bites operators on first contact — read "When the cascade helps and
|
||
when it fights you" below before designing a layout.
|
||
|
||
```yaml
|
||
# Example .zddc — modern schema with verbs and roles
|
||
roles:
|
||
_company:
|
||
members: ["*@mycompany.com"]
|
||
_doc_controller:
|
||
members: [dc@mycompany.com]
|
||
acl:
|
||
permissions:
|
||
_company: r # everyone at mycompany.com gets read
|
||
_doc_controller: rwcda # doc controller gets full control
|
||
"contractor@partner.com": rw # specific external — read + overwrite
|
||
"intern@mycompany.com": "" # explicit deny (empty verb set)
|
||
|
||
# Legacy form below still works — equivalent to the new form, with
|
||
# allow → "rwcd" and deny → "" entries auto-merged into permissions.
|
||
# acl:
|
||
# allow: ["*@mycompany.com", "contractor@partner.com"]
|
||
# deny: ["intern@mycompany.com"]
|
||
```
|
||
|
||
### Permission verbs
|
||
|
||
Every access decision resolves to a verb set drawn from `r/w/c/d/a`:
|
||
|
||
| Verb | Allows |
|
||
|---|---|
|
||
| `r` | read file bytes; list directory |
|
||
| `w` | overwrite an existing file; rename existing file |
|
||
| `c` | create a new file or directory |
|
||
| `d` | delete a file |
|
||
| `a` | modify the ACL of this subtree (write `.zddc`) |
|
||
|
||
The verb set is written as concatenated lowercase letters in canonical order — `""` (none / explicit deny), `r`, `cr`, `rwcd`, `rwcda`. Common archetypes:
|
||
|
||
- **`r`** — read-only (typical company default).
|
||
- **`cr`** — append-only / drop-box (the doc controller in `Issued`/`Received`: can file new documents, cannot overwrite or delete).
|
||
- **`rwcd`** — full content control without the right to change the ACL (vendor inside their working subtree).
|
||
- **`rwcda`** — full control including the ability to grant access to others (subtree creator; project owner).
|
||
|
||
### Roles
|
||
|
||
`roles:` defines named principal groups, available at the level they're declared and all descendants:
|
||
|
||
```yaml
|
||
roles:
|
||
vendor_acme:
|
||
members: ["*@acme.com"]
|
||
_doc_controller:
|
||
members: [dc@mycompany.com, alice@mycompany.com]
|
||
```
|
||
|
||
Members are email patterns using the same glob syntax as legacy `acl.allow`. Underscore-prefixed names are conventional (`_doc_controller`, `_company`) but not magic. A role redefined closer to the leaf shadows the ancestor's definition. Permission-map keys without `@` are treated as role references first; if no role of that name exists in the visible cascade, they fall back to legacy email-pattern matching (so `*@example.com` and bare `*` continue to work).
|
||
|
||
### Step 1: starter `.zddc`
|
||
|
||
Every install should write a root `.zddc` before exposing the bind address. The
|
||
minimum is an `admins:` line so the admin debug page works (see "Admin Debug Page"
|
||
below) — adding `acl:` is optional at this step:
|
||
|
||
```yaml
|
||
# <ZDDC_ROOT>/.zddc — bare minimum
|
||
admins:
|
||
- you@yourcompany.com
|
||
```
|
||
|
||
With this single file in place, `HasAnyFile` becomes `true` for every directory in
|
||
the tree and the default switches from "allow-all-anonymous" to "deny-anything-not-
|
||
explicitly-allowed." From here you grant access by adding `acl:` rules at the levels
|
||
that need them. (See worked examples below.)
|
||
|
||
### How a request is evaluated
|
||
|
||
Each request carries an **action verb** (`r` for `GET`, `w` for `PUT` to an
|
||
existing file, `c` for `PUT` to a new file or `mkdir`, `d` for `DELETE`, `a`
|
||
for writes to `.zddc`). zddc-server reads every `.zddc` along the chain from
|
||
`ZDDC_ROOT` down to the request directory, then walks **leaf → root** looking
|
||
for a level whose `acl.permissions` map matches the user.
|
||
|
||
1. **Admin bypass.** If the email is in the root `admins:` list (root admin) or
|
||
any subtree-level `admins:` list on the chain (subtree admin), grant
|
||
`rwcda` and skip the cascade entirely.
|
||
2. **At each level**, find every `permissions:` entry whose principal matches
|
||
the user (direct email pattern, or role membership via `roles:` lookup).
|
||
- If any matching entry has the empty verb set `""` → **403 Forbidden**, stop.
|
||
- Otherwise, take the **union** of matching verb sets at this level. If the
|
||
union is non-empty, the level "wins" — the requested verb is allowed iff
|
||
it's present in the union. Stop walking.
|
||
3. **No match at this level** → walk up to the parent directory's `.zddc`.
|
||
4. **No level matched anywhere in the chain:**
|
||
- No `.zddc` anywhere (`HasAnyFile=false`) → **allow** (the empty-tree default).
|
||
- At least one `.zddc` existed (`HasAnyFile=true`) → **403 Forbidden** (default-deny).
|
||
|
||
Implementation: `GrantedVerbsAtLevel` (`zddc/internal/zddc/acl.go`) computes the
|
||
per-level grant; `EffectiveVerbs` / `AllowedAction` walk the chain; the chain
|
||
itself is built by `EffectivePolicy` (`zddc/internal/zddc/cascade.go`).
|
||
|
||
#### Cascade mode
|
||
|
||
The leaf-overrides-ancestor behavior above is the default — it's the historical
|
||
commercial-tenant model where a subtree owner can grant access without
|
||
root-admin involvement. Federal deployments needing absolute parent denies
|
||
(NIST AC-6) start the server with `--cascade-mode=strict` (or
|
||
`ZDDC_CASCADE_MODE=strict`):
|
||
|
||
- **`delegated`** (default) — leaf grant overrides ancestor explicit-deny.
|
||
- **`strict`** — two-pass evaluation. First pass walks **root → leaf** for any
|
||
matching explicit-deny; if found, denied (subject to root-admin bypass).
|
||
Second pass is the leaf→root grant walk above. An ancestor explicit-deny
|
||
cannot be overridden by any leaf grant.
|
||
|
||
The mode is logged at startup and surfaced on `/.profile/config`. Subtree
|
||
`.zddc` files cannot change the mode — it's a deployment-wide policy.
|
||
|
||
#### Special folders
|
||
|
||
Five folder names trigger built-in behaviors regardless of cascade mode (canonical list in `zddc/internal/zddc/special.go`):
|
||
|
||
- **`Incoming`, `Working`, `Staging`** — *auto-ownership*. When the file API processes `POST /<parent>/<new>/ X-ZDDC-Op: mkdir` and the parent is one of these three, the server writes a `.zddc` into the new folder containing `created_by: <email>` and `permissions: { <email>: rwcda }`. The grant uses the same direct email-pattern form an operator would write by hand; the creator can edit the `.zddc` later to add collaborators. `created_by` is an audit field — the cascade evaluator does not consult it.
|
||
- **`Issued`, `Received`** — *write-once / immutable archive*. When a request path crosses an `Issued` or `Received` segment, the server applies a **WORM split**: cascade grants inherited from ancestors above the WORM folder are masked to `r` only; grants at-or-below the WORM folder retain `r,c`. Anyone with `w`/`d`/`a` from inheritance loses those verbs once they enter the archive. To grant write-once (`cr`) to the doc controller, the operator places an explicit `.zddc` at the `Issued` or `Received` folder:
|
||
|
||
```yaml
|
||
# /<vendor>/Issued/.zddc
|
||
acl:
|
||
permissions:
|
||
_doc_controller: cr
|
||
```
|
||
|
||
The mask preserves the `c` from this same-level grant, so the doc controller can file new documents — but they still cannot overwrite, delete, or change the ACL. **Only admins (root or subtree) can mutate filed documents.** The mask is server-enforced and not configurable in v1; operators who want a non-WORM directory must avoid the names `Issued` and `Received`.
|
||
|
||
### Glob patterns
|
||
|
||
`*` matches any sequence of characters within one side of the `@` boundary:
|
||
|
||
| Pattern | Matches |
|
||
|---|---|
|
||
| `*@mycompany.com` | Any user at mycompany.com |
|
||
| `alice@*` | alice at any domain |
|
||
| `*` | Any non-empty email |
|
||
| `alice@example.com` | Exact match only |
|
||
|
||
The `*` does **not** cross the `@`. Implementation at `zddc/internal/zddc/acl.go:52`.
|
||
|
||
### When the cascade helps and when it fights you
|
||
|
||
The cascade is well-suited to one shape and clumsy at another. Internalize the
|
||
asymmetry before designing your layout:
|
||
|
||
- **Adding a new email at a leaf is easy** (third parties, occasional contractors).
|
||
The new email doesn't match any rule at higher levels, so the cascade just grants
|
||
the leaf-level allow and silently default-denies them everywhere else. No extra
|
||
`deny:` rules needed.
|
||
- **Excluding insiders from a leaf is harder** (commercially sensitive subset of
|
||
an otherwise company-wide tree). If a parent has `allow: ["*@company.com"]`, a
|
||
leaf-level `allow: [alice@company.com]` *adds* alice on top of everyone — it
|
||
does not subtract everyone-else. Subtracting requires either (a) the two-level
|
||
gate-and-reallow pattern in the worked example below, or (b) not having the
|
||
permissive parent rule in the first place.
|
||
|
||
This shape is intentional: the cascade is designed for **delegation** (subtree
|
||
owners can grant access without coordinating with central admin), not for
|
||
fine-grained subtractive policy. If your model is "everyone has access by default,
|
||
specific dirs are restricted," push the wildcards downward off the root rather than
|
||
fighting the cascade.
|
||
|
||
### Pick your layout
|
||
|
||
| Your shape | Recommended pattern |
|
||
|---|---|
|
||
| Solo / single-user archive | One `.zddc` at root with `admins: [you]`, no `acl:` block — root-only restriction |
|
||
| Single small team, full sharing | Root `.zddc` with `acl: {allow: ["*@team.com"]}`. No subdir overrides needed |
|
||
| Multi-tenant: each tenant in own subdir | Empty root `.zddc` (just `admins:`), per-tenant `<tenant>/.zddc` with `acl: {allow: [<tenant emails>]}`. Tenants don't see each other's listings |
|
||
| Mixed: half open within company, half locked-down to a subset, plus third-party folders | The worked example below — careful: do **not** put `*@company.com` at root |
|
||
|
||
### Worked example: paired open/closed projects + third-party archive
|
||
|
||
This is the deployment shape most operators end up with: technical projects are
|
||
shared company-wide; their commercial siblings are restricted to a subset; a
|
||
separate `/Archive/` tree holds per-vendor folders where each vendor sees only
|
||
their own subdir. The cascade handles all three with no `deny:` rules — but only
|
||
if you keep the `*@company.com` wildcard *off the root*.
|
||
|
||
```yaml
|
||
# <ZDDC_ROOT>/.zddc — admins only, no broad ACL
|
||
admins:
|
||
- admin@mycompany.com
|
||
```
|
||
|
||
```yaml
|
||
# <ZDDC_ROOT>/Acme-tech/.zddc — open employee project (technical)
|
||
acl:
|
||
allow: ["*@mycompany.com"]
|
||
```
|
||
|
||
```yaml
|
||
# <ZDDC_ROOT>/Acme-comm/.zddc — closed sibling (commercially sensitive)
|
||
acl:
|
||
allow:
|
||
- alice@mycompany.com
|
||
- bob@mycompany.com
|
||
```
|
||
|
||
```yaml
|
||
# <ZDDC_ROOT>/Archive/.zddc — employees can browse the vendor list
|
||
acl:
|
||
allow: ["*@mycompany.com"]
|
||
```
|
||
|
||
```yaml
|
||
# <ZDDC_ROOT>/Archive/Acme/.zddc — vendor's only window
|
||
acl:
|
||
allow:
|
||
- acme-rep@acme.com
|
||
```
|
||
|
||
Trace for an insider (`alice@mycompany.com`) and a vendor (`acme-rep@acme.com`)
|
||
hitting representative paths:
|
||
|
||
| Path | alice@mycompany.com | acme-rep@acme.com |
|
||
|---|---|---|
|
||
| `/Acme-tech/` | ✅ matches `*@mycompany.com` at this level | ❌ no match anywhere; `HasAnyFile=true` → 403 |
|
||
| `/Acme-comm/` | ✅ matches `alice` at this level | ❌ default-deny |
|
||
| `/<other-closed-project>/` | ❌ no match anywhere → 403 | ❌ default-deny |
|
||
| `/Archive/` | ✅ matches `*@mycompany.com` | ❌ no match; default-deny |
|
||
| `/Archive/Acme/` | ✅ falls up to `/Archive/.zddc`, matches | ✅ matches `acme-rep@acme.com` at this level |
|
||
| `/Archive/Acme/Incoming/` | ✅ inherits from `/Archive/.zddc` | ✅ inherits from `/Archive/Acme/.zddc` |
|
||
| `/Archive/<other-vendor>/` | ✅ inherits from `/Archive/` | ❌ no match; default-deny |
|
||
| `/` (project picker) | Lands; sees the projects she has access to | Lands; project picker filtering hides everything she can't reach |
|
||
|
||
The vendor reaches `/Archive/Acme/...` only via direct URL (a bookmark or a
|
||
transmittal email link). Trying to navigate up to `/Archive/` returns 403; the
|
||
archive tool itself treats `/Archive/Acme/` as the root of *its* world (see
|
||
"Tool-rooted view" in `ARCHITECTURE.md` § Server security model), so there is
|
||
no breadcrumb leading anywhere they can't see.
|
||
|
||
### Patterns that look secure but aren't
|
||
|
||
These are the traps. Each is plausible at first glance and doesn't behave as
|
||
naive intuition suggests.
|
||
|
||
1. **Same-level `allow + deny "*@company.com"` does NOT lock the level down to the
|
||
allow's targets.** Deny is checked before allow within a single `.zddc`, so the
|
||
allowed user's email matches the deny first and is blocked. To exclude insiders
|
||
from a leaf, use the two-level gate-and-reallow (parent denies, deeper level
|
||
re-allows) — *or* avoid putting `*@company.com` at any ancestor.
|
||
|
||
```yaml
|
||
# /Closed/.zddc — DOES NOT WORK as intended
|
||
acl:
|
||
allow: [alice@company.com]
|
||
deny: ["*@company.com"] # blocks alice too — deny is checked first at same level
|
||
```
|
||
|
||
2. **A leaf-level `allow: [subset]` does NOT restrict when a parent has
|
||
`allow: ["*@domain"]`.** Non-subset users hit the leaf with no match, walk up,
|
||
match the parent wildcard, and are allowed. Adding a leaf allow is *additive*,
|
||
never *subtractive*. (See the asymmetry section above.)
|
||
|
||
3. **`admins:` in any `.zddc` other than the root is silently ignored.** The check
|
||
at `zddc/internal/zddc/file.go:17-20` (and `IsAdmin`) only reads root. This is
|
||
the only upward-escalation gate; subtree write access never grants admin.
|
||
|
||
4. **An `apps:` URL override is a full UI mount, not just a tool version pin.**
|
||
Any `.zddc` writer in a subtree can pin `archive: https://attacker.example/...`
|
||
and serve arbitrary HTML to every viewer below that level. Subtree write
|
||
authority on `.zddc` should be treated as full UI-mounting authority. The
|
||
`_app/` cache is fetch-once-and-keep — operators clear it by deleting
|
||
`<ZDDC_ROOT>/_app/<scheme>/<host>[:<port>]/<path>`. (See "Apps: virtual tool HTMLs" below for
|
||
the resolver order; SHA-256 pinning is on the federal-readiness list, not
|
||
currently implemented.)
|
||
|
||
5. **Relying on `/Archive/` being unbrowsable to "hide" sibling vendor folders'
|
||
existence.** Sibling-vendor names are hidden because directories the caller
|
||
can't access are omitted from listings (see "Directory visibility" below) —
|
||
that's the actual mechanism. Don't rely on the parent dir being denied; rely
|
||
on the listing filter.
|
||
|
||
### Trust model and invariants
|
||
|
||
These are the invariants security reviewers should expect to find, stated plainly
|
||
and tied to the code that enforces them:
|
||
|
||
- **Auth boundary.** zddc-server does not authenticate. The user's email is read
|
||
from the configured request header (default `X-Auth-Request-Email`) set by an
|
||
upstream reverse proxy. If the proxy is misconfigured or the bind address is
|
||
reachable without traversing the proxy, every request is anonymous — and any
|
||
caller able to set the header can claim any email. Network isolation between
|
||
the proxy and zddc-server is required (see "Trust boundary" below).
|
||
- **Subtree authority.** Whoever can write a `.zddc` controls that subtree's
|
||
ACL — including overriding a parent *deny* with a leaf *allow* (test:
|
||
`zddc/internal/zddc/acl_test.go:212` "leaf allows user that parent denies →
|
||
leaf wins"). This is intentional delegation, not a bug. If you grant write
|
||
access to `/Project-A/.zddc`, you've granted full ACL authority over the
|
||
Project-A subtree.
|
||
- **Root-only escalation gate.** `admins:` is honored *only* at
|
||
`<ZDDC_ROOT>/.zddc`. Subtree `admins:` entries are silently ignored
|
||
(`zddc/internal/zddc/file.go:17-20`). This is the *only* upward escalation
|
||
block; without it, anyone with subtree write authority could promote
|
||
themselves to admin.
|
||
|
||
### Trust boundary
|
||
|
||
What zddc-server enforces stops at the network boundary. The deployment must
|
||
guarantee these for the model above to hold:
|
||
|
||
1. **The bind address must be reachable only via the authenticating proxy.** The
|
||
email-header trust assumes the proxy is the only path to the server. In
|
||
Kubernetes: a `NetworkPolicy` restricting ingress to the proxy pod. On a
|
||
single host: bind to loopback (`ZDDC_ADDR=127.0.0.1:8080`) and run the proxy
|
||
on the same host. **Without this**, anyone reaching the bind address can
|
||
forge any email by setting `X-Auth-Request-Email` themselves. zddc-server
|
||
refuses to start with `ZDDC_TLS_CERT=none` on a non-loopback bind unless
|
||
`ZDDC_INSECURE_DIRECT=1` is set as an explicit acknowledgement that an
|
||
authenticating proxy is enforcing this.
|
||
2. **Anonymous information disclosure on `/` is by design.** The public landing
|
||
page returns a project picker filtered by ACL — anonymous callers see only
|
||
projects with no `.zddc` rules along their chain (in a properly-configured
|
||
deployment, none). The *existence* of the server and the names of any
|
||
ACL-free projects are disclosed without authentication. For deployments
|
||
where this disclosure is unacceptable, gate `/` itself behind the proxy's
|
||
auth-required path; zddc-server's public-landing logic does not need to be
|
||
disabled, it simply never receives an anonymous request. `/.profile` is also
|
||
reachable anonymously by design — same caveat applies.
|
||
3. **Audit log integrity is filesystem-level, not application-level.** The
|
||
tee'd JSON access log lives at `<ZDDC_ROOT>/.zddc.d/logs/access-<host>.log`
|
||
on the served volume. While `.`-prefixed paths return 404 over HTTP (so the
|
||
log is not readable through zddc-server), anything with **filesystem write
|
||
authority on the served volume** (a sidecar, a backup-restore process, an
|
||
admin shell) can modify log entries after the fact. For tamper-evident
|
||
logging, ship the JSON-line file to an external append-only sink (syslog,
|
||
SIEM) via a sidecar; do not treat the local rotation as the system of
|
||
record.
|
||
4. **`apps:` URL fetches have no integrity check.** Fetched once on first
|
||
miss, cached at `<ZDDC_ROOT>/_app/<scheme>/<host>[:<port>]/<path>` forever — no SHA-256 pin,
|
||
no signature. Use only URLs you control, treat the apps cache as a trust
|
||
boundary, and audit who has `.zddc` write authority where.
|
||
|
||
### Debugging permissions
|
||
|
||
When a user reports "I can't see /Project-X/" and you need to figure out why,
|
||
the fastest path is the built-in cascade tracer:
|
||
|
||
```
|
||
GET /.profile/effective-policy?path=/Project-X/sub/&email=alice@mycompany.com
|
||
```
|
||
|
||
(Admin-only — 404 to non-admins. Same gate as `/.profile/whoami`,
|
||
`/.profile/config`, `/.profile/logs`.)
|
||
|
||
Returns JSON with the resolved policy chain (every level along the
|
||
walk from `ZDDC_ROOT` to the requested directory), the decision the
|
||
active decider produces, the per-level email-match breakdown
|
||
(`decision_at_level: "allow" | "deny" | "no_match"`), and which
|
||
decider is wired in (`*policy.InternalDecider`,
|
||
`*policy.cachingDecider`, etc.). With `ZDDC_OPA_URL` pointing at an
|
||
external OPA, the decision goes through that OPA — so this endpoint
|
||
also doubles as a smoke test for the OPA wiring.
|
||
|
||
Manual procedure (if the endpoint isn't reachable for some reason):
|
||
|
||
1. **Confirm the resolved email** — hit `/.profile/whoami` as the
|
||
user. Shows every header on the request and the `email` field
|
||
zddc-server resolved.
|
||
2. **List the chain.** From `<ZDDC_ROOT>` down to the requested
|
||
directory, inspect each `.zddc` (most directories have none). For
|
||
`/Project-X/sub/sub/`, that's `/.zddc`, `/Project-X/.zddc`,
|
||
`/Project-X/sub/.zddc`, `/Project-X/sub/sub/.zddc` — read whatever
|
||
exists.
|
||
3. **Walk bottom-up.** At each level, mentally run `AllowedAtLevel`:
|
||
deny patterns first (any match → blocked), then allow (any match
|
||
→ allowed). First explicit match in the bottom-up walk is the
|
||
answer. Default-deny if `HasAnyFile=true` and nothing matches.
|
||
|
||
### Directory visibility
|
||
|
||
Directories for which the user lacks access are **omitted** from JSON listings entirely —
|
||
they are neither listed nor queryable. A direct request to a denied path returns `403`.
|
||
|
||
This is the mechanism that hides sibling subtrees from a caller. Vendor `acme-rep`
|
||
sees an empty-looking `/Archive/` (in fact returns 403 since they're not allowed
|
||
there at all in the worked example), and no other vendor's name leaks via listing.
|
||
|
||
### Reserved hidden segments
|
||
|
||
Two prefixes are filtered from listings under `ZDDC_ROOT`:
|
||
|
||
- **`.`-prefixed** (e.g. `/.devshell/`, `/Project-A/.internal/notes.md`) — excluded
|
||
from listings **and** 404 on direct HTTP access. The recognized virtual prefixes
|
||
(`.archive`, `.admin`) are explicitly permitted through. This lets operators store
|
||
side-state (caches, dev-shell home dirs, snapshot staging) on the same volume
|
||
that's served, without exposing it.
|
||
- **`_`-prefixed** (e.g. `/_template/`) — excluded from listings only. Direct URL
|
||
access still works. Use this for operator-managed scaffolding the user shouldn't
|
||
browse to but might link to (e.g. a `_template/` directory of stub-HTML examples
|
||
to copy into project subdirs).
|
||
|
||
### How to verify in 5 minutes
|
||
|
||
This recipe stands up the worked-example layout in a tmpdir, hits each
|
||
(email, path) cell with `curl`, and asserts the documented behavior. Run it on
|
||
your own deployment to confirm the cascade is doing what you think:
|
||
|
||
```sh
|
||
ROOT=$(mktemp -d)
|
||
mkdir -p "$ROOT/Acme-tech" "$ROOT/Acme-comm" "$ROOT/Archive/Acme"
|
||
|
||
cat > "$ROOT/.zddc" <<'YAML'
|
||
admins: [admin@mycompany.com]
|
||
YAML
|
||
cat > "$ROOT/Acme-tech/.zddc" <<'YAML'
|
||
acl: {allow: ["*@mycompany.com"]}
|
||
YAML
|
||
cat > "$ROOT/Acme-comm/.zddc" <<'YAML'
|
||
acl: {allow: [alice@mycompany.com]}
|
||
YAML
|
||
cat > "$ROOT/Archive/.zddc" <<'YAML'
|
||
acl: {allow: ["*@mycompany.com"]}
|
||
YAML
|
||
cat > "$ROOT/Archive/Acme/.zddc" <<'YAML'
|
||
acl: {allow: [acme-rep@acme.com]}
|
||
YAML
|
||
|
||
# Plain HTTP on loopback so curl doesn't need TLS
|
||
ZDDC_ROOT="$ROOT" ZDDC_TLS_CERT=none ZDDC_ADDR=127.0.0.1:8090 \
|
||
./zddc-server &
|
||
SERVER_PID=$!
|
||
sleep 1
|
||
|
||
probe() { # email path expected_status
|
||
got=$(curl -s -o /dev/null -w '%{http_code}' \
|
||
-H "X-Auth-Request-Email: $1" \
|
||
"http://127.0.0.1:8090$2")
|
||
printf '%-40s %-30s expected=%s got=%s %s\n' \
|
||
"$1" "$2" "$3" "$got" \
|
||
"$([ "$got" = "$3" ] && echo OK || echo FAIL)"
|
||
}
|
||
|
||
# Insider — alice should see all the technical + her closed project + Archive tree
|
||
probe alice@mycompany.com /Acme-tech/ 200
|
||
probe alice@mycompany.com /Acme-comm/ 200
|
||
probe alice@mycompany.com /Archive/ 200
|
||
probe alice@mycompany.com /Archive/Acme/ 200
|
||
|
||
# Insider not on the closed project — bob should hit the technical and Archive,
|
||
# NOT the closed sibling
|
||
probe bob@mycompany.com /Acme-tech/ 200
|
||
probe bob@mycompany.com /Acme-comm/ 403
|
||
probe bob@mycompany.com /Archive/ 200
|
||
probe bob@mycompany.com /Archive/Acme/ 200
|
||
|
||
# Vendor — acme-rep should ONLY see /Archive/Acme/, blocked everywhere else
|
||
probe acme-rep@acme.com /Acme-tech/ 403
|
||
probe acme-rep@acme.com /Acme-comm/ 403
|
||
probe acme-rep@acme.com /Archive/ 403
|
||
probe acme-rep@acme.com /Archive/Acme/ 200
|
||
|
||
# Anonymous — root .zddc exists, so HasAnyFile=true → default-deny everywhere
|
||
# (the root / itself is the public-landing exception; subdirs are gated)
|
||
probe '' /Acme-tech/ 403
|
||
probe '' /Archive/Acme/ 403
|
||
|
||
# Anti-pattern: same-level allow + deny *@company.com does NOT lock alice in
|
||
mkdir -p "$ROOT/Trap"
|
||
cat > "$ROOT/Trap/.zddc" <<'YAML'
|
||
acl:
|
||
allow: [alice@mycompany.com]
|
||
deny: ["*@mycompany.com"] # deny is checked first → blocks alice too
|
||
YAML
|
||
probe alice@mycompany.com /Trap/ 403 # the trap docs warn about
|
||
|
||
kill $SERVER_PID
|
||
```
|
||
|
||
Every line should print `OK`. If any prints `FAIL`, the cascade isn't behaving
|
||
as documented — file an issue with the failing line. **After every `.zddc`
|
||
change in production**, retest at minimum as the editing user (to confirm you
|
||
haven't locked yourself out).
|
||
|
||
### Federal-readiness gap analysis
|
||
|
||
The current model is well-suited for commercial-tenant ACL with delegated
|
||
authentication. To clear federal hurdles (FedRAMP Moderate, NIST 800-53 Rev. 5
|
||
baseline, FIPS 140-3, DoD STIG), these gaps would need to be closed. None are
|
||
implemented today — this list is informational so security reviewers don't
|
||
have to redo the gap analysis from scratch.
|
||
|
||
- **FIPS 140-3 cryptography** (NIST SC-13) — current build uses Go stdlib
|
||
crypto. *Required:* parallel `zddc-server-fips` build target using the
|
||
`microsoft/go` toolchain on a RHEL/UBI base with the validated OpenSSL
|
||
FIPS module. See `federal.html` for the deployment-shape narrative and
|
||
the §"FIPS-validated cryptography" subsection below for the engineering
|
||
detail.
|
||
- **TLS hardening** (NIST SC-8(1)) — *partially complete.* Server now
|
||
sets `MinVersion: tls.VersionTLS12`, the NIST SP 800-52 Rev. 2
|
||
AEAD-only cipher allowlist (ECDHE+AES-GCM and ECDHE+ChaCha20Poly1305
|
||
variants), curve preferences (X25519, P-256, P-384), and emits HSTS
|
||
(`max-age=31536000; includeSubDomains`) when zddc-server itself
|
||
terminates TLS. *Still required for full DoD STIG conformance:*
|
||
OCSP stapling, certificate-transparency-log inclusion, and an
|
||
audit-grade documentation pack mapping the cipher list to FIPS
|
||
140-3 validated implementations.
|
||
- **Authenticated proxy↔server channel** (NIST IA-3) — current trust is
|
||
network-level isolation only. *Required:* mTLS or a signed forwarding
|
||
token (JWT). See §"Authenticated proxy↔server channel" subsection
|
||
below for the design trade-off and recommended path.
|
||
- **Multi-factor authentication** (NIST IA-2(1)) — delegated to upstream
|
||
proxy. Required: documented reference deployment with PIV/CAC via
|
||
oauth2-proxy or equivalent.
|
||
- ~~**Role-based access control** (NIST AC-3(7))~~ — *closed.* Roles are
|
||
first-class entities defined under `roles:` in any `.zddc`, available
|
||
at the level they're declared and all descendants. `acl.permissions`
|
||
grants verb sets (`r`/`w`/`c`/`d`/`a`) per role or per email pattern.
|
||
Identity-source-driven role assignment plumbs through unchanged
|
||
(the upstream proxy still asserts the email; role membership is
|
||
evaluated server-side against the cascade).
|
||
- ~~**Least-privilege bounding** (NIST AC-6)~~ — *closed.* Operators
|
||
set `--cascade-mode=strict` (or `ZDDC_CASCADE_MODE=strict`) to
|
||
switch the in-process Go evaluator into the federal posture: any
|
||
ancestor explicit-deny is absolute and cannot be overridden by a
|
||
leaf grant. The mode is logged at startup and surfaced on
|
||
`/.profile/config`. The legacy commercial behavior is preserved as
|
||
the default `delegated` mode. External OPA (`ZDDC_OPA_URL`) remains
|
||
available for org-specific Rego on top of this.
|
||
- **Account lifecycle** (NIST AC-2) — emails as identifiers must tie to
|
||
authoritative sources (PIV cert subject, IdP-managed identity). Required:
|
||
documented integration with at least one IdP supporting federal identity
|
||
attestation.
|
||
- **Audit log integrity & retention** (NIST AU-9, AU-11) — current 90-day
|
||
local rotation is a starting point. Required: tamper-evident logs (signed
|
||
log chain or external append-only sink) with 1-year minimum online and
|
||
3-year archive.
|
||
- **Continuous monitoring hooks** (NIST CA-7) — automated alerting on
|
||
`.zddc` changes, admin endpoint use, repeated 403s from one identity.
|
||
Required: structured event emission to syslog/SIEM beyond the local file.
|
||
- **Configuration baseline export** (NIST CM-3) — change-control review
|
||
needs a way to enumerate every `.zddc` file's resolved effective ACL
|
||
for diff against a baseline. *Required:* `zddc-server policy export`
|
||
subcommand. See §"Policy export for change control" below.
|
||
- **Supply-chain integrity** (NIST SI-7) — vendored libs (jszip,
|
||
docx-preview, xlsx) need SBOM, CVE tracking, automated update pipeline.
|
||
`apps:` URL fetches need code signing (operator trusts a published
|
||
public key once; no per-artifact hash management). See §"Code-signed
|
||
apps: URL fetches" below.
|
||
- **Data-at-rest encryption** (NIST SC-28) — delegated to the deployment
|
||
platform. Required: documented baseline (cloud KMS, LUKS, dm-crypt) with
|
||
key-rotation procedures.
|
||
- **Vulnerability disclosure process** (NIST SI-5) — repo lacks
|
||
`SECURITY.md`. Required: documented disclosure procedure, embargoed-fix
|
||
workflow, CVE-assignment policy.
|
||
|
||
A full SSP / control-by-control mapping consumes this list as input; it is
|
||
not a substitute for one.
|
||
|
||
The four bullets most likely to need engineering depth — FIPS, the
|
||
authenticated proxy channel, policy export, and signed `apps:` URL
|
||
fetches — have their own subsections below capturing the design
|
||
considerations and effort estimates so a future implementor doesn't
|
||
restart from zero.
|
||
|
||
#### FIPS-validated cryptography (NIST SC-13)
|
||
|
||
**Why the current build doesn't qualify.** Go's stdlib crypto is
|
||
correct and well-reviewed but has not been submitted to NIST's CMVP
|
||
(Cryptographic Module Validation Program). Federal evaluators want a
|
||
specific module identifier ("the cryptography is performed by validated
|
||
module #4282" or whatever certificate number applies); Go stdlib has
|
||
none.
|
||
|
||
**Why "swap in microsoft/go" isn't free.** The microsoft/go fork
|
||
redirects every `crypto/*` call into the host's OpenSSL via cgo. The
|
||
moment we adopt it as the default toolchain:
|
||
|
||
- The "single static binary, CGO-free" property in the Features list
|
||
(line 19) goes away. The binary dynamically links `libssl`/`libcrypto`
|
||
and needs OpenSSL installed at the target ABI.
|
||
- Cross-compile becomes harder: cgo cross-compilation needs the target
|
||
platform's OpenSSL during the build. Today `./build` cross-compiles
|
||
all four release platforms from one container with no extra tooling.
|
||
- macOS/Windows builds either bundle a non-validated OpenSSL (defeats
|
||
the FIPS purpose) or fall back to Go stdlib for those platforms
|
||
(validation not transitive). The four-platform release matrix
|
||
collapses to one (linux-amd64).
|
||
- Binary size grows ~5-10 MB; deployment gains a runtime dependency.
|
||
- Commercial customers gain literally nothing — Go stdlib crypto is
|
||
cryptographically correct; FIPS is a procurement-checkbox property,
|
||
not a strength property.
|
||
|
||
**Recommended path: parallel `zddc-server-fips` build target.** Don't
|
||
swap the default toolchain. Instead, ship a second binary built with
|
||
microsoft/go on a RHEL/UBI base with the validated OpenSSL FIPS package
|
||
installed. linux-amd64 only. Distributed at
|
||
`releases/zddc-server-fips_<channel>_linux-amd64`. Federal customers
|
||
download the fips variant; everyone else continues to download the lean
|
||
pure-Go binary.
|
||
|
||
The validation belongs to OpenSSL, not to microsoft/go — the cert
|
||
number is for the OpenSSL FIPS provider. The microsoft/go toolchain is
|
||
the bridge that lets Go-built apps consume that validated module. So
|
||
the deployment commitment is "RHEL/UBI image, kernel-side
|
||
`update-crypto-policies --set FIPS`, validated OpenSSL on the host." A
|
||
helm chart `helm/zddc-server-fips/` would bundle the matching base
|
||
image and `securityContext`.
|
||
|
||
**Effort estimate:**
|
||
- `Containerfile.fips` and `helm/zddc-server-fips/` chart: ~50 lines
|
||
each.
|
||
- `./build` updates to add the new target: ~30 lines (one
|
||
`podman build -f Containerfile.fips` invocation, one
|
||
`promote_zddc_server_fips` helper, integration into the existing
|
||
release flow).
|
||
- Verification: a smoke test that runs the fips binary, hits a TLS
|
||
endpoint, and confirms the negotiated cipher came from the FIPS
|
||
provider (visible via `SSL_CIPHER` exposed from OpenSSL).
|
||
- Documentation: ~50 lines updating `federal.html` and this README to
|
||
describe the supported deployment shape concretely.
|
||
|
||
Total: ~250 lines net once the customer engages.
|
||
|
||
**What stays unchanged.** Application code is identical between tracks.
|
||
The TLS hardening already shipped (cipher allowlist, curve preferences,
|
||
HSTS) reuses verbatim — every cipher we ship is on the OpenSSL FIPS
|
||
allowlist, so the same `tls.Config` works in both tracks. The
|
||
difference is purely the toolchain and the host OS configuration.
|
||
|
||
#### Authenticated proxy↔server channel (NIST IA-3)
|
||
|
||
**The current trust assumption.** zddc-server reads
|
||
`X-Auth-Request-Email` from any client that can connect to the bind
|
||
address. Network isolation (loopback bind + same-host proxy, or k8s
|
||
NetworkPolicy gating ingress) is the only thing preventing email
|
||
forgery. NIST IA-3 (Device Identification and Authentication) requires
|
||
**cryptographic** binding between system components for higher assurance
|
||
levels.
|
||
|
||
**Two design options.**
|
||
|
||
**Option A — mTLS.**
|
||
|
||
The proxy presents a client certificate during the TLS handshake to
|
||
zddc-server. zddc-server verifies the cert against a trusted CA and
|
||
extracts the email from the cert's subject alternative names (or
|
||
matches the cert against an allowlist).
|
||
|
||
- *Pros:* Works at the connection layer; transparent to handler code
|
||
(the email comes from `r.TLS.PeerCertificates[0]` instead of a
|
||
header). Reuses existing TLS infrastructure. Zero per-request
|
||
overhead beyond the standard TLS handshake.
|
||
- *Cons:* Cert distribution. Every proxy needs its own client cert;
|
||
every cert eventually expires. Operator runs a small private CA
|
||
(cert-manager + k8s, smallstep, Vault PKI) or rotates manually.
|
||
Operationally heavier than the alternative.
|
||
|
||
**Option B — JWT (signed forwarding token).**
|
||
|
||
The proxy signs a JWT for each forwarded request using its private key.
|
||
zddc-server verifies the signature with the proxy's public key. The JWT
|
||
carries the email, an issuer claim, an audience claim, and a short
|
||
expiry (replay protection). Sent in `Authorization: Bearer <jwt>` or a
|
||
custom `X-Auth-Forwarding-Token` header.
|
||
|
||
- *Pros:* Lightweight. No PKI — just one public key to distribute to
|
||
zddc-server. Easy to debug (JWT is base64 JSON; `jwt.io` exists).
|
||
Easy to rotate (run two trusted public keys during rotation).
|
||
Integrates with existing oauth2-proxy / nginx pipelines that have
|
||
"sign and forward" modes built in.
|
||
- *Cons:* Adds a JWT library dep (~50KB). Requires a freshness model
|
||
(short expiry, e.g. 30s — so an exfiltrated token is useless after
|
||
half a minute). Requires key distribution + rotation, but at much
|
||
smaller scale than mTLS.
|
||
|
||
**Recommended path: JWT first; mTLS as a sidecar option.**
|
||
|
||
JWT is the better starting point. Lighter operationally, doesn't
|
||
compete with the existing TLS termination story (an stunnel-FIPS or
|
||
nginx-FIPS sidecar can still do mTLS at the network layer if needed),
|
||
and integrates with future role-based access (item AC-3(7) above) —
|
||
`roles` claims in the JWT can flow into the ABAC roles work.
|
||
|
||
**Implementation sketch.**
|
||
|
||
- New env var `ZDDC_JWT_PUBKEY=/etc/zddc/proxy-pub.pem` or
|
||
`ZDDC_JWT_PUBKEY_INLINE=...PEM bytes...`.
|
||
- New middleware (peer to `ACLMiddleware` in
|
||
`zddc/internal/handler/middleware.go`) that, when configured, reads
|
||
`Authorization: Bearer <jwt>` from the request, verifies the
|
||
signature, validates `aud == zddc-server`, `exp` in the future,
|
||
`nbf` in the past, and extracts `email` (and `roles`) from the
|
||
verified claims.
|
||
- Refusal mode: when `ZDDC_JWT_PUBKEY` is set and a request lacks a
|
||
valid JWT, return 401. No silent fallback — the operator opted in
|
||
to cryptographic binding.
|
||
- Library choice: `github.com/golang-jwt/jwt/v5` (pure Go, well
|
||
maintained) or `github.com/lestrrat-go/jwx` (more features, larger).
|
||
Lean toward the simpler `golang-jwt/jwt`.
|
||
|
||
**Effort estimate:** ~150 lines of Go (middleware + config wiring +
|
||
key parsing) + the JWT library dependency + ~30 lines of deployment
|
||
docs showing oauth2-proxy and nginx sign-and-forward configurations.
|
||
|
||
#### Policy export for change control (NIST CM-3)
|
||
|
||
**Why this is needed.** Federal change-control review wants a baseline
|
||
file that can be diffed against current state. For zddc-server, "the
|
||
configuration" is the union of every `.zddc` file in the served tree.
|
||
Today an operator wanting "what's the effective access at /Project-A?"
|
||
has to walk the cascade by hand. The `/.profile/effective-policy`
|
||
endpoint shortcuts a single `(path, email)` query, but it doesn't
|
||
enumerate the whole policy.
|
||
|
||
**What the export produces.** For every directory containing a `.zddc`,
|
||
the command emits:
|
||
|
||
- The directory path
|
||
- The raw `.zddc` contents
|
||
- The resolved effective ACL (the chain back to root, flattened into
|
||
this directory's effective allow/deny lists)
|
||
- Equivalence-class summary: `(email-class, decision)` pairs covering
|
||
groups like `*@mycompany.com → allow`, `*@external.com → deny`,
|
||
specific exceptions
|
||
|
||
Output formats:
|
||
|
||
- **JSON** — machine-readable for `diff` against a baseline file in a
|
||
Git repo or a change-control tool
|
||
- **Markdown** — human-readable summary suitable for ATO documentation
|
||
- **CSV** — pivot-ready for spreadsheet review
|
||
|
||
**Use cases:**
|
||
|
||
- *Baseline + diff workflow.* Operator commits the export to a Git repo.
|
||
Every `.zddc` change auto-runs the export in CI; the diff is the
|
||
change-control artifact. Reviewers see "alice was added to
|
||
/Project-X/.zddc, which means alice can now access /Project-X/,
|
||
/Project-X/sub/, … because the cascade propagates."
|
||
- *Drift detection.* Export at deploy time vs export every day; any
|
||
unexpected change triggers an alert.
|
||
- *ATO documentation.* Plug the export into the SSP's "Configuration
|
||
Inventory" section.
|
||
|
||
**Implementation sketch.** Mostly mechanical reuse of existing pieces:
|
||
|
||
- `zddc.ScanZddcFiles(root)` already walks the tree (used by the
|
||
profile page's admin-subtrees listing).
|
||
- `zddc.EffectivePolicy(root, dir)` already builds the chain (used by
|
||
every ACL check).
|
||
- `zddc.MatchesPattern` already covers the email matching.
|
||
|
||
What's missing is the CLI subcommand wiring (today the binary takes
|
||
flags only; needs subcommand parsing for `policy export`), the
|
||
equivalence-class computation (group emails by which patterns they
|
||
match — small graph problem), and the format renderers.
|
||
|
||
**Effort estimate:** ~250 lines of Go (CLI subcommand + equivalence-
|
||
class computation + JSON/Markdown/CSV renderers) + ~100 lines of tests.
|
||
|
||
#### Code-signed `apps:` URL fetches (NIST SI-7)
|
||
|
||
**The supply-chain risk today.** The `apps:` mechanism fetches a URL
|
||
once on first request, caches the bytes forever, and serves them to
|
||
every viewer below that level. There's no integrity check. If the
|
||
fetched URL is ever compromised — DNS hijack, CDN account takeover,
|
||
malicious upstream commit, MITM during the one fetch window — every
|
||
customer caches the bad bytes. The blast radius is "every user who
|
||
visits an archive page in a subtree where this `.zddc` applies."
|
||
|
||
**Why code signing instead of SHA-256 pinning.** SHA-256 pinning would
|
||
require operators to track-and-update a hash in `.zddc` every time an
|
||
artifact changes. Wrong workflow for this product. Code signing
|
||
sidesteps the operator entirely:
|
||
|
||
- Release pipeline signs each artifact once at publish time with a
|
||
long-lived private key.
|
||
- Operator trusts the published public key once and never deals with
|
||
hashes.
|
||
- zddc-server fetches the URL, downloads the detached signature
|
||
(e.g. `<artifact>.sig`), verifies against the configured public key,
|
||
caches if valid.
|
||
|
||
**Implementation has three parts** that interlock:
|
||
|
||
1. **Signing in the build pipeline.** `./build alpha|beta|release` runs
|
||
`sign_release_artifacts` (in `./build`) after promote: walks
|
||
`dist/release-output/` and produces a detached Ed25519 signature
|
||
(`<artifact>.sig`) alongside every real file. Private key path comes
|
||
from `ZDDC_SIGNING_KEY`; absent or unreadable → release fails.
|
||
Symlinks (channel mirrors, partial-version pins) skip — the .sig
|
||
at the symlink target is what counts.
|
||
|
||
2. **Public key on the website.** `pubkey.pem` is a real file in
|
||
`~/src/zddc-website/`, deployed to `zddc.varasys.io/pubkey.pem`.
|
||
The releases-page index includes a "Verify your downloads"
|
||
section with a download link, the SHA-256 fingerprint shown in
|
||
plain text, and a `curl + openssl pkeyutl -verify` example.
|
||
|
||
3. **Verifier in zddc-server (`apps/fetch.go`).** When fetching a
|
||
URL-pinned `apps:` artifact, also fetch `<url>.sig`, then call
|
||
`VerifyEd25519` against `Fetcher.VerifyKey`. The key is loaded
|
||
at startup with this resolution order:
|
||
1. `--apps-pubkey` / `ZDDC_APPS_PUBKEY` (path to PEM file)
|
||
2. `apps_pubkey:` inline PEM in the root `.zddc` file (root-only,
|
||
same trust-anchor treatment as `admins:`)
|
||
3. nothing → URL fetches refused
|
||
Failure cases at fetch time — sig 404, transport error, wrong key,
|
||
tampered body — all reject; the body is dropped and the apps
|
||
resolver falls back to the embedded copy. No baked-in default
|
||
public key; same posture as TLS certificates.
|
||
|
||
**Trust model.** The operator decides which signing infrastructure to
|
||
trust by configuring `--apps-pubkey`. The website publishes the
|
||
canonical-channel public key; operators who use `apps: archive: stable`
|
||
download it once, save the file on their server, and configure the
|
||
path. Operators running their own signing point at their own pubkey
|
||
instead. zddc-server has no opinion.
|
||
|
||
**Future direction.** A `signed_by:` field per-`apps:` entry would let
|
||
a single deployment trust multiple signing keys (one for canonical
|
||
channels, one for an in-house mirror). Sigstore integration
|
||
(transparency-log-backed signing via `github.com/sigstore/sigstore`)
|
||
is the federally-acceptable evolution. Both are additive on top of
|
||
the current single-key-per-server model.
|
||
|
||
**What's currently in place.** All three parts. The scaffolding
|
||
matches the design above one-to-one; future enhancements are
|
||
extensions, not refactors.
|
||
|
||
### External policy decider (OPA-compatible)
|
||
|
||
For deployments that need policy decisions made by an external,
|
||
independently-audited engine — typically federal customers using
|
||
[Open Policy Agent](https://www.openpolicyagent.org/) — zddc-server can
|
||
delegate every access decision to an HTTP/Unix-socket endpoint that
|
||
speaks OPA's `/v1/data/...` JSON wire protocol.
|
||
|
||
Set one of:
|
||
|
||
```sh
|
||
ZDDC_OPA_URL=internal # built-in Go evaluator (default)
|
||
ZDDC_OPA_URL=http://127.0.0.1:8181 # OPA via HTTP
|
||
ZDDC_OPA_URL=https://opa.internal:8181 # OPA via HTTPS
|
||
ZDDC_OPA_URL=unix:///run/opa/opa.sock # OPA via Unix socket
|
||
```
|
||
|
||
Internal mode uses zddc-server's in-process evaluator — same Go code that
|
||
backs the cascade rules above, no network round-trip, no external dependency.
|
||
This is the default.
|
||
|
||
External mode POSTs each access decision to
|
||
`<ZDDC_OPA_URL>/v1/data/zddc/access/allow` with body:
|
||
|
||
```json
|
||
{
|
||
"input": {
|
||
"user": {"email": "alice@mycompany.com"},
|
||
"path": "/Project-A/sub/",
|
||
"policy_chain": {
|
||
"levels": [
|
||
{"acl": {}, "admins": ["admin@mycompany.com"]},
|
||
{"acl": {"allow": ["*@mycompany.com"]}}
|
||
],
|
||
"has_any_file": true
|
||
}
|
||
}
|
||
}
|
||
```
|
||
|
||
OPA evaluates the deployment's Rego policy against this input and returns:
|
||
|
||
```json
|
||
{"result": true}
|
||
```
|
||
|
||
Rego policy authors can implement either:
|
||
|
||
- **Same semantics as our internal evaluator** — walk
|
||
`input.policy_chain.levels` bottom-up, deny-first within a level,
|
||
default-deny when `input.policy_chain.has_any_file` is true. The
|
||
`.zddc` files in the served tree continue to drive policy unchanged.
|
||
- **Federal-mode tightening** — same chain, but parent denies are
|
||
absolute (no leaf-allow override of an ancestor's deny). NIST AC-6
|
||
least-privilege posture.
|
||
- **RBAC-from-IdP** — read additional fields from `input.user`
|
||
(e.g. `roles` populated by the upstream proxy from SAML/OIDC claims)
|
||
and decide based on those, treating `.zddc` as a file-tree map of
|
||
required-roles instead of explicit emails.
|
||
- **Anything else** — Rego is general-purpose policy; once you're
|
||
running real OPA, the constraints are whatever you write.
|
||
|
||
### OPA failure modes
|
||
|
||
External mode adds a network call to the request hot path. zddc-server
|
||
treats unreachable / non-2xx / malformed-response cases as **deny**
|
||
(fail-closed) by default, with a `WARN` log. Operators who prefer
|
||
availability over correctness — typically not federal — can flip this:
|
||
|
||
```sh
|
||
ZDDC_OPA_FAIL_OPEN=1 # allow on transport error
|
||
```
|
||
|
||
Always-WARN logging means a healthy run is silent and a sick OPA is
|
||
loud regardless of which mode you pick.
|
||
|
||
### OPA decision cache
|
||
|
||
External OPA mode wraps the HTTP/socket client in a small per-decision
|
||
cache (default 1 s TTL). A single `.archive` listing or directory
|
||
enumeration walks the cascade for every entry, which would otherwise
|
||
issue one OPA round-trip per entry; the cache collapses identical
|
||
`(email, decision-input)` tuples down to one call per TTL window.
|
||
|
||
The 1-second default is short enough that a `.zddc` edit is reflected
|
||
in the next listing (it's the same window as the fsnotify watcher's
|
||
debounce). Operators who want zero staleness — or who are running
|
||
their own caching layer in front of OPA — can disable:
|
||
|
||
```sh
|
||
ZDDC_OPA_CACHE_TTL=0 # no caching, every request → OPA
|
||
ZDDC_OPA_CACHE_TTL=5s # longer window for batchy workloads
|
||
```
|
||
|
||
The cache is per-process, in-memory, and capped at ~4096 entries with
|
||
opportunistic eviction of expired entries. Internal mode (the default)
|
||
gets no cache — the in-process Go evaluator is already cheaper than a
|
||
cache lookup would be.
|
||
|
||
### Reference Rego policy
|
||
|
||
The `--print-rego` flag emits the bundled reference Rego policies. Two
|
||
variants ship:
|
||
|
||
```sh
|
||
zddc-server --print-rego # standard cascade (commercial)
|
||
zddc-server --print-rego=standard # same
|
||
zddc-server --print-rego=federal # parent-deny-is-absolute (NIST AC-6)
|
||
```
|
||
|
||
The standard variant mirrors internal-mode semantics exactly — leaf-
|
||
level allows can override an ancestor's deny (the cascade's intentional
|
||
delegation property). The federal variant is the strict-least-privilege
|
||
posture: any deny anywhere in the chain is absolute, no leaf-level
|
||
override possible. Federal customers running their own OPA can drop
|
||
the federal Rego in unchanged, or use either as a starting point for
|
||
further customization.
|
||
|
||
Parity is enforced at build time. `zddc/internal/policy/parity_test.go`
|
||
imports the OPA Go module **as a test-only dependency**, evaluates both
|
||
bundled Regos against fixture sets and asserts:
|
||
|
||
- The standard Rego matches the internal Go evaluator on every documented
|
||
cascade scenario (`TestRegoParity_AllInternalCases`).
|
||
- The federal Rego agrees with the standard policy on every case where
|
||
no ancestor-deny intersects a leaf-allow, AND **disagrees** on the
|
||
cases where the AC-6 rule differs (`TestFederalRego_DivergencesFromStandard`).
|
||
This way both policies are guaranteed to behave as documented.
|
||
|
||
The test-only import means the production binary stays OPA-free (still
|
||
13 MB) — the OPA library is in `go.mod` but not in `go build`'s output.
|
||
|
||
This gives you both ends of the spectrum: a single OPA-aware codebase
|
||
where the production decider is pure Go (no library bloat, no extra
|
||
process), the wire format is OPA-canonical (just point an external OPA
|
||
at it and decisions delegate seamlessly), and the bundled reference
|
||
Rego is a parity-tested artifact you can ship alongside or extend.
|
||
|
||
Typical federal customizations on top of the bundled Rego:
|
||
|
||
- **Parent-deny-is-absolute** — flip the leaf-allow-overrides-parent-deny
|
||
rule for NIST AC-6 least-privilege posture.
|
||
- **Role-based access** — read additional input fields like
|
||
`input.user.roles` populated by the upstream proxy from SAML/OIDC
|
||
claims, and decide based on those instead of (or alongside) email.
|
||
- **Time-of-day or IP-range constraints** — Rego can read
|
||
`input.context.now` and request metadata for context-aware
|
||
decisions.
|
||
- **SIEM-shipped decision logs** — OPA's logging plugins emit every
|
||
decision in a structured format ready for Splunk Government, Elastic
|
||
Federal, etc.
|
||
|
||
### Reference deployment shapes
|
||
|
||
**Commercial / default**: nothing to set. `ZDDC_OPA_URL=internal` is
|
||
the implicit default; the in-process evaluator handles every decision.
|
||
No sidecar, no extra port, no extra binary.
|
||
|
||
**Federal sidecar**: deploy OPA alongside zddc-server (k8s sidecar,
|
||
nomad task, or systemd service on the same host), bind it to
|
||
`127.0.0.1:8181` (or a Unix socket), point `ZDDC_OPA_URL` at it. OPA
|
||
loads the deployment's bundled Rego policy from a configured source
|
||
(filesystem, signed bundle from S3, OPAL, etc.) and is patched
|
||
independently of zddc-server.
|
||
|
||
**Per-tenant policy variants**: run multiple OPA instances each loaded
|
||
with a different bundle, point each zddc-server replica at the
|
||
appropriate one. The `.zddc` files in the served tree stay the same;
|
||
the *interpretation* of those files differs per tenant.
|
||
|
||
### Future work
|
||
|
||
Items the conversation flagged as friction in operator setup or as documented
|
||
gaps that warrant code, in addition to the federal-readiness items above:
|
||
|
||
- `.zddc.form.yaml` ACL editor (built on the form-data system) once
|
||
file-as-truth round-trip preserves comments — turn the manual YAML edit
|
||
into a self-service UI for project owners.
|
||
- Save-time validation that warns when a `.zddc` change would lock the
|
||
editing user out (or have a measurable effect they didn't anticipate).
|
||
- `zddc-server policy export` command emitting every `.zddc` file's resolved
|
||
effect, suitable for change-control review (and a prerequisite for the
|
||
CM-3 federal control above).
|
||
- Per-decision caching for external OPA mode (small TTL on (email, path)
|
||
to amortize the .archive listing's per-entry round-trip).
|
||
- A reference Rego bundle shipped alongside the binary that exactly
|
||
reproduces internal mode, plus a "federal-mode" variant that flips
|
||
the parent-deny-is-absolute toggle. Useful as a starting point for
|
||
customers who want to extend rather than write from scratch.
|
||
|
||
## Admin Debug Page
|
||
|
||
`zddc-server` exposes a built-in debug page at `/.profile/` for operators who can
|
||
push code/images but cannot `kubectl exec` into the running container. It surfaces:
|
||
|
||
- **`/.profile/whoami`** — every header on the current request, the configured email
|
||
header name, the value observed at that name, and the resolved email. This is the
|
||
first thing to look at when access logs show `email=anonymous` — it tells you
|
||
exactly which (if any) header the upstream proxy is sending.
|
||
- **`/.profile/config`** — the resolved `Config` (env vars). Equivalent to
|
||
`kubectl exec -- env | grep ^ZDDC_` for diagnosing chart / deployment overrides.
|
||
- **`/.profile/logs`** — recent log entries (last 500) from an in-memory ring buffer.
|
||
Optional `?level=info|warn|error|debug` and `?since=<RFC3339>` query params.
|
||
At `ZDDC_LOG_LEVEL=debug` every request also logs its full header map under
|
||
`msg=request headers` — useful for diagnosing proxy / SSO header passthrough
|
||
(e.g. confirming which header carries the email). Note: that dump includes
|
||
auth tokens and cookies; only enable debug in trusted environments.
|
||
- **`/.profile/effective-policy?path=...&email=...`** — cascade tracer.
|
||
Returns the resolved policy chain (every level along the walk from
|
||
`ZDDC_ROOT` to the requested path), the active decider's allow/deny
|
||
verdict, the per-level email-match breakdown, and the decider kind
|
||
(`*policy.InternalDecider` / `*policy.cachingDecider`). When
|
||
`ZDDC_OPA_URL` points at an external OPA, the decision goes through
|
||
that OPA — also a useful smoke test for OPA wiring. See "Debugging
|
||
permissions" above.
|
||
- **`/.profile/`** — HTML dashboard that fetches the JSON endpoints client-side.
|
||
|
||
### Authorization
|
||
|
||
Authorization is via an `admins:` list in the **root** `.zddc` file (`<ZDDC_ROOT>/.zddc`).
|
||
Patterns use the same glob syntax as `acl.allow` / `acl.deny`:
|
||
|
||
```yaml
|
||
admins:
|
||
- alice@mycompany.com
|
||
- "*@admin.mycompany.com"
|
||
acl:
|
||
allow:
|
||
- "*@mycompany.com"
|
||
```
|
||
|
||
The root-only invariant (subdirectory `admins:` entries are silently ignored, so
|
||
subtree write authority cannot be self-promoted to admin) is documented under
|
||
"Trust model and invariants" in the access-control section above — that's the
|
||
canonical home; this section just shows the syntax.
|
||
|
||
If the root `.zddc` has no `admins` list (or no `.zddc` exists), every admin
|
||
endpoint returns **404** to every caller. Non-admin requests also receive 404
|
||
(not 403) so the existence of the admin page is invisible to unauthorized
|
||
callers.
|
||
|
||
### Forward-auth target for upstream proxies
|
||
|
||
`zddc-server` also exposes `GET /.auth/admin` — a machine-only endpoint that
|
||
returns **200** if the caller's resolved email is in the root `.zddc` `admins:`
|
||
list, **403** otherwise. No body, no redirect, no UI; it is a pure
|
||
authorization decision intended to be polled by an upstream proxy's
|
||
forward-auth directive (Caddy `forward_auth`, nginx `auth_request`, Traefik
|
||
`ForwardAuth`, etc.).
|
||
|
||
The intended use case is gating *adjacent* services on the same pod / host
|
||
that don't have their own ACL. Concretely: the dev-shell deployment runs
|
||
both `zddc-server` and `code-server` behind one Caddy listener; Caddy uses
|
||
`forward_auth` to ask `/.auth/admin` whether the caller is allowed to reach
|
||
`/devshell/*` (the IDE) before forwarding. zddc-server's own routes (`/`,
|
||
`/<project>/`, `/.archive/`, etc.) keep their existing `.zddc`-cascade ACL
|
||
and don't go through this endpoint.
|
||
|
||
```caddy
|
||
# example: protect /devshell/* with forward_auth on /.auth/admin
|
||
handle_path /devshell/* {
|
||
forward_auth 127.0.0.1:9090 {
|
||
uri /.auth/admin
|
||
copy_headers X-Auth-Request-Email
|
||
}
|
||
reverse_proxy 127.0.0.1:8443 # code-server
|
||
}
|
||
```
|
||
|
||
The check is cheap (one map lookup against the cached `PolicyChain`); calling
|
||
it on every request is fine. Edits to `/srv/.zddc` propagate within the
|
||
fsnotify watcher's debounce window (~2 s) — no service restart needed.
|
||
|
||
### Caveats
|
||
|
||
- Logs are in-memory and lost on restart. The buffer holds the most recent 500
|
||
records; for long-term audit, parse the stderr stream the way you already do.
|
||
- The page reads only configuration and request state — it does not modify anything.
|
||
- An interactive terminal is not yet available; that's planned as a follow-up
|
||
behind a separate `ZDDC_ADMIN_TERM=1` env-var gate so it stays opt-in.
|
||
|
||
## Apps: virtual tool HTMLs
|
||
|
||
`zddc-server` virtually serves the five tool HTMLs (archive, transmittal,
|
||
classifier, mdedit, landing) at the appropriate paths. The current-stable
|
||
build of each tool is **baked into the binary at compile time** via
|
||
`//go:embed`; that's the default. No fetch happens out of the box.
|
||
|
||
### Where each tool is served
|
||
|
||
| App | Available at |
|
||
|---------------|-------------------------------------------------------------------------|
|
||
| `archive` | every directory (multi-project, project, archive, vendor) |
|
||
| `classifier` | any `Incoming`, `Working`, or `Staging` directory and its subtree |
|
||
| `mdedit` | any `Working` directory and its subtree |
|
||
| `transmittal` | any `Staging` directory and its subtree |
|
||
| `landing` | only at the deployment root (the project picker) |
|
||
|
||
Outside these locations, the corresponding `<app>.html` URL returns 404.
|
||
|
||
### Override and version-pin
|
||
|
||
For any path, the resolution order is:
|
||
|
||
1. **Real file at the path** — operator drops `archive.html` (or any other)
|
||
into a directory; the static handler serves it. Beats everything below.
|
||
2. **Closer-to-leaf `.zddc apps:` entry** — walks `.zddc` files leaf→root
|
||
for an `apps.<app>` entry. The first match wins. Spec is one of:
|
||
- `stable` / `beta` / `alpha` (canonical upstream channel)
|
||
- `v0.0.4` / `v0.0` / `v0` (canonical upstream version pin)
|
||
- `https://...` (full URL to a custom mirror)
|
||
- `./local.html` / `/abs/path.html` (local file)
|
||
3. **Embedded** — the build-time HTML compiled into the binary.
|
||
|
||
URL sources are fetched once on first request and cached forever in
|
||
`<ZDDC_ROOT>/_app/<scheme>/<host>[:<port>]/<path>`. There is no background refresh and no
|
||
hash verification — to pull a new build, delete the cache file. Concurrent
|
||
misses for the same URL share one outbound fetch (singleflight). Direct
|
||
URL access to `/_app/...` is blocked at dispatch; cached HTMLs are served
|
||
only via the apps resolver.
|
||
|
||
If a configured URL fetch fails (network down, 5xx), the server falls back
|
||
to the embedded copy and emits a one-time WARN log per source. The
|
||
`X-ZDDC-Source` response header always reports what was served:
|
||
`fetch:URL`, `cache:URL`, `path:/abs`, or `embedded:<app>@<build>`.
|
||
|
||
### Example
|
||
|
||
```yaml
|
||
# <ZDDC_ROOT>/Project-A/.zddc
|
||
apps:
|
||
classifier: alpha # track alpha for this project
|
||
archive: https://my-mirror.internal/zddc/archive_v0.0.4.html # custom mirror, pinned
|
||
mdedit: ./our-mdedit.html # local fork
|
||
```
|
||
|
||
### Env vars
|
||
|
||
| Variable | Default | Purpose |
|
||
|----------------------|---------|----------------------------------------------------------|
|
||
| `ZDDC_BUILD_VERSION` | `dev` | String stamped into `X-ZDDC-Source: embedded:<app>@<v>` |
|
||
|
||
The landing page fetches `GET /` (with `Accept: application/json`) to retrieve the list
|
||
of top-level project directories the requesting user has access to. It renders checkboxes
|
||
for each project and opens `archive.html?projects=Proj-A,Proj-B` when the user clicks
|
||
"Open Archive".
|
||
|
||
**Presets** (named project selections) are stored in the browser's `localStorage` — no
|
||
server-side state required.
|
||
|
||
**Shared URLs**: the `?projects=` parameter is preserved in the archive browser URL so
|
||
users can email direct links to a pre-filtered view. If the recipient does not have
|
||
access to a project listed in the URL, a warning banner is shown.
|
||
|
||
## Caching and ETags
|
||
|
||
zddc-server uses content-hash ETags wherever a re-fetch of identical
|
||
content is plausible — directory listings, the project list, the
|
||
embedded HTML tools.
|
||
|
||
| Endpoint | ETag source | Notes |
|
||
|---|---|---|
|
||
| `GET /` (project list, `Accept: application/json`) | SHA-256 prefix of the response body | Refetched on every request; the JSON is rebuilt from current FS state. The hash is the actual response, so 304s are always trustworthy regardless of filesystem-watcher reliability. |
|
||
| `GET /<dir>/` (directory listing, `Accept: application/json`) | SHA-256 prefix of the response body | Same. |
|
||
| `GET /<dir>/` (HTML browse) | Hash of the embedded `browse.html` template | Computed once at startup, memoized. Changes only on binary redeploy. |
|
||
| `GET /<app>.html` (embedded tools) | Hash of the embedded bytes | Same — memoized at startup. |
|
||
|
||
**Why content-hash and not server-side caching?** The cascade walks
|
||
`.zddc` files on every directory access; an `os.ReadDir` runs to build
|
||
the listing; the ACL filter applies. A cache keyed on directory mtime
|
||
would save that server work but depends on reliable filesystem
|
||
watching — and Azure Files SMB mounts (a common deployment substrate)
|
||
do not support `inotify`/`fsnotify` reliably. Content-hash ETags
|
||
deliver only the bandwidth savings, not the server-work savings, but
|
||
they cannot lie about staleness regardless of watcher behavior. A
|
||
future enhancement could add an mtime-keyed cache for environments
|
||
with reliable watchers, behind a feature flag.
|
||
|
||
The response headers are:
|
||
|
||
```
|
||
Cache-Control: private, max-age=0, must-revalidate
|
||
ETag: "<16-hex-char hash>"
|
||
```
|
||
|
||
`must-revalidate` ensures every refresh round-trips the server (which
|
||
re-runs the cascade and ACL filter); `max-age=0` means no client-side
|
||
freshness window; `private` prevents intermediary caches from sharing
|
||
responses across users (each user has their own ACL-filtered view).
|
||
|
||
## Access Logging
|
||
|
||
Every HTTP request is logged as a structured `slog` entry at `INFO` level:
|
||
|
||
| Field | Description |
|
||
|---|---|
|
||
| `ts` | Request arrival timestamp (RFC3339) |
|
||
| `email` | User email from the configured header, or `anonymous` |
|
||
| `method` | HTTP method |
|
||
| `path` | URL path |
|
||
| `status` | HTTP response status code |
|
||
| `bytes` | Response body bytes written |
|
||
| `duration_ms` | Request duration in milliseconds |
|
||
|
||
Log output goes to `stderr`. Use `ZDDC_LOG_LEVEL=warn` to suppress access logs if needed,
|
||
or pipe `stderr` to a log aggregator.
|
||
|
||
## Virtual Archive Index (`.archive`)
|
||
|
||
Any URL path segment named `.archive` (configurable via `ZDDC_INDEX_PATH`) is intercepted
|
||
by the server and treated as a virtual document index.
|
||
|
||
The index is built at startup by scanning all transmittal folders under `ZDDC_ROOT`. It
|
||
maps each `(project, trackingNumber, revision, modifier)` tuple to the file from the
|
||
**chronologically earliest** transmittal folder within that project that contains it.
|
||
|
||
### Project scoping
|
||
|
||
The `.archive` index is **scoped to the project** — i.e. the first slash-separated
|
||
segment of the request's `.archive` context path. The same tracking number issued
|
||
under two different projects does NOT collide; each project's `.archive/` surfaces
|
||
only that project's documents.
|
||
|
||
A request to `/.archive/...` at the very root has no project segment to scope by
|
||
and returns **404 Not Found**. Stable references must always be project-rooted
|
||
(e.g. `/ProjectA/.archive/TRK-001.html`).
|
||
|
||
Within one project, two different files claiming to be the same `(tracking, rev)`
|
||
are an authoring mistake. The chronological winner still wins, but a `WARN`
|
||
log is emitted with both paths so the conflict can be diagnosed and corrected.
|
||
|
||
### URL patterns
|
||
|
||
| URL | Resolves to |
|
||
|---|---|
|
||
| `GET /Project/.archive/TRK-001.html` | Latest base revision of TRK-001 within Project |
|
||
| `GET /Project/.archive/TRK-001_A.html` | Base revision A of TRK-001 within Project |
|
||
| `GET /Project/.archive/TRK-001_A+C1.html` | Modifier C1 of revision A of TRK-001 within Project |
|
||
| `GET /Project/.archive/` | JSON listing of Project's resolvable entries |
|
||
| `GET /Project/sub/sub/.archive/TRK-001.html` | Same as the top-level Project listing — depth within a project doesn't change scope |
|
||
| `GET /.archive/...` | **404** — root has no project segment |
|
||
|
||
Successful `.html` responses **serve the resolved file's bytes inline** at the
|
||
`.archive/` URL — no `Location` redirect. The per-transmittal URL is hidden on
|
||
purpose: external links of the form `.archive/<tracking>.html#section` keep
|
||
tracking the latest revision. A redirect would expose the snapshot URL and any
|
||
forwarded link would pin to that snapshot instead of "latest." Cache-Control is
|
||
`no-cache` so each load revalidates against the on-disk file's
|
||
`Last-Modified`/`ETag`; when a new revision lands the resolver picks it and the
|
||
browser refetches. ACL is enforced on both the `.archive` context directory and
|
||
the resolved target file (per-target denial returns 404, not 403, to avoid
|
||
disclosing that the tracking number exists in a hidden subtree).
|
||
|
||
### Why "earliest" transmittal?
|
||
|
||
Within one project, any file claiming to be `TRK-001_A (IFC)` should be identical
|
||
across transmittals (same content, same SHA-256). If the same tracking number and
|
||
revision appears in multiple transmittals, the first one received chronologically is
|
||
treated as the authoritative copy. A later arrival with a different file path is an
|
||
error condition; the server logs a `WARN` with both paths but does not change the
|
||
winner.
|
||
|
||
### Index refresh
|
||
|
||
The index refreshes automatically via an `fsnotify` filesystem watcher. Changes are
|
||
debounced by 2 seconds before the relevant transmittal folder is re-indexed.
|
||
|
||
> **Note for Azure Files**: Azure SMB mounts do not support `inotify`/`fsnotify` reliably.
|
||
> The watcher will log a warning and the index will only be updated by restarting the server.
|
||
|
||
## ZDDC Filename Convention
|
||
|
||
The server parses filenames following the ZDDC convention:
|
||
|
||
```
|
||
trackingNumber_revision (status) - title.extension
|
||
```
|
||
|
||
| Part | Format | Example |
|
||
|---|---|---|
|
||
| `trackingNumber` | No spaces or underscores | `123456-EL-SPC-2623` |
|
||
| `revision` | `~?[A-Z0-9]+(\+[CBNQ][0-9]+)?` | `A`, `~B`, `C+C1` |
|
||
| `status` | One of the valid status codes | `IFC`, `REC`, `---` |
|
||
| `title` | Free text | `Electrical Specification` |
|
||
|
||
Valid status codes: `IFA IFB IFC IFD IFI IFP IFR IFU REC RSA RSB RSC RSD RSI ---`
|
||
|
||
Transmittal folder format: `YYYY-MM-DD_trackingNumber (STATUS) - title`
|
||
|
||
## Integration with Archive Browser
|
||
|
||
The Archive Browser (`archive.html`) can connect to zddc-server in HTTP mode. The server
|
||
returns JSON directory listings in exactly the same format as Caddy's `file-server --browse`
|
||
— no changes to `archive/js/source.js` are needed.
|
||
|
||
To use: install `archive.html` at `ZDDC_ROOT/archive.html` (or any subdirectory) — either
|
||
the actual built tool downloaded by the self-contained install snippet, or one of the
|
||
six-line stubs from the project-subdir / track-upstream snippets that fetches it. Then
|
||
open it via the zddc-server URL; the app will auto-connect and scan the directory tree.
|
||
|
||
## Distribution
|
||
|
||
Each stable release is a Codeberg git tag (`zddc-server-vX.Y.Z`) with four pre-built binaries attached as release assets:
|
||
|
||
| File | Platform |
|
||
|---|---|
|
||
| `zddc-server-linux-amd64` | Linux (x86-64) |
|
||
| `zddc-server-darwin-amd64` | macOS (Intel) |
|
||
| `zddc-server-darwin-arm64` | macOS (Apple Silicon) |
|
||
| `zddc-server-windows-amd64.exe` | Windows (x86-64) |
|
||
|
||
All binaries are statically linked (CGO disabled), built with `-trimpath -ldflags="-s -w -X main.version=<ver>"`. No runtime dependencies.
|
||
|
||
Download URLs from Codeberg directly:
|
||
|
||
```
|
||
https://codeberg.org/VARASYS/ZDDC/releases/download/zddc-server-vX.Y.Z/zddc-server-linux-amd64
|
||
```
|
||
|
||
Browse all releases at <https://codeberg.org/VARASYS/ZDDC/releases>.
|
||
|
||
There is no alpha/beta channel for binary distribution. Active dev/soak happens via the [`helm/zddc-server-dev/`](../helm/zddc-server-dev/) chart, which builds zddc-server from source on every pod restart against any commit you point it at. There is no container image; if you want your own, copy the static binary into a `FROM scratch` or `FROM alpine` base in a few lines, or use one of the helm charts which compile from source via init container.
|
||
|
||
### Env-var contract (for chart consumers)
|
||
|
||
Downstream Helm charts and Compose files should set these explicitly:
|
||
|
||
| Variable | Typical value (behind ingress + SSO) | Purpose |
|
||
|---|---|---|
|
||
| `ZDDC_ROOT` | `/srv` | Path of the served archive (volume mount) |
|
||
| `ZDDC_TLS_CERT` | `none` | TLS terminated upstream |
|
||
| `ZDDC_INSECURE_DIRECT` | `1` | Acknowledge plain HTTP behind a trusted proxy |
|
||
| `ZDDC_ADDR` | `:8080` | Match service / probe port |
|
||
| `ZDDC_EMAIL_HEADER` | `X-Auth-Request-Email` | Header your auth proxy sets |
|
||
| `ZDDC_CORS_ORIGIN` | *(unset)* | Leave unset for embedded-tools deployments (same-origin); set to your tool host (`https://tools.acme.com`) only for self-hosted-tools or CDN-bootstrap layouts |
|
||
|
||
See "Environment Variables" above for the full list.
|
||
|
||
## Building from source
|
||
|
||
Requires Go 1.24+.
|
||
|
||
```sh
|
||
# Single binary for the host platform
|
||
(cd zddc && go build -o zddc-server ./cmd/zddc-server)
|
||
|
||
# All four release platforms (cross-compiled, statically linked)
|
||
./build # at the repo root — silently skips if Go isn't on PATH
|
||
# → outputs to zddc/dist/zddc-server-{linux,darwin,windows}-*
|
||
```
|
||
|
||
To run unit tests:
|
||
|
||
```sh
|
||
(cd zddc && go test ./...)
|
||
```
|
||
|
||
## Release tagging
|
||
|
||
zddc-server has no separate release script. The repo's top-level `./build alpha|beta|release [version]` is the canonical path: it cross-compiles the four binaries inside the containerized Go toolchain, copies them into `dist/release-output/` with the lockstep symlink chain (one set per platform), regenerates the per-version + per-channel stub pages, refreshes the index, and (on stable cuts) tags `zddc-server-v<X.Y.Z>` alongside the five HTML-tool tags.
|
||
|
||
```sh
|
||
./build release # lockstep stable, coordinated next version
|
||
./build release 1.2.0 # lockstep stable, explicit version
|
||
./build alpha # lockstep alpha cut
|
||
./build beta # lockstep beta cut
|
||
./deploy --releases # publish dist/release-output/ to /srv/zddc/releases/
|
||
```
|
||
|
||
The script tags every tool but does NOT push — finish with `git push origin main && git push origin --tags` (and run `./deploy` to put the artifacts on the live site).
|
||
|
||
Prerequisites:
|
||
|
||
- Go 1.24+ available inside the build container (downloaded automatically — `docker.io/golang:1.24-alpine`).
|
||
- `podman` (preferred) or `docker` on PATH.
|
||
|
||
Single-developer / solo-release flow by design — no CI babysitting, no separate dashboard to debug. The script fails loudly and visibly on the developer's terminal if anything goes wrong.
|
||
|
||
### Versioning
|
||
|
||
Clean semver, lockstep across all six tools (5 HTML + zddc-server). Stable cuts get `<tool>-vX.Y.Z` tags for every tool, all six sharing the same X.Y.Z. There are no alpha/beta tags — channel URLs are stable URLs by design (counters defeat that). Active dev runs via `helm/zddc-server-dev/`, which builds from source on each rollout.
|
||
|
||
The two existing `zddc-server-v0.0.8-alpha.1` and `zddc-server-v0.0.8-alpha.2` tags from a previous experiment stay as historical artifacts; no new alpha/beta tags are created going forward.
|
||
|
||
---
|
||
|
||
**Notes:**
|
||
|
||
- The `.archive` virtual path resolves ZDDC tracking numbers to their earliest-received revision
|
||
- ACL is enforced via bottom-up `.zddc` file evaluation
|