ZDDC
6d132572d3
Decision: external OPA is a bring-your-own-policy escape hatch, not a supported turnkey mode — so stop shipping access_federal.rego. A verb-blind read-ACL policy under NIST AC-6 branding is a liability to hand a federal evaluator, and (like access.rego before the fail-close) it over-granted writes and ignored WORM. The HTTPDecider + Decider interface stay: operators who want an AC-6 ancestor-deny-absolute posture write their own Rego. - Delete rego/access_federal.rego, FederalRego, --print-rego=federal, and federal_parity_test.go; trim the federal cases from rego_failclosed_test.go. - Reframe every doc reference (rego.go, main.go, file.go, ARCHITECTURE.md, README.md) to "operators write their own Rego"; rewrite the README "Reference Rego policy" section to describe the single fail-closed read-ACL skeleton accurately (it also still carried the now-removed "mirrors exactly" parity claim). Out of scope (flagged): the broader federal-readiness narrative (FedRAMP/FIPS/IdP) and the separate website page federal.html still discuss federal posture — the OPA bring-your-own-Rego path stays valid, but a deliberate review with the federal go-to-market in mind is warranted. go vet + full go test ./... green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
956 lines
81 KiB
Markdown
956 lines
81 KiB
Markdown
# ZDDC Architecture
|
|
|
|
This document is the single authoritative reference for how ZDDC tools are designed and built. It covers the shared single-file HTML application pattern, the build system, tool-specific architectural decisions, and contribution guidelines.
|
|
|
|
---
|
|
|
|
## Why Single-File HTML Applications
|
|
|
|
Every ZDDC tool compiles to a single self-contained `.html` file — no servers, no installers, no subscriptions.
|
|
|
|
| Principle | Rationale |
|
|
|-----------|-----------|
|
|
| **Reliability** | Opens in any modern Chromium-based browser without network access or external services |
|
|
| **Portability** | Can be emailed, archived, or deployed to air-gapped environments with no tooling |
|
|
| **Auditability** | Source, embedded data, and output travel together, satisfying ZDDC traceability requirements |
|
|
| **Longevity** | Static assets remain functional long after build environments have changed |
|
|
| **Simplicity** | A single `.html` file eliminates deployment steps and brittle dependency chains |
|
|
|
|
---
|
|
|
|
## ADR: Browse-as-shell with preview-pane plugins (target architecture)
|
|
|
|
**Status:** accepted; migrating incrementally (2026-06).
|
|
|
|
**Context.** The seven tools have started converging on browse: it already hosts classifier (grid iframe), tables (the table-leaf iframe), forms, and the md / yaml / `.zddc`-form editors in its preview pane, and the header chrome (profile menu + elevation) is shared across every tool. Rather than maintain seven parallel apps, the target is **one shell with a plugin content pane**.
|
|
|
|
**Decision.** Browse is the shell — header + tree + preview pane, one top-level document. Content tools render into the preview pane as **plugins**. Server-only behaviour (the account menu, permission-gated affordances) is **progressive enhancement**: it activates when zddc-server serves the page and `/.profile/access` answers, and is simply absent on `file://`. We do **not** iframe browse inside a server-rendered header — browse owns its header and the server enhances it in place. (So "browse opened locally is missing the server header" resolves to "the same header with its server-only items dormant," not a separate page.)
|
|
|
|
- **Server mode** is the security boundary: browse fetches ACL-gated listings + per-entry verbs; plugins act through a capability object and can't exceed what the server grants.
|
|
- **Local mode** (`file://`) is unrestricted: a picked FS-Access directory handle, no server, no account menu — by design.
|
|
|
|
**Plugin contract.** A plugin is a module on `window.app.modules`; the shell dispatches to the first whose `handles` returns true:
|
|
|
|
```
|
|
handles(node, ctx) -> bool // claim this node / selection?
|
|
render(node, container, ctx) // mount into the preview pane (or a host element)
|
|
dispose?() // tear down (called before switching away)
|
|
isDirty?() / currentNode?() // optional: unsaved-edit guard + re-render hooks
|
|
```
|
|
|
|
`ctx` is the capability object the shell supplies — the ONLY thing that differs between server and local mode, so a plugin is written once:
|
|
|
|
```
|
|
ctx = {
|
|
mode: 'server' | 'fs',
|
|
getArrayBuffer(node), getContentWithVersion(node), // read (etag/lastmod → optimistic concurrency)
|
|
saveFile(node, bytes, contentType, opts), // write: ACL-enforced (server) / FS-Access (local)
|
|
cap.has(node, verb), // 'rwcda' subset; '' or unknown offline
|
|
// server-only (undefined offline): access(path), elevation, history(node)
|
|
}
|
|
```
|
|
|
|
The md / yaml / `.zddc`-form editors already follow this shape (`handles` / `render` / `isDirty` / `currentNode` + a ctx with `getArrayBuffer` / `getContentWithVersion`); table-leaf and classifier-grid are the same idea via an iframe bridge. Formalising `ctx` makes the contract explicit and lets the heavy tools migrate from iframe to in-pane module — preferred, for shared selection / theme / permission state with no `postMessage`.
|
|
|
|
**Migration (incremental; standalone tools keep working throughout).**
|
|
1. ✓ Editors are in-pane modules; classifier / tables / forms embed in the pane; the shell header carries the profile menu + progressive-enhancement elevation.
|
|
2. ✓ The two bespoke, chrome-less server pages — `/.tokens` and `/.profile` — now render through the tables engine via server-injected `#table-context` + the generic `apiActions` layer (see AGENTS.md "Server-injected collections"). That's the "dynamic collection → declarative table, not a bespoke page" half proven.
|
|
3. Fold `archive` into the tree + a listing plugin.
|
|
4. Make `landing` the shell's root ("no project selected") view — note `landing` is feature-rich (saved groups, multi-select, filters), so this is a *plugin* migration that preserves those, NOT a tables-fication.
|
|
5. Move `transmittal` into a workflow plugin.
|
|
6. Flip `default_tool` routing to "browse + plugin X"; retire each standalone `<app>.html` only once its plugin lands. (`archive`/`landing`/`transmittal` are all feature-rich — each fold is a deliberate, scoped effort, not a quick tables swap.)
|
|
|
|
**Consequences / tradeoffs.**
|
|
- Preserves the single-file + offline value: the shell still builds to one `browse.html` that runs from `file://`. Heavy plugins should lazy-load in server mode to keep the bundle reasonable.
|
|
- The server stays the only security boundary; local is unrestricted by definition.
|
|
- Seven lockstep release artifacts collapse toward one shell (plus optionally-separate plugins).
|
|
- Not every tool is a clean pane plugin — `transmittal` is workflow-heavy, `landing` is really the root view — called out above.
|
|
|
|
---
|
|
|
|
## Repository Structure
|
|
|
|
Every HTML tool follows the same directory layout:
|
|
|
|
```
|
|
tool/
|
|
README.md # Feature scope, UI design, domain rules, help content
|
|
css/ # Logically separated stylesheets (one responsibility per file)
|
|
js/ # Vanilla ES modules (one responsibility per file)
|
|
template.html # Shell markup with {{PLACEHOLDER}} markers for development
|
|
build.sh # Inlines css/ and js/ into dist/tool.html
|
|
dist/
|
|
tool.html # Generated output — never edit this manually
|
|
```
|
|
|
|
Website files (what `zddc.varasys.io` serves) live on a **separate Codeberg repo** (`codeberg.org/VARASYS/ZDDC-website`, typically cloned at `~/src/zddc-website/`) for hand-edited content, plus the **deploy host's `/srv/zddc/`** for the assembled live site. The system Caddy bind-mounts `/srv/zddc/`. `./deploy` rsyncs both into it.
|
|
|
|
```
|
|
~/src/zddc-website/ (clone of codeberg.org/VARASYS/ZDDC-website)
|
|
index.html # hand-edited intro page + install snippets (root URL)
|
|
reference.html # hand-edited file-naming convention spec
|
|
css/, js/, img/ # hand-edited static assets
|
|
README.md, LICENSE # repo housekeeping
|
|
# NO releases/ — release artifacts are NOT in any git history.
|
|
|
|
~/src/zddc/dist/release-output/ (gitignored, produced by ./build release)
|
|
index.html # download page, regenerated by build
|
|
<tool>_v<X.Y.Z>.html # real per-version HTML (immutable)
|
|
<tool>_v<X.Y.Z>.html.sig # detached Ed25519 signature
|
|
<tool>.html → <tool>_v<X.Y.Z>.html # canonical "current stable" symlink
|
|
<tool>.html.sig → <tool>_v<X.Y.Z>.html.sig # canonical .sig symlink (chains to per-version .sig)
|
|
zddc-server_v<X.Y.Z>_<platform> # real per-version cross-compiled binary (raw bytes, no LFS)
|
|
zddc-server_v<X.Y.Z>_<platform>.sig # detached signature
|
|
zddc-server_<platform> → ... # canonical per-platform symlink → current stable
|
|
zddc-server_<platform>.sig → ... # canonical .sig symlink
|
|
zddc-server_v<X.Y.Z>.html # per-version stub: 4 platform downloads for that version
|
|
zddc-server.html # canonical stub: 4 platform downloads for current stable
|
|
pubkey.pem # signing pubkey (seeded from live)
|
|
|
|
/srv/zddc/ (deploy host; Caddy bind-mount)
|
|
index.html, reference.html, css/, js/, img/ ← rsync'd from ~/src/zddc-website/
|
|
releases/ ← rsync'd from ~/src/zddc/dist/release-output/
|
|
```
|
|
|
|
`<tool>` ∈ {archive, transmittal, classifier, landing, form, tables, browse}. `<platform>` ∈ {linux-amd64, darwin-amd64, darwin-arm64, windows-amd64.exe}.
|
|
|
|
Every URL under `/releases/` resolves directly via the symlink chain — no `manifest.json`, no Caddy regex-rewrite, no JavaScript indirection, no third-party mirror. Caddy serves these as plain static files. Two URL shapes per tool: `<tool>.html` (canonical, mutable symlink → current stable) and `<tool>_v<X.Y.Z>.html` (immutable per-version pin). Same for zddc-server per platform. The May 2026 simplification dropped channel mirrors (`_stable`, `_beta`, `_alpha`) and partial-version pins (`_v<X.Y>`, `_v<X>`) — operators pin to exact versions when they want stability, otherwise track the canonical URL.
|
|
|
|
**zddc-server binaries are reproducible from a tag, not in git** — `./build release` cross-compiles them into `dist/release-output/`, `./deploy` rsyncs them to `/srv/zddc/releases/`, Caddy serves from there. Older versions: `git checkout zddc-server-v0.0.8 && ./build release 0.0.8`. The `helm/zddc-server-{prod,dev,cache}/` charts build from source via init container, but operators who want a prebuilt binary just `curl -O https://zddc.varasys.io/releases/zddc-server_linux-amd64`. The four-platform fan-out lives at `zddc-server.html` (current stable) or `zddc-server_v<X.Y.Z>.html` (per-version).
|
|
|
|
To preview a build locally, open `dist/tool.html` directly via the dev server. To publish on `zddc.varasys.io`, cut a release with `./build release` and then `./deploy`.
|
|
|
|
Vendor dependencies (bundled third-party libraries) live in `tool/vendor/` if present. The build script is responsible for inlining them into the output.
|
|
|
|
---
|
|
|
|
## Documentation ownership
|
|
|
|
Each topic has exactly one authoritative home; everything else links to it.
|
|
|
|
| Topic | Single home | Linked from |
|
|
|---|---|---|
|
|
| What ZDDC is + tool channel links + dual-mode (local/server) overview + install snippets | `~/src/zddc-website/index.html` (hand-edited intro for `zddc.varasys.io/`, in the `ZDDC-website` repo) | repo `README.md`, `zddc/README.md` |
|
|
| File-naming convention spec (status codes, modifiers, folder format) | `~/src/zddc-website/reference.html` | repo `README.md`, in-tool help text |
|
|
| Versions + channel builds index of every tool | `dist/release-output/index.html` (regenerated by `./build`; deployed to `/srv/zddc/releases/index.html`) | website intro nav, "Browse all versions" link |
|
|
| Customer-deployment install (`zddc-server` binary embeds current-stable tools; local override via an on-disk `<app>.html` or the site `<root>/.zddc.zip` bundle — no fetch) | `zddc/README.md` "Apps: virtual tool HTMLs" section | website intro, `AGENTS.md` |
|
|
| zddc-server operations: env vars, ACL syntax, `.archive` URLs, container vs binary | `zddc/README.md` | `AGENTS.md`, website intro |
|
|
| Build / release / channel commands | `AGENTS.md` | repo `README.md` ("see AGENTS.md") |
|
|
| Architecture & internal patterns | `ARCHITECTURE.md` (this file) | `AGENTS.md` |
|
|
| Per-tool internal design quirks | `<tool>/README.md` | (linked from website intro tool cards) |
|
|
|
|
`index.html` in the `ZDDC-website` repo (working dir `~/src/zddc-website/index.html`) is **hand-edited static content** (analogous to `reference.html`), not the landing-tool output. The install section points operators at two paths: **local** (download a `.html` file from `/releases/`) and **server** (run `zddc-server`; current-stable builds of all eight HTML tools are baked into the binary at compile time via `//go:embed`). The landing tool's released bytes live at `/srv/zddc/releases/landing_v<X.Y.Z>.html` (rsync'd from `dist/release-output/`); the embedded copy serves at the deployment root by default. The public website at `zddc.varasys.io/` is the same hand-edited `index.html` — its root URL is the introduction page, not the project picker (because there are no projects to pick from a static site).
|
|
|
|
When updating documentation, prefer linking over duplicating. If you find yourself rewriting the file-naming convention in a tool's README, link to `reference.html` instead.
|
|
|
|
---
|
|
|
|
## Build System
|
|
|
|
### How It Works
|
|
|
|
Each HTML tool's `build.sh`:
|
|
|
|
1. Reads CSS files in declaration order, concatenates them
|
|
2. Reads JS files in declaration order, concatenates them
|
|
3. Processes `template.html` with `awk`, replacing `{{PLACEHOLDER}}` markers with the concatenated content and stripping CDN `<script>`/`<link>` tags
|
|
4. Writes the result to `dist/tool.html`
|
|
5. If `--release [<version>]` was passed (stable cut), calls `promote_release` to write into `dist/release-output/`: per-version immutable file + canonical `<tool>.html` symlink + .sig companion. Beta cuts skip — `./build beta` is internal-only (embedded regen + chore commit), no public artifact.
|
|
|
|
The top-level `./build` at the repository root is the canonical lockstep entry point. It:
|
|
|
|
1. On a stable cut, **seeds `dist/release-output/` from `/srv/zddc/releases/`** — copying only the immutable per-version files (`<tool>_v<X.Y.Z>.html`, `zddc-server_v<X.Y.Z>_<plat>`) + their `.sig` sidecars + `pubkey.pem`. The canonical symlinks get rewritten by this cut; any stale files in the live tree are cleaned by deploy's `--delete-after`.
|
|
2. Forwards `--release [version]` to every HTML tool's build (or `--release beta` for the snapshot path), computing a coordinated next-stable target via `_coordinated_next_stable` (max of every tool's latest tag + 1) when no explicit version is given.
|
|
3. Cross-compiles zddc-server for the four target platforms inside a containerized Go toolchain (podman/docker).
|
|
4. On a stable cut, calls `promote_zddc_server` to copy the freshly cross-compiled binaries into `dist/release-output/` with per-platform canonical symlinks (`zddc-server_<platform>`) and `.sig` companions; tagging `zddc-server-v<X.Y.Z>` is deferred to the embedded-commit block at the end.
|
|
5. Calls `write_zddc_server_stubs_all` to refresh the per-version + canonical stub HTML pages from whatever artifacts are in `dist/release-output/`.
|
|
6. Regenerates `dist/release-output/index.html` as the action-first download page.
|
|
|
|
Then `./deploy --releases` rsyncs `dist/release-output/` → `/srv/zddc/releases/` with `--delete-after`.
|
|
|
|
### Release verbs
|
|
|
|
Two release verbs (plus dev). The May 2026 simplification dropped alpha and made beta internal-only.
|
|
|
|
- **Stable** — versioned, immutable. `./build release [version]` writes per-version HTML for the seven HTML tools and per-version binaries for zddc-server (real bytes), writes the canonical `<tool>.html` and `zddc-server_<platform>` symlinks → the new version, signs every per-version artifact (and companion `.sig` symlink for the canonical URL), and tags `<tool>-v<X.Y.Z>` for every tool at the release commit.
|
|
- **Beta** — `./build beta` is the BMC dev chart's plumbing. It regenerates `zddc/internal/apps/embedded/*` with beta-labeled HTML bytes and makes a `chore(embedded): cut v<X.Y.Z>-beta` commit. No public artifact in `dist/release-output/`. The chart's appVersion pins to `"<X.Y.Z>-beta-<sha>"`; its Dockerfile parses the suffix and `git fetch`-es that SHA, compiling its own binary from the fetched source.
|
|
- **Dev** — plain `./build` (no arg) produces `dist/<tool>.html` and `zddc/dist/zddc-server-<platform>` binaries; doesn't touch `dist/release-output/`, the live site, or `embedded/`.
|
|
|
|
The on-page `{{BUILD_LABEL}}` is rendered red+bold for dev/beta builds (`is_red=1`) and black for stable releases. The label format is:
|
|
|
|
| Build | Label |
|
|
|--------------------|--------------------------------------------------------|
|
|
| dev (no `--release`) | `v0.0.6-dev · 2026-04-27 14:00:00 · abc1234[-dirty]` |
|
|
| `--release beta` | `v0.0.6-beta · 2026-04-27 14:00:00 · abc1234` |
|
|
| `--release [ver]` | `v0.0.5` |
|
|
|
|
`X.Y.Z` for non-stable labels is the **next-stable target** — patch+1 from the latest clean `<tool>-vX.Y.Z` tag. Dev builds use the `-dirty` marker so iterative work is distinguishable from a formal beta cut.
|
|
|
|
### Install distribution model
|
|
|
|
Two orthogonal axes: how the bytes get there (this section), and what runtime mode the tool ends up in (next section).
|
|
|
|
**Local mode** (no server): download a tool `.html` from <https://zddc.varasys.io/releases/> and open it. Tools are self-contained — no install, no install script.
|
|
|
|
**Server mode** (`zddc-server`): the binary `//go:embed`s the current-stable build of every tool at compile time. That's the default — no fetching happens out of the box. The server virtually serves each tool only at directories where the convention says it belongs (`internal/apps/availability.go`):
|
|
|
|
| App | Available at |
|
|
|---------------|-------------------------------------------------------------------------|
|
|
| `archive` | every directory (multi-project, project, archive, vendor) |
|
|
| `classifier` | any `Incoming`, `Working`, or `Staging` directory and its subtree |
|
|
| `browse` | every directory (hosts the markdown editor as a preview plugin) |
|
|
| `transmittal` | any `Staging` directory and its subtree |
|
|
| `landing` | only at the deployment root |
|
|
|
|
Resolution order at a request to `<dir>/<app>.html` where the app is available:
|
|
|
|
1. **On-disk override** — real `.html` file at the path → static handler.
|
|
2. **Site bundle** — an `<app>.html` member of `<ZDDC_ROOT>/.zddc.zip`, read server-side via `internal/zipfs` (see `internal/apps/bundle.go`). Local file, no fetch, no signature; re-stat'd each request for free hot-reload.
|
|
3. **Embedded** — the build-time HTML compiled into the binary.
|
|
|
|
Resolution is LOCAL-ONLY — no network fetch, no signatures, no channels/versions, and no `apps:` `.zddc` key (all removed in favour of this model). `.zddc.zip` is config, not content: a direct `GET /.zddc.zip` is 404 for everyone, while the server reads its members from the filesystem internally. To change a tool's HTML: drop a file at the path, add `<app>.html` to `.zddc.zip`, or rebuild the binary.
|
|
|
|
The `X-ZDDC-Source` response header always reports what was served: `bundle:<app>.html`, `embedded:<app>@<build>`, or (for an on-disk override) the static handler's own headers.
|
|
|
|
### Runtime mode detection
|
|
|
|
Independent of how the tool got installed. `archive` auto-detects from the URL and folder shape (`?projects=` set → multi-project; scan root has an `archive/` child → project-root; otherwise → in-archive). The other tools don't care — `transmittal`, `classifier`, `browse` work the same regardless of where they live.
|
|
|
|
### Build Script Requirements
|
|
|
|
Every `build.sh` must:
|
|
|
|
- Begin with `#!/bin/sh` and `set -eu` (POSIX sh, not bash)
|
|
- Source `shared/build-lib.sh` first (provides `ensure_exists`, `concat_files`, `build_timestamp`, `compute_build_label`, `promote_release`, plus the lockstep helpers `_coordinated_next_stable`, `promote_zddc_server`, `write_zddc_server_stubs_all`)
|
|
- Fail immediately on missing source files (`ensure_exists` pattern)
|
|
- Clean up temp files on exit (use `trap cleanup EXIT`)
|
|
- Accept `--release [<version>]` for stable cuts or `--release beta` for snapshot cuts; otherwise produce a dev build
|
|
|
|
### HTML Embedding Safety
|
|
|
|
When inlining JavaScript into a `<script>` block, the HTML parser scans for the exact string `</script>` to terminate the block — backslash escaping (`<\/script>`) does **not** prevent termination. Any JS source file or vendor library that contains `</tag>` sequences inside string literals or template literals will break the inline `<script>` block.
|
|
|
|
The rule is:
|
|
|
|
> **All `</` sequences in inlined JavaScript must be escaped as `<\/` using `sed`.**
|
|
|
|
Both the app JS concatenation step and any vendor JS bundling step must run through:
|
|
|
|
```bash
|
|
sed 's#</#<\\/#g' "$input_js" > "$safe_js"
|
|
```
|
|
|
|
Then use `</script>` (not `<\/script>`) to close the `<script>` block, since the content no longer contains any `</` sequences that the parser could misread.
|
|
|
|
This is already enforced for browse's Toast UI bundling. It is the contributor's responsibility to ensure new tools follow this pattern.
|
|
|
|
### Vendor Dependencies
|
|
|
|
Some tools bundle third-party libraries. These live in `tool/vendor/` and are committed to the repository. The build script inlines them into `dist/tool.html`.
|
|
|
|
**Current vendor files:**
|
|
|
|
| Tool | Library | File | Notes |
|
|
|------|---------|------|-------|
|
|
| browse | Toast UI Editor v3.2.2 | `shared/vendor/toastui-editor-all.min.js` | Markdown editor (loaded by `browse/js/preview-markdown.js`) |
|
|
| browse | Toast UI Editor CSS | `shared/vendor/toastui-editor.min.css` | Editor stylesheet |
|
|
| shared | jszip | `shared/vendor/jszip.min.js` | ZIP read for previews + classifier hash-export |
|
|
| shared | docx-preview | `shared/vendor/docx-preview.min.js` | DOCX preview |
|
|
| shared | xlsx (SheetJS) | `shared/vendor/xlsx.full.min.js` | XLSX/XLS preview |
|
|
| shared | UTIF | `shared/vendor/utif.min.js` | TIFF preview |
|
|
|
|
**No runtime CDN loads.** Every external dependency is vendored into
|
|
`shared/vendor/` and concatenated into each tool's bundle at build
|
|
time. Tools that need a given library include the vendor path in
|
|
their `build.sh`'s `concat_files` JS list. The "ship the record
|
|
player with the record" philosophy: a downloaded `.html` file works
|
|
offline against any file the user can open, with no network
|
|
dependency at runtime.
|
|
|
|
Trade-off accepted: bundle sizes are larger. archive, classifier,
|
|
transmittal land around 1.5 MB after gzip; browse lands around 2 MB
|
|
because it carries Toast UI + jszip + docx-preview + xlsx + UTIF
|
|
for the in-place markdown editor and the preview pane.
|
|
Justified by the offline-first guarantee: any tool downloaded from
|
|
`/releases/` works without network, against air-gapped archives,
|
|
forever. See ARCHITECTURE.md § "Why Single-File HTML Applications"
|
|
for the longer rationale.
|
|
|
|
`template.html` for tools with vendor deps loads those deps from CDN
|
|
purely for **dev convenience** — opening a template.html directly in
|
|
Chromium gives you a working tool without running a build. The build
|
|
script strips/replaces those CDN tags so the dist HTML has every
|
|
dependency inlined. No CDN URLs survive into the dist.
|
|
|
|
### Development vs Production
|
|
|
|
| Context | Tailwind / Vendor | How to run |
|
|
|---------|-------------------|-----------|
|
|
| Development | CDN (live, from `template.html`) | Open `template.html` directly in Chromium |
|
|
| Production | Bundled / Static CSS | Run `bash tool/build.sh`, open `dist/tool.html` |
|
|
|
|
For browse specifically: `template.html` loads Toast UI from CDN for dev convenience. The build replaces it with the bundled vendor file (`shared/vendor/toastui-editor-all.min.js`).
|
|
|
|
---
|
|
|
|
## JavaScript Architecture
|
|
|
|
### Vanilla JS Only
|
|
|
|
All tools use plain JavaScript — no TypeScript, no frameworks, no bundlers. Dependencies are managed manually via vendor files.
|
|
|
|
### Module Pattern
|
|
|
|
Each JS file wraps its code in an IIFE or module-scope block and registers its API on `window.app.modules`:
|
|
|
|
```javascript
|
|
// js/mymodule.js
|
|
(function() {
|
|
function doSomething() { ... }
|
|
|
|
window.app.modules.mymodule = { doSomething };
|
|
})();
|
|
```
|
|
|
|
Two top-level globals:
|
|
|
|
- `window.app` — per-tool app state, modules, and debug surface (every tool)
|
|
- `window.zddc` — shared filename/folder/revision parsers from `shared/zddc.js` (every tool)
|
|
|
|
No other globals. Never expose implementation internals beyond what's needed for testing.
|
|
|
|
### Module Load Order
|
|
|
|
JS files are concatenated in the order declared in `build.sh`. Each file can assume earlier files' modules are available on `window.app`. Circular dependencies are not permitted — modules must be layered.
|
|
|
|
Typical ordering:
|
|
|
|
```
|
|
app.js ← Declares window.app and top-level state
|
|
utils.js ← Stateless helpers (no dependencies)
|
|
store.js ← State management (depends on app.js)
|
|
[domain].js ← Feature modules (depend on store/utils)
|
|
main.js ← Initialization (depends on all modules)
|
|
```
|
|
|
|
### State Management
|
|
|
|
Three patterns coexist. **For new tools, prefer the first one** — direct mutation on `window.app` with explicit re-render. It's debuggable, it's the most common pattern in this codebase (archive, browse, form, tables), and it doesn't hide control flow.
|
|
|
|
**1. Direct mutation on `window.app` + explicit re-render** *(recommended for new tools)*
|
|
|
|
```javascript
|
|
window.app = { files: [], selectedFolders: new Set(), modules: {}, ... };
|
|
// Mutate then re-render:
|
|
window.app.files.push(newFile);
|
|
window.app.modules.table.render();
|
|
```
|
|
|
|
State is read directly. Mutations trigger explicit `render()` calls — no auto-tracking, no surprise updates. Used by archive, browse, form, tables, landing.
|
|
|
|
**2. Pub-sub store on top of #1** (classifier)
|
|
|
|
```javascript
|
|
store.set('files', newFiles);
|
|
store.on('files', render);
|
|
```
|
|
|
|
Adds a tiny `store.on(key, fn)` / `store.notify(key)` layer in `classifier/js/store.js`. Justification: classifier has multiple independent panels (file list, spreadsheet, validation pane) that all need to react to the same state changes; calling three separate `render*()` functions from every mutation site would invite forgetting one.
|
|
|
|
**3. Proxy-based reactive state** (transmittal)
|
|
|
|
```javascript
|
|
const state = createReactiveState({ mode: 'edit', published: false });
|
|
state.subscribe((prop, newVal) => { /* auto-update UI */ });
|
|
state.mode = 'view'; // Proxy notifies all subscribers automatically
|
|
```
|
|
|
|
Used by transmittal because a single state change (e.g. `mode`) drives ≥3 independent UI regions (header chrome, body editability, action toolbar). Reactive shines when the cross-cutting wiring would otherwise be tedious. **Don't reach for this pattern unless you have at least three subscribers per state property.**
|
|
|
|
### `zddcMode` dispatcher (form / tables unified bundle)
|
|
|
|
The form and tables tools share a single compiled bundle (`tables/dist/tables.html`, also `//go:embed`d into `zddc-server` at `zddc/internal/handler/tables.html`). One window, two views. The bundle holds both `window.tablesApp` and `window.formApp`; whichever app paints is decided by a single global:
|
|
|
|
```javascript
|
|
// Set by the server-injected context (or absent for standalone form.html):
|
|
window.zddcMode = 'form' // → form renderer paints; tables app no-ops
|
|
window.zddcMode = 'table' // → tables app paints; form app no-ops
|
|
window.zddcMode = undefined // → standalone form.html, treated as 'form'
|
|
```
|
|
|
|
Each app's `main.js` checks `window.zddcMode` first and returns early when it's not their mode (see `form/js/main.js:10`, `tables/js/mode.js`). Rules for adding a third mode:
|
|
|
|
1. Set `window.zddcMode = '<new>'` in `tables/js/context.js` based on server context shape.
|
|
2. Add the new app's main module with the same early-return guard.
|
|
3. Keep the standalone-fallback rule consistent: undefined `zddcMode` should still mean "the lightest, most common mode for this bundle's standalone HTML."
|
|
|
|
Standalone `form/dist/form.html` uses this contract too — it has no `zddcMode` set, so form's main runs unconditionally and renders either the schema (when injected) or a friendly empty-state welcome (`form/js/main.js renderStandaloneWelcome`).
|
|
|
|
---
|
|
|
|
## Tool-Specific Architecture
|
|
|
|
### Archive Browser
|
|
|
|
**Pattern:** Direct mutation of `window.app.{directories, files, filteredFiles, selectedFiles, ...}`, helper modules namespaced under `window.app.modules.{events, table, urlState, source, ...}`. Supports two source modes (`window.app.sourceMode`): `'local'` (File System Access API) and `'http'` (zddc-server JSON browse).
|
|
|
|
**Two-level directory structure required:**
|
|
|
|
```
|
|
root-directory/
|
|
transmittal-folder/ ← "grouping folder" — must be a subdirectory
|
|
123456-EL-SPC-0001_A (IFC) - Spec.pdf
|
|
...
|
|
```
|
|
|
|
Files at the root level are ignored. The grouping folder list and transmittal folder list are populated from the first two levels of the selected directory. Files are only counted in `filteredFiles` after ZDDC filename parsing succeeds.
|
|
|
|
**Key DOM IDs:** `#addDirectoryBtn`, `#noDirectoryMessage`, `.main-container`, `#filesTableBody`, `#fileCount`, `#selectedCount`, `#selectAllGroupingCheckbox`.
|
|
|
|
---
|
|
|
|
### Document Classifier
|
|
|
|
**Pattern:** Event-driven store (`store.js`) with `notify()` / `on()` pub-sub, spreadsheet rendering on `'files'` events.
|
|
|
|
**File object shape** (as produced by `scanner.js`):
|
|
|
|
```javascript
|
|
{
|
|
trackingNumber: '123456-EL-SPC-2623',
|
|
title: 'Specification',
|
|
revision: 'A',
|
|
status: 'IFC',
|
|
extension: 'pdf', // no leading dot
|
|
originalFilename: '...', // filename without extension
|
|
name: '...', // full filename with extension
|
|
path: 'folder/filename.pdf',
|
|
size: 45000,
|
|
isDirectory: false,
|
|
manualFilename: null // set if user overrides computed name
|
|
}
|
|
```
|
|
|
|
**`computeNewFilename(file)`** (in `utils.js`) returns `file.originalFilename + '.' + file.extension` if any required field is missing.
|
|
|
|
**Main app panel** (`#mainApp`) stays hidden (class `hidden`) until a real directory is opened via `showDirectoryPicker`. State can be injected via `store.setFolderTree()` + `store.setSelectedFolders()` for testing without triggering the picker.
|
|
|
|
---
|
|
|
|
### Markdown Editor (browse preview plugin)
|
|
|
|
**Lives at:** `browse/js/preview-markdown.js`, registered on `window.app.modules.markdown` and invoked by `browse/js/preview.js` for `.md`/`.markdown` files. The standalone `mdedit/` tool was retired in favour of this plugin.
|
|
|
|
**Pattern:** Editor instances per-file (constructed by `render(node, container, ctx)`, disposed by `dispose()`). CSS Grid layout for the shell — sidebar (FM textarea on top, outline below) on the left, content (info header + Toast UI editor) on the right.
|
|
|
|
**Front matter:** Parsed off the file on load by `parseFrontMatter()` (a small `---\n…\n---` parser); the FM body goes into a sidebar `<textarea>`, the markdown body into the Toast UI editor. On save, `assembleContent()` recombines them with the envelope on top. The textarea is always present so authoring brand-new FM is a single click; dirty tracking covers both halves via a SHA-256 hash of the assembled bytes.
|
|
|
|
**Dependencies:** Toast UI Editor v3.2.2 (vendored at `shared/vendor/toastui-editor-all.min.js`, concatenated into `browse/dist/browse.html` at build time). No runtime CDN, no Tailwind.
|
|
|
|
**Server-mode features:** When the file handle is an `HttpFileHandle` (so `node.url` is set and `state.source === 'server'`), three Download buttons appear in the file header — DOCX/HTML/PDF — fetching `?convert=<fmt>` via `window.zddc.source.downloadConverted()`. Clicks auto-save first if the buffer is dirty so converted bytes reflect what's on screen. The server-side engine is in `zddc/internal/convert`: zddc-server `exec.Command`s `pandoc` and `chromium-browser` directly, and the runtime image's wrapper at `/usr/local/bin/<name>` (see `zddc/runtime.Containerfile` + `zddc/runtime/zddc-sandbox-exec`) handles the per-call cgroup v2 + bubblewrap sandbox between that exec and the real binary at `/usr/bin/<name>`. Isolation strategy lives entirely in the image; swap the wrapper for firejail / nspawn / podman-run and Go doesn't change.
|
|
|
|
---
|
|
|
|
### Transmittal Creator
|
|
|
|
**Pattern:** Proxy-based reactive state, two-phase hydration, ECDSA digital signatures, SHA-256 file integrity.
|
|
|
|
**Two-phase hydration:**
|
|
|
|
1. **`populateStatic()`** — called before publishing. Fills all form fields and the file table into the HTML so the output is readable without JavaScript (progressive enhancement for SharePoint, email clients, etc.).
|
|
2. **`hydrate()`** — called on page load of a published transmittal. Hides the "Not Validated" static warning, runs signature verification, and enables interactive features.
|
|
|
|
**Progressive enhancement matrix:**
|
|
|
|
| Feature | No JavaScript | With JavaScript |
|
|
|---------|--------------|-----------------|
|
|
| Content display | ✅ Full | ✅ Full |
|
|
| File table | ✅ Shown | ✅ Shown |
|
|
| Digest / signatures | ✅ Listed | ✅ Listed + cryptographically verified |
|
|
| Validation status | ⚠️ "Not Validated" badge | ✅ "Verified" / ❌ "Invalid" |
|
|
| Editing | ❌ Disabled | ✅ Enabled (if draft) |
|
|
| Column filtering | ❌ No | ✅ Yes |
|
|
|
|
**Data store:** A `<script id="transmittal-data" type="application/json">` element embedded in the published HTML holds the full transmittal payload. On load, `data.js` reads and parses it; all UI state derives from this JSON.
|
|
|
|
**Reactive state:**
|
|
|
|
```javascript
|
|
// app.state is a Proxy — assignments auto-notify subscribers
|
|
app.state.mode = 'view'; // Triggers UI updates automatically
|
|
```
|
|
|
|
Subscribe for cross-cutting concerns:
|
|
```javascript
|
|
app.state.subscribe((property, newValue) => {
|
|
if (property === 'mode') updateModeToggleLabel(newValue);
|
|
});
|
|
```
|
|
|
|
**Security model:** ECDSA P-256 signing of the SHA-256 digest. Signatures are stored in the JSON payload. Any number of signers can co-sign. Verification runs client-side in the browser's Web Crypto API — no server required.
|
|
|
|
**Key module globals:** `window.transmittalApp` exposes `app.data`, `app.state`, and `app.modules` for debugging and testing.
|
|
|
|
---
|
|
|
|
### Form Renderer (`form/`)
|
|
|
|
**Pattern:** Schema-driven renderer for the form-data system. Reads a JSON Schema 2020-12 + RJSF-style `ui:*` hints from a server-injected `<script id="form-context">` block; recursively walks the schema and mounts a tree of widgets; on submit, walks the widget tree to serialize back to JSON and POSTs to the URL the form was loaded from.
|
|
|
|
**Why schema-driven** (vs. transmittal's hardcoded HTML): the form tool is generic — one renderer serves any form spec a user (or LLM) drops into the file tree. Adding a new form requires no code change; adding a new field type to an *existing* form requires only a YAML edit.
|
|
|
|
**Widget interface** — every widget exposes:
|
|
- `el` — DOM root
|
|
- `read()` — current value (recurses into children for object / array)
|
|
- `setError(msg)` / `clearErrors()` — show / clear field-level errors
|
|
- `child(name|idx)` — for container widgets, look up nested widget by JSON-Pointer segment (used by `errors.js` to attach server-side validation messages by path)
|
|
|
|
**Module layout:**
|
|
- `js/app.js` — `window.formApp = { context, rootWidget, modules }`
|
|
- `js/context.js` — read injected `#form-context` JSON
|
|
- `js/util.js` — `h()` DOM builder, JSON-Pointer encode/parse
|
|
- `js/widgets.js` — primitives (string/number/integer/boolean/enum, format date/email, textarea)
|
|
- `js/object.js` — fieldset rendering with `ui:order` resolution
|
|
- `js/array.js` — repeating-row UX (add/remove)
|
|
- `js/render.js` — type-triage dispatcher
|
|
- `js/serialize.js` — read tree → JSON
|
|
- `js/errors.js` — distribute errors by JSON Pointer path
|
|
- `js/post.js` — POST + handle 200/201/422/403/409 responses
|
|
- `js/main.js` — boot: load context, mount root widget, wire submit
|
|
|
|
**Server-side counterpart:** `zddc/internal/handler/formhandler.go` recognizes `*.form.html` and `*.yaml.html` URLs, parses the spec, validates submissions via `zddc/internal/jsonschema/`, writes via `zddc.WriteAtomic` (plain submissions) or `zddc/internal/handler/history.go` `WriteWithHistory` (record-typed YAML — mdl rows, rsk rows, ssr.yaml). Existence of `<name>.form.yaml` is the trigger; without it, the URL falls through to static-file serving.
|
|
|
|
**Record-vs-submission distinction.** "Records" are the three table-store types (mdl/rsk/ssr); everything else is a "submission." Records get server-stamped audit fields (`created_at`/`_by`, `updated_at`/`_by`, `revision`, `previous_sha`), an immutable per-record history at `<dir>/.zddc.d/history/<base>/<RFC3339Nano>-<sha8>.<ext>`, cascade-driven filename composition (via the `records:` + `field_codes:` `.zddc` keys), per-folder field locking (e.g. type=RSK in rsk/), and folder-bound fields (`folder_fields`, e.g. originator = party-folder name). The mechanism intercepts at every write entry point — the file-API `serveFilePut` (if `isRecordPath` matches → `WriteWithHistory`, else `WriteAtomic`), the in-dir form create/update (`serveFormCreate`/`serveFormUpdate`), and the project rollup (`serveFormCreateRollup`). Each resolves the `records:` rule for the target directory and, when one with a `filename_format` applies, composes the name via the shared `recordCreatePrep` and routes through `WriteWithHistory`; non-record paths keep the historical date+email `WriteAtomic` write. The convergence means there's no back door that writes an un-stamped, un-composed record. All of it is server-side: the tools opened offline (`file://` / FS-Access, no server) can't enforce audit, composition, `field_codes`, or `folder_fields` — record writes need zddc-server. See AGENTS.md "Records, audit, and history" for the operator surface (incl. the offline gap and pre-folder-binding upgrade notes); `zddc/internal/handler/history.go` for the orchestration.
|
|
|
|
**Round-trip philosophy:** v0 is "form-as-truth" — submission YAML is regenerated from form state on every save. Hand-edits to submission files are not preserved across re-edit→re-submit. v1 will add an opt-in "file-as-truth" mode (eemeli/yaml Document API) for forms like `.zddc` itself where users hand-edit and comments must survive.
|
|
|
|
---
|
|
|
|
## Server security model
|
|
|
|
zddc-server is the deployable surface — it gates access to the file tree that
|
|
all the HTML tools work against. The trust story is intentionally narrow,
|
|
delegated, and small enough to audit. This section frames it; operator detail
|
|
(syntax, anti-patterns, worked layouts, verification recipe, federal-readiness
|
|
gap analysis) lives in [`zddc/README.md`](zddc/README.md) § "Access control:
|
|
the `.zddc` cascade."
|
|
|
|
### Cooperating layers
|
|
|
|
Six layers cooperate to bound what a request can reach. Each does one job;
|
|
none of them is load-bearing alone.
|
|
|
|
| Layer | Job | Implementation |
|
|
|---|---|---|
|
|
| Authentication | Establish caller identity (email) | Two paths: `Authorization: Bearer <token>` validated against `<ZDDC_ROOT>/.zddc.d/tokens/<sha256-hex>` (CLI / scripted callers); or `X-Auth-Request-Email` injected by an upstream auth proxy (browser users). Token system is built-in and self-issuing — no external IDP required |
|
|
| Policy decider | Yield an allow/deny verdict for (identity, path, chain) | Pluggable via `ZDDC_OPA_URL`: in-process Go evaluator (default) or external OPA-compatible HTTP/socket endpoint. `zddc/internal/policy/` |
|
|
| ACL cascade | The default decider's rule set | Per-directory `.zddc` YAML with verb-set permissions (`r`/`w`/`c`/`d`/`a`) and roles, plus a baked-in default tree bottom layer (`zddc-server show-defaults`) that uses a recursive `paths:` tree to describe subfolder rules even before those folders exist. Walked deepest-first first-match-wins (`zddc/internal/zddc/cascade.go`, `walker.go`, `acl.go`). External OPA can replace this rule set with arbitrary Rego the operator writes (e.g. ancestor-deny-absolute for NIST AC-6) while keeping the same `.zddc` files as input data; zddc-server ships only a fail-closed read-ACL skeleton (`--print-rego`) as a starting point |
|
|
| Canonical-folder behaviour | Codify the bilateral exchange-record archetype | All driven by `.zddc` keys (baked into the embedded default tree): `auto_own:` / `auto_own_fenced:` — mkdir here writes a creator-owned `.zddc` (`<email>: rwcda`; fenced adds `acl.inherit:false`); `worm: [principal…]` — write-once-read-many (`w`/`d`/`a` stripped for everyone non-admin, `c` survives only for the listed principals; admins exempt); `virtual:` — never materialise on disk; `drop_target:` — browse shows a drag-drop upload overlay. The defaults put `auto_own` on `working`/`staging`/`archive-party`/`incoming` and `worm: [document_controller]` on `archive/<party>/{received,issued}`, so the canonical convention is unchanged — but an operator can reshape it (rename `received`/`issued`, mark any path WORM, …) without a code change. `zddc/internal/zddc/lookups.go`, `worm.go`, `roles.go`; the embedded default tree |
|
|
| Tool-rooted view | Make the caller's accessible subtree feel like their entire world (UX containment) | Archive auto-served at every directory; the URL it's served at *is* its root. No breadcrumb leads above |
|
|
| URL canonicalization | Resolve URL paths to on-disk casing before any layer below sees them | `zddc/internal/fs/resolve.go ResolveCanonical` — case-insensitive lookup with lowercase-wins tiebreak when sibling case variants exist on disk. File and folder names preserve case on disk; the canonicalization is purely URL→FS-name mapping. Virtual prefixes (`.archive`, `.profile`, `.tokens`) flow through verbatim |
|
|
| Reserved hidden prefixes | Hide operator side-state (caches, dev-shell home dirs) from listings and direct fetch | `.`-prefixed → 404 + listing-filtered; `_`-prefixed → listing-filtered only |
|
|
| Audit log | Reconstruct who did what after the fact | JSON-line tee per request to `<ZDDC_ROOT>/.zddc.d/logs/access-<host>.log`; writes also emit `file_write` op records |
|
|
| File API | Authenticated CRUD over the served tree | `zddc/internal/handler/fileapi.go` — PUT/DELETE/POST routed through the same ACL chain as GET, with per-method verbs (`r`/`w`/`c`/`d`/`a`). Mkdir under `Incoming`/`Working`/`Staging` writes a creator-owned `.zddc` automatically |
|
|
|
|
### Master + proxy / cache / mirror
|
|
|
|
The same `zddc-server` binary runs in two distinct topologies:
|
|
|
|
- **Master mode** (default): the binary owns a file tree under `--root`, applies `.zddc` ACL cascades to incoming requests, serves files / virtual app HTML / archive listings / form submissions / table views. The "normal" zddc-server. All of `cmd/zddc-server/main.go` lives here.
|
|
- **Client mode** (`--upstream <url>` set): the binary becomes a downstream proxy/cache/mirror against another zddc-server. The master-side machinery (archive index, apps server, watcher, OPA decider, ACL middleware, token store) is **bypassed entirely**. `zddc/internal/cache/` is the entire request handler.
|
|
|
|
Three sub-modes within client mode, controlled by `--mode <proxy|cache|mirror>`:
|
|
|
|
| Mode | Persists responses? | Subtree warmer? | Use case |
|
|
|---|---|---|---|
|
|
| `proxy` | no | no | thin pass-through; nothing on local disk |
|
|
| `cache` (default) | yes | no | field engineer — what you've viewed is available offline |
|
|
| `mirror` | yes | yes (access-triggered, subtree-scoped) | vendor mirrors of their subtree; admin backups; complete offline working set |
|
|
|
|
Internally the modes collapse to two switches on a single request-handling pipeline (`persist`, `warm`). Proxy is cache without disk writes; mirror is cache plus an access-triggered walker. Implementation factor: `cache.New` reads `cfg.Mode` once and sets `c.persist = mode != "proxy"`; the warmer is the only path that doesn't yet exist (phase 3).
|
|
|
|
**Mirror scope falls out of auth.** Whatever the client's bearer can see at upstream is what the cache can populate. Admin's bearer → mirror gets everything (full backup). Vendor's bearer → mirror is exactly that vendor's permitted subtree. No code distinguishes admin-vs-user — master-side ACL filtering does it.
|
|
|
|
#### Cache directory IS a normal ZDDC root
|
|
|
|
The cache directory layout is intentionally a regular ZDDC root: `<master>/foo/bar.txt` is stored at `<root>/foo/bar.txt`. No sidecar metadata files. The local file's `mtime` is set to the upstream's `Last-Modified` header (so revalidation via `If-Modified-Since` reflects the master's notion of file age, not local fetch time). A small `.zddc-upstream` marker file at the root records the upstream URL and first-cached-at timestamp, written once by `sync.Once` on first persist.
|
|
|
|
Two consequences:
|
|
|
|
- `zddc-server --root <cache-dir>` (without `--upstream`) serves whatever's been cached as a plain master. Useful for portable offline snapshots — tar the directory, hand it to a colleague, they have a working ZDDC.
|
|
- The master/client boundary is one flag: setting/unsetting `--upstream` switches behavior on the same on-disk root.
|
|
|
|
#### Pipeline
|
|
|
|
Phase 2 ships GET/HEAD only; writes are deferred to a later phase. For each incoming request:
|
|
|
|
1. **Directory request** (URL ends in `/`): always proxied live. Listing-cache support belongs with the mirror walker (phase 3) — the bare cache directory's contents only reflect visited files, so a local-walk listing would be misleading.
|
|
2. **File request, cache hit** (`persist` mode): serve cached bytes via `http.ServeContent` (which handles `Range` natively + 304 conditional GETs). Header `X-ZDDC-Cache: hit`. Background goroutine fires an `If-Modified-Since` revalidate; on `304` no-op, on `200` overwrite the cache atomically, on `403`/`404` purge.
|
|
3. **File request, cache miss**: build an upstream request preserving `Range`, `If-Range`, `Accept`, `Accept-Encoding`; attach the configured bearer. Stream the response simultaneously to the client AND to a tmp file in the cache directory; rename atomically only on success. Header `X-ZDDC-Cache: miss`.
|
|
4. **Proxy mode** (no persist): same as miss but skip the tmp-file teeing. Header `X-ZDDC-Cache: proxy`.
|
|
5. **Network error + cached version exists**: serve the cached bytes with `X-ZDDC-Cache: offline`. (When the cache hits before any network attempt, the header is `hit` — there's no way to distinguish "hit while online" from "hit while offline" without an extra round-trip; the header tells the user "this is from disk," and the user infers freshness from context or a future explicit freshness probe.)
|
|
6. **Network error + no cached version**: `503 Service Unavailable` + `X-ZDDC-Cache: offline`.
|
|
|
|
Responses with `Cache-Control: no-store` or `Cache-Control: private` pass through but are not persisted. Non-200 responses (including 206 partial content) are forwarded but not persisted — caching a partial body would corrupt subsequent full-body reads.
|
|
|
|
Hop-by-hop headers per RFC 7230 §6.1 (`Connection`, `Keep-Alive`, `Transfer-Encoding`, etc.) are dropped from forwarded responses; Go's transport drops most automatically, but the cache layer adds a guard for the cases that slip through.
|
|
|
|
#### Mirror walker (access-triggered)
|
|
|
|
`--mode mirror` adds an access-triggered subtree warmer (`zddc/internal/cache/walker.go`) on top of the cache pipeline. Naive design ("walk on a fixed timer") would scale poorly: many vendor mirrors against one master would generate thundering-herd polls of subtrees no human has looked at in months. Instead, walks are demand-triggered, rate-limited per-subtree.
|
|
|
|
Trigger policy (`MirrorScheduler.Trigger(urlPath)` is installed as the cache layer's `onAccess` hook, called in a goroutine on every authenticated request):
|
|
|
|
1. Match `urlPath` against the configured `--mirror-subtree`s. Longest prefix wins; `/` is a catch-all (full mirror).
|
|
2. If a walk is already in flight for that subtree, no-op.
|
|
3. If `now - last_walk_at < --mirror-min-interval` (default 1h), no-op.
|
|
4. Otherwise, mark in-flight and kick a walk goroutine.
|
|
|
|
Walk:
|
|
|
|
1. Recursively fetch JSON listings under the subtree, persisting each as `<dir>/.zddc-listing.json` (so directory browsing works offline for walked subtrees).
|
|
2. For each file, fire a conditional `If-Modified-Since` GET (bounded parallelism — default 4 concurrent, configurable). 304 = no-op; 200 = overwrite; 403/404 = purge.
|
|
3. Per-directory orphan purge: any local file present locally but absent from the upstream listing is removed (handles upstream deletes + ACL revocations).
|
|
|
|
State persists at `<root>/.zddc-mirror-state.json` as `{subtrees: {<path>: {last_walk_at}}}`. In-flight tracking is in-memory only — a crash mid-walk lets the next access retry without manual cleanup.
|
|
|
|
Properties:
|
|
- **Idle mirrors are quiet.** No requests means no walks means zero upstream traffic.
|
|
- **Active mirrors stay current as a side effect of normal use** (no explicit refresh gesture).
|
|
- **Revocation latency** is bounded by access frequency. Documented behavior, not a guarantee.
|
|
- **Bounded concurrency** keeps walks from starving the user's interactive requests on the same connection pool.
|
|
|
|
#### Writes: outbox + offline replay
|
|
|
|
`PUT` / `POST` / `DELETE` are handled by `cache.handleWrite`. Online: forwarded to upstream; on success the cached entry for the path (if any) is dropped so the next read fetches fresh. PUT/DELETE include `If-Unmodified-Since` from the cached file's mtime — the master returns `412 Precondition Failed` if its file changed since the cache observed it, so concurrent writes can't silently clobber.
|
|
|
|
When upstream is unreachable, the request is captured in the **outbox** (`zddc/internal/cache/outbox.go`) under `<root>/.zddc-outbox/<id>/` — `meta.json` (method, raw URI, content-type, base mtime, queued-at) + `body.bin` (request body, capped at `MaxOutboxBodyBytes` = 256 MiB). The client gets back `202 Accepted` + `X-ZDDC-Cache: queued` and a JSON envelope referencing the queued entry.
|
|
|
|
A background `RunReplayLoop` started by `runClient` in main.go replays in queue order:
|
|
- `2xx` → entry deleted; cached entry for the path (if any) dropped so the next read fetches fresh.
|
|
- `412` → entry renamed to `<id>.conflict-<RFC3339>/`. The conflict directory keeps both `meta.json` and `body.bin` intact for manual reconciliation.
|
|
- `4xx` other than `412` → entry dropped (won't succeed on retry; logged at `WARN`).
|
|
- `5xx` / transport error → left in place for the next pass.
|
|
|
|
Replay schedule: an eager pass at startup, then 30s while pending, 5min while idle. Honors graceful-shutdown context cancellation. Disabled in `--mode=proxy` (proxy mode persists nothing by design — offline writes just return `503`).
|
|
|
|
ID encoding (`<unix-nano-base16>-<hex-random>`) is lex-sortable so directory iteration replays in queue order without an explicit index. `MarkConflict` appends `.conflict-<ts>` to the directory name; if a same-second conflict collides (unlikely), a 4-char random suffix is appended.
|
|
|
|
The local cache is not updated for offline writes by design — until upstream confirms, the user reads still see the upstream-cached version (or 503 if uncached). Trade-off: the user doesn't see their own offline edits immediately, but no "did the queued write actually win?" ambiguity. Phase 5 will add a conflict-resolution UI that surfaces `.conflict-<ts>/` directories alongside the cached files in browse views.
|
|
|
|
#### Multi-tenancy: explicitly out of scope (v1)
|
|
|
|
The local instance forwards a single bearer (loaded from `--bearer-file` at startup) regardless of who's calling locally. Single-user-trust on a laptop. For multi-user scenarios, run multiple instances on the same host, or front the local server with your own auth proxy that injects per-user bearers downstream — both options keep the cache layer's design surface minimal.
|
|
|
|
#### Confused-deputy guard at startup
|
|
|
|
Because the cache forwards a bearer upstream without authenticating the local caller, exposing the bind on a non-loopback interface would turn the binary into an open-proxy laundering anyone's request through the master. The config layer (`zddc/internal/config/config.go`) enforces two defenses:
|
|
|
|
1. **Loopback default in client mode.** When `--upstream` is set, `--addr` defaults to `127.0.0.1:8443` instead of `:8443` — but only when `--addr` / `ZDDC_ADDR` was *not* set explicitly. CLI users on a laptop get safe-by-default; operators who want a non-loopback bind opt in explicitly.
|
|
2. **Refuse non-loopback bind + bearer without acknowledgement.** A non-loopback `--addr` *with* a configured `--bearer-file` *without* `--insecure-direct` (`ZDDC_INSECURE_DIRECT=1`) refuses to start. The error message names the bind, names the flag to acknowledge, and names the threat (open proxy confused-deputy). The helm `zddc-server-cache/` chart sets `ZDDC_INSECURE_DIRECT=1` and relies on Kubernetes-namespaced networking for the gating — that path is unaffected. The guard is bearer-file-conditional because proxy mode without a bearer doesn't have a credential to launder, and refusing it would needlessly block proxy-without-auth deployments.
|
|
|
|
### Bearer token issuance
|
|
|
|
zddc-server issues its own bearer tokens for non-browser callers (CLI tools, scripts, downstream proxy/cache/mirror instances). The master is the identity provider; no external IDP, no JWKS rotation.
|
|
|
|
**Storage** — `<ZDDC_ROOT>/.zddc.d/tokens/<sha256-hex>` per token. Filename is the **hash** of the token, never the plaintext value. File contents are YAML (`email`, `created`, `expires`, `description`). Mode 0600, directory mode 0700, atomic writes via temp+rename.
|
|
|
|
**Why hash-as-filename**: a leak of the tokens directory (backup tools, FS-level audit logs, accidental `ls` in a screen recording) exposes hashes, not credentials. Same posture as `/etc/shadow` storing password hashes rather than passwords. The plaintext exists only in transit (HTTP `Authorization` header) and on the operator's disk (a 0600 file they manage).
|
|
|
|
**Self-service flow**:
|
|
|
|
1. User signs in via the browser (master's normal upstream auth).
|
|
2. Visits `/.tokens` — small HTML page (`zddc/internal/handler/tokenhandler.go`) listing existing tokens and offering a creation form.
|
|
3. JS fetches the JSON API (`/.api/tokens`), POSTs a new token, displays the plaintext **once**.
|
|
4. User copies into a 0600 file; passes `--bearer-file <path>` to a CLI.
|
|
|
|
**API**:
|
|
|
|
| Method | Path | Purpose |
|
|
|---|---|---|
|
|
| `GET` | `/.api/tokens` | list current user's tokens (no plaintext) |
|
|
| `POST` | `/.api/tokens` | create; plaintext returned exactly once |
|
|
| `DELETE` | `/.api/tokens/<id>` | revoke (8-char ID or full 64-char hash) |
|
|
|
|
**Validation in the request path**: `ACLMiddleware` in `zddc/internal/handler/middleware.go` checks `Authorization: Bearer …` first; on success, sets the request email from the token file and falls through. Any failure (missing / malformed / expired) → `401`. There is no silent fallback to anonymous on Bearer failure — a misconfigured client must fail loudly rather than escalate to "no auth at all." When no Bearer is present, the existing `cfg.EmailHeader` path runs unchanged.
|
|
|
|
**Directory shielding**: the tokens path is shielded by the existing `.`-prefix rules — `dispatch()` 404s any URL containing a dot-prefixed segment (other than the recognized virtual prefixes), and `fs.ListDirectory` filters dot entries from listings. The token system relies on this; a regression here is a credentials-leak vector. The token-handler test suite (`tokenhandler_test.go`) exercises the auth path; verifying the URL-level guard is the responsibility of `main_test.go` (`TestDispatchHidesDotPrefixedSegments`).
|
|
|
|
### `--no-auth` / "this instance is not the ACL boundary"
|
|
|
|
A symmetric flag, used in two distinct deployment shapes:
|
|
|
|
- **Master with `--no-auth`**: no ACL enforcement, no auth required. Anyone hitting the port reads everything in scope. Suitable for dev, internal trusted-LAN read-only tooling, or genuinely public archives.
|
|
- **Client with `--no-auth`** (downstream proxy/cache/mirror — see "Master + proxy / cache / mirror" below for context): the client trusts upstream's ACL filtering. Whatever the upstream returned is what the client serves; no per-request re-evaluation against `.zddc` files in the cache directory. Single-user-trust model on a laptop.
|
|
|
|
Implementation is a single swap: `policy.AllowAllDecider{}` replaces the configured decider when `cfg.NoAuth` is true. All existing handlers continue to call `policy.AllowFromChain` (or equivalent) unchanged; they just always get `allowed=true`. Logged at `WARN` on every restart so operators who set the flag inadvertently see it on stderr.
|
|
|
|
Distinct from `--insecure`, which only relaxes a startup-time safety check (refuse to start when no root `.zddc` exists). The two flags are independent.
|
|
|
|
### Commercial vs federal trust model
|
|
|
|
The current implementation is well-shaped for a commercial-tenant model with
|
|
delegated auth. Federal-grade qualification (FedRAMP Moderate, NIST 800-53,
|
|
FIPS 140-3, DoD STIG) requires several layers to harden. Operators deciding
|
|
whether to deploy the system should know which column they're in.
|
|
|
|
| Property | Commercial trust model (current) | Federal trust model (gaps to close) |
|
|
|---|---|---|
|
|
| Identity | Email from upstream proxy header | mTLS or signed forwarding token; PIV/CAC via IdP |
|
|
| Cryptography | Go stdlib defaults | FIPS 140-3 validated module (microsoft/go or RHEL FIPS) |
|
|
| TLS | Go stdlib defaults | Explicit MinVersion ≥ TLS 1.2, DoD-approved cipher allowlist, OCSP stapling, HSTS |
|
|
| Access model | Per-verb (`r`/`w`/`c`/`d`/`a`) with first-class roles and an admin escape hatch — closes NIST AC-3(7) | (closed by default; external Rego still available for org-specific policy via `ZDDC_OPA_URL`) |
|
|
| Subtree authority | In-process decider: leaf grants override ancestor denies (delegation primitive). Federal posture: deploy OPA with the operator's own ancestor-deny-absolute Rego (NIST AC-6) | (closed; federal posture is the OPA path) |
|
|
| Audit log integrity | Local lumberjack rotation, filesystem-trusted | Tamper-evident (signed chain or external append-only sink), 1y online + 3y archive |
|
|
| Information disclosure | Anonymous reaches `/` and `/.profile` (project picker, public-projects names) | All endpoints behind authenticated proxy; no anonymous discovery |
|
|
| Apps URL fetches | Fetch-once-cached, no integrity check | SHA-256 pin + signature verification |
|
|
| Disclosure process | Not formalized | `SECURITY.md`, embargoed-fix workflow, CVE assignment |
|
|
|
|
The full bullet list with NIST control references is in
|
|
[`zddc/README.md`](zddc/README.md) § "Federal-readiness gap analysis."
|
|
|
|
### Permission model: roles + verbs
|
|
|
|
Five permission verbs gate every read and write:
|
|
|
|
| Verb | Allows |
|
|
|---|---|
|
|
| `r` | read file bytes; list directory |
|
|
| `w` | overwrite an existing file; rename existing file |
|
|
| `c` | create a new file or directory |
|
|
| `d` | delete a file |
|
|
| `a` | modify the ACL of this subtree (write `.zddc`) |
|
|
|
|
`.zddc` files express grants under `acl.permissions: { principal → verb-set }`. A principal containing `@` is an email pattern matched by `MatchesPattern` (existing glob); a bare name is a role looked up against `roles:` definitions, walking the cascade for the closest definition. Empty verb set is an explicit deny.
|
|
|
|
Cascade evaluation walks leaf→root for the first level whose entries match the user; the union of matching verb sets at that level wins. A leaf allow overrides an ancestor explicit-deny — that's the load-bearing delegation primitive that lets a subtree owner grant access without root-admin involvement. Operators who need the opposite rule (ancestor-deny-absolute, NIST AC-6) deploy OPA with their own Rego (zddc-server ships only the fail-closed read-ACL skeleton at `--print-rego`).
|
|
|
|
The `admins:` field (root or any subtree `.zddc`) confers admin authority over that level and below, but it splits into two powers — see the elevation section below:
|
|
- **Standing config-edit (no elevation):** an admin — or anyone with the `a` verb — may edit the `.zddc`/roles of subtrees they administer. `IsConfigEditor` grants `VerbA` above the WORM clamp; it owns the subtree's policy but cannot write/delete records.
|
|
- **The unconditional `rwcda` + WORM/cascade bypass requires elevation:** `IsActiveAdmin = admin-on-chain AND Elevated` is the single bypass site. Un-elevated, an admin is a config-editor, not a WORM-bypassing superuser.
|
|
|
|
#### Canonical folders, URL routing & the `.zddc` cascade
|
|
|
|
There are **no hardcoded folder names** — the canonical project structure is described by a baked-in baseline `.zddc` (`zddc/internal/zddc/defaults/`), loaded as the bottom layer of the cascade. `zddc-server show-defaults` exports it as a `.zddc.zip`; operators override at the on-disk root (or any deeper level) by mirroring the structure and changing what they need (on-disk wins per field). Setting file-scope `inherit: false` on an on-disk `.zddc` rejects the embedded layer entirely — **including the structural convention (WORM zones, per-user fences, virtual folders)**, not just the default ACLs, so it's a blunt instrument.
|
|
|
|
**Project shape (after the May 2026 reshape).** `archive/` is the only physical project-root directory. Everything party-scoped lives uniformly under `archive/<party>/{ssr.yaml, mdl/, rsk/, received/, issued/, incoming/, working/<email>/, staging/<batch>/, reviewing/<tracking>/}`. Six sibling top-level URLs are **virtual aggregators**, never on disk:
|
|
|
|
- **Row rollups** (tables tool, `default_tool: tables`) — `<project>/ssr`, `<project>/mdl`, `<project>/rsk`. Synthesise one row per party (SSR) or per row file across parties (MDL/RSK), with the source party injected as a synthesised `$party` column. The `$` sigil marks the column system-managed: the tables tool renders it read-only and strips it before submitting a write. Form-mode "+ Add row" on a rollup view prompts for `party` (the routing key, stored in the form schema as a real input field; stripped on write because the folder name *is* the identity).
|
|
- **Folder-nav aggregators** (browse tool, `default_tool: browse`) — `<project>/working`, `<project>/staging`, `<project>/reviewing`. All three are purely virtual (no on-disk presence); they list the parties whose `archive/<party>/<slot>/` has non-empty content (the in-flight filter — empty or .zddc-only slots are suppressed). Per-party URLs `<project>/<slot>/<party>[/<rest>]` 302-redirect to the canonical `<project>/archive/<party>/<slot>[/<rest>]`. Sharing/bookmarks land on the canonical path after the redirect.
|
|
|
|
**Creating in an aggregator → party picker.** Because content is party-scoped, there's nowhere to put a folder/file at `<project>/<slot>/` itself. `EnsureCanonicalAncestors` rejects any physical write under all six aggregator names, and `serveFileMkdir` additionally 409s a mkdir *inside* one (`rejectProjectAggregatorMkdir`) with a message pointing at `archive/<party>/<slot>/` — rather than silently materialising an unreachable shadow folder. The browse "New folder" / "New file" action detects an aggregator root client-side (`events.js: aggregatorRoot`) and opens a **party picker** (`openPartyPicker`, styled like the stage modal): the user chooses an existing party — or "+ New party…", which issues `mkdir archive/<newparty>/…` and is gated to the `document_controller` by the existing `archive/` create ACL (a 403 surfaces a clear message). The chosen folder/file is then created at `archive/<party>/<slot>/<name>`, where the per-party `working/` slot carries its own `history: true` + auto-own convention. Right-clicking a party row shown in an aggregator listing rewrites to the canonical path directly (`rewriteAggregatorPath`), so it never re-prompts. (An earlier iteration made `<project>/working/` materialise project-level creator-owned folders directly; that was dropped in favour of this uniform party-scoped model.)
|
|
|
|
Mkdir at the project root is restricted: only `archive` and `_`/`.`-prefixed system names are accepted (`handler/fileapi.go: rejectProjectRootMkdir`). Any other name — including the six virtual aggregator names, which would shadow the virtual surface — returns 409 Conflict. This is the only structural mkdir guard; deeper paths are governed by `auto_own:` + `worm:` + ACL.
|
|
|
|
Plan Review (`X-ZDDC-Op: plan-review`) hardcodes the scaffold convention: workflow folders always land at `<project>/archive/<party>/{reviewing,staging}/<tracking>/`, derived from the originating submittal's path. The pre-reshape `on_plan_review.reviewing_root` / `staging_root` cascade keys were dropped — one convention, no per-project override surface. The `X-ZDDC-On-Plan-Review` response header (set by `directory.go`) lights up on every `/<project>/archive/<party>/received/<tracking>/` URL via the structural `zddc.IsPlanReviewURL` test, so the browse client knows when to show the menu item without re-implementing the cascade.
|
|
|
|
The schema keys that drive built-in behavior:
|
|
|
|
| Key | Effect | Cascade rule |
|
|
|---|---|---|
|
|
| `default_tool` | tool served at `<dir>` (no trailing slash) — the "specialized app" | leaf→root (parent applies to descendants) |
|
|
| `dir_tool` | tool served at `<dir>/` (trailing slash) — the directory view; floors at `browse` | leaf→root |
|
|
| `auto_own` / `auto_own_fenced` | mkdir here writes a creator-owned `.zddc` (`<email>: rwcda`); fenced adds `acl.inherit:false` (private) | leaf-only |
|
|
| `virtual` | never materialise on disk; requests are virtual routes (`reviewing/`, `mdl`) | leaf-only |
|
|
| `drop_target` | browse shows a drag-drop upload overlay (surfaced via `X-ZDDC-Drop-Target`) | leaf-only |
|
|
| `worm` | list of principals — see WORM below | union across cascade (no reset) |
|
|
| `available_tools` | tools the server may auto-serve / browse may offer here | union leaf→root |
|
|
| `admins` | subtree-admin principals (email globs or role names) | concat-dedupe across cascade |
|
|
| `roles` | `{ name → { members:[], reset:bool } }` | members union across cascade; `reset:true` stops the walk |
|
|
| `paths` | recursive map of child-path → `.zddc` overlay; the engine of the whole convention | replaces (the walker threads ancestor `paths:` to the right level) |
|
|
|
|
**Slash / no-slash URL routing.** Every directory URL has two forms: `<dir>/` serves `dir_tool` (defaults to `browse`, the file-tree navigator) and `<dir>` serves `default_tool` (the specialized app — `archive` at `archive/`, `transmittal` at `archive/<party>/staging/`, `browse` at `archive/<party>/{working,reviewing}/`, `tables` at `archive/<party>/{mdl,rsk}` and at the project-level `ssr/mdl/rsk` rollups). A folder with no `default_tool` 302s the no-slash form to the slash form, so you land on `dir_tool`. JSON listing requests ignore both keys — the raw listing is always served, so the browse SPA can enumerate entries regardless. The dispatcher's `serveSpecializedNoSlash` (in `cmd/zddc-server/main.go`) is the single chokepoint for the no-slash side; `handler.ServeDirectory` (via `zddc.DirToolAt`) handles the slash side.
|
|
|
|
**Zip-backed directories.** A `.zip` file is also a navigable directory: `GET …/Foo.zip/` returns a JSON listing of the zip's members (or the browse SPA for an HTML request) and `GET …/Foo.zip/sub/doc.pdf` extracts and streams that one member — so a client navigating a zipped transmittal folder never downloads the whole archive. `GET …/Foo.zip` (no trailing slash) is unchanged: the raw `.zip` download. Read-only: `PUT`/`DELETE`/`POST` to a path inside a `.zip` is rejected (405). ACL is the chain of the directory *containing* the zip — a zip carries no `.zddc` of its own, the same model as the `.archive` virtual surface. Implemented by `internal/zipfs` + `handler.ServeZip`, routed via `splitZipPath` in the dispatcher (before the file-API branch). Offline tools (archive's scanner, browse's tree) get the same capability client-side via `shared/zip-source.js` — a `ZipDirectoryHandle`/`ZipFileHandle` pair over JSZip that mimics the File-System-Access surface. The archive tool treats a `.zip` whose name minus `.zip` parses as a transmittal-folder name as that transmittal folder; the browse tool expands *any* `.zip`.
|
|
|
|
**Subtree download.** `GET /some/dir/?zip=1` (the query form works on both `/dir` and `/dir/`) streams an `application/zip` of every readable file under that directory, recursively — `Content-Disposition: attachment; filename="<dir>.zip"`. It's `handler.ServeSubtreeZip`: a `filepath.WalkDir` that ACL-gates each file by the `.zddc` chain of its containing directory (the same per-directory decision cache `serveArchiveListing` uses), skips hidden entries (`.`/`_`-prefixed: `.zddc`, `_template`, `_app`), and adds any `.zip` *file* it meets as opaque bytes (it does **not** recurse into it — that's the navigable-surface above, a different feature). The response is streamed straight onto the `ResponseWriter` (`zip.Store` for already-compressed extensions, `zip.Deflate` otherwise), so a fully-ACL-denied or empty subtree yields a valid empty zip rather than a 403 (a stream can't change status after the headers go out). The browse tool's toolbar **Download (zip)** button hits this for the directory in view in server mode; offline (file://) it walks the picked folder itself with JSZip (with a `confirm()` above ~2000 files / ~500 MB, since the whole tree is buffered in browser memory).
|
|
|
|
**WORM** (write-once-read-many). A `worm: [principal...]` list on a `.zddc` marks that path (and descendants) immutable: `w`/`d` are stripped for everyone non-admin; `c` survives only for the listed principals (who get read + write-once-create); `r` for outsiders is whatever the normal ACL granted (the worm list doesn't itself confer read). Two carve-outs: an **elevated** admin (root / subtree) bypasses the clamp entirely — the escape hatch for mis-filed documents — and a **standing** config-editor keeps `a` (so a subtree admin can edit the `.zddc` that *governs* a WORM zone without elevating; that grants only config mutation, never record write/delete). the embedded default tree puts `worm: [document_controller]` on `archive/<party>/{received,issued}`, so the canonical immutable-archive convention is unchanged; the difference is an operator can mark any path WORM, or rename `received`/`issued`, without a code change.
|
|
|
|
**Standard roles.** the embedded default tree references three roles (all shipped empty — a fresh deployment grants nothing until an operator populates them):
|
|
|
|
- `document_controller` — read/write across a project, `rwc` at `archive/`. When a DC mkdir's `archive/<party>/`, the auto-own `.zddc` grants both their email AND the `document_controller` role `rwcda` at that party (via `auto_own_roles: [document_controller]` in the defaults) — so any peer DC has full authority at every party without needing subtree-admin status. Explicit `rwcd` at `incoming/` and `staging/` shadows the inherited `rwcda` to make the transfer-workflow's `d` requirement obvious. WORM-create principal in `received/issued` via the `worm:` list. NOT a subtree-admin anywhere — admin elevation is reserved for the root `admins:` list (the human escape hatch). Plan-Review approval is part of this role; there is no separate `approver` — two-person sign-off, when needed, is expressed via per-folder `.zddc` overrides rather than baked-in roles.
|
|
- `project_team` — read-only across the project; their own `archive/<party>/working/<email>/` home and anything they create under `incoming/` get a creator-owned auto-own `.zddc` that wins via deepest-match, so "read-only except what I own" falls out of the cascade with no special rule.
|
|
- `observer` — pure read-only across the project. Distinct from `project_team` in that the role itself carries no `c` anywhere, so an observer can't bring a working home into existence under auto-own. Intended for auditors, regulators, and external read-only viewers who must not contribute content.
|
|
|
|
The role invariants (verb sets at each canonical path, subtree-admin scope) are locked down in `zddc/internal/zddc/standardroles_test.go`. New roles, when added, should ship with a parallel test in that file.
|
|
|
|
#### Client-side capability gating
|
|
|
|
Three server surfaces feed the front-end's hide/disable model:
|
|
|
|
- **Per-entry `verbs` on every directory listing item** (`zddc/internal/listing/types.go`). Canonical `"rwcda"` subset granted to the calling principal at that entry's URL. For files it reflects the parent dir's chain (matches Writable's gate); for directories it reflects the subdir's OWN chain. `Writable` stays in lockstep during the transition window; new clients should read `verbs` and let `writable` wither.
|
|
- **`GET /.profile/access?path=<urlpath>`** returns the global view (Email, IsSuperAdmin, CanElevate, …) plus three path-scoped fields: `path_verbs` (verbs at the requested path under the caller's CURRENT elevation), `path_is_admin` (subtree-admin authority at that path under current elevation), and `path_can_elevate_grant` (verbs the caller WOULD hold at that path if they elevated, empty when elevation wouldn't change anything). Each tool fetches its current directory once on load to gate top-of-page affordances.
|
|
- **403 ACL-deny responses carry a JSON body** `{"error": "Forbidden", "missing_verb": "<r|w|c|d|a>"}` (`zddc/internal/handler/errors.go writeForbidden`). Other 403 conditions (no authenticated principal, existence-leak guards) keep plain-text bodies — `missing_verb` only applies to ACL denies.
|
|
|
|
Client side, `shared/cap.js` consumes all three: `zddc.cap.has(node, verb)` reads the listing's verbs string (falling back to `node.writable` for `w` on offline FS-API listings); `zddc.cap.at(path)` memo-fetches the path-scoped profile view; `zddc.cap.handleForbidden(resp, opts)` renders an error toast naming the missing verb and offers an Elevate button when `path_can_elevate_grant` covers it.
|
|
|
|
Each tool gates per the hide/disable rules:
|
|
- **Hide** admin-only actions (`a`), WORM-zone destructive items, and flow-terminal steps (Publish, advance state) when the verb is unattainable.
|
|
- **Disable + tooltip** everyday write affordances (Rename/Delete in the context menu, Save in editors, `+ Add row`, `+ New folder`, `Submit`) so the user discovers what permission is missing and can elevate if applicable.
|
|
- **Optimistic** for bulk / cross-directory operations — let the server return 403 and surface it via `cap.handleForbidden`.
|
|
|
|
Browse implements the per-entry gating (rename/delete + editor save); tables and form pre-flight their primary writes via `cap.at` + route 403s through `cap.handleForbidden`. Transmittal and classifier write through the FS Access API rather than the server, so server-side gating doesn't apply to their UI controls.
|
|
|
|
### File API (authenticated CRUD)
|
|
|
|
zddc-server exposes write methods on the same URL space as GET. Each method maps to a specific verb and is gated against the cascade-derived verb set:
|
|
|
|
| Method | URL | Headers | Action verb | Status |
|
|
|---|---|---|---|---|
|
|
| `PUT` | `/<new-path>` | `If-Match: "<etag>"` (optional) | `c` | 201 created |
|
|
| `PUT` | `/<existing-path>` | `If-Match: "<etag>"` (optional) | `w` | 200 overwritten |
|
|
| `PUT` | `/<dir>/.zddc` | — | `a` | 200/201 |
|
|
| `DELETE` | `/<path>` | `If-Match: "<etag>"` (optional) | `d` | 204 |
|
|
| `POST` | `/<path>` | `X-ZDDC-Op: move` + `X-ZDDC-Destination: /new/path` | `w` (src) + `c` (dst) | 200 |
|
|
| `POST` | `/<path>/` | `X-ZDDC-Op: mkdir` | `c` | 201 created / 200 idempotent |
|
|
|
|
Writes use `WriteAtomic` (temp file → fsync → rename) for partial-write safety. Move uses `os.Rename` for same-FS atomicity. Body size capped by `--max-write-bytes` (default 256 MiB). Reserved hidden segments (`.`-prefixed, `_app`, `_template`) are 404'd uniformly with the read path. Every write logs a structured `file_write` event (op, path, email, status, bytes) into the same audit stream as access logs.
|
|
|
|
Browser clients reach the API through `shared/zddc-source.js` — an FS Access API polyfill (`HttpDirectoryHandle`, `HttpFileHandle`) that lets tools written against `showDirectoryPicker()` work unchanged when served by zddc-server. classifier, transmittal, and browse auto-detect HTTP mode at startup, build a polyfill handle for `location.pathname`'s directory, and skip the file picker entirely. A 403 on the initial listing surfaces a "no permission to list this directory" message instead of the welcome screen.
|
|
|
|
#### `zddc-source.js` known gaps
|
|
|
|
The polyfill covers the FS Access surface tools actually use. A few corners are intentionally unimplemented — note them when adding new tool features:
|
|
|
|
- **Recursive directory removal is not implemented.** `HttpDirectoryHandle.removeEntry(name, { recursive: true })` is a no-op against the server because there is no recursive-DELETE endpoint. Tools that rename a non-empty directory by copy + remove (the FS-Access idiom) will leave the source directory orphaned in HTTP mode. Detect this case and either guard the operation or implement server-side `POST X-ZDDC-Op: move` for the directory.
|
|
- **Writes have no truncate semantics.** Each PUT replaces the whole file. There's no `FileSystemWritableFileStream.truncate(size)` analogue; partial-write support means partial-overwrite-via-streaming is the polyfill's only write path.
|
|
- **Directory listings are not cached on the client side.** Cache mode does cache file responses (and persists `.zddc-listing.<json|html>` sidecars on the *server* side), but the polyfill itself re-fetches `?json=1` listings on every traversal. Tools that re-enter the same directory many times in quick succession should cache results in tool state.
|
|
|
|
These are deliberate scope decisions, not bugs. Lift any of them only when a concrete tool feature pays for the implementation cost.
|
|
|
|
### Why the tool-rooted view matters for third-party containment
|
|
|
|
A vendor given access to `/Archive/Acme/Incoming/` lands at the archive tool
|
|
served at that URL, with that subtree as its world. There is no breadcrumb to
|
|
`/Archive/`, no "go to root" button, no listing of sibling vendor folders.
|
|
This is **UX containment** — it makes the deployment feel self-contained and
|
|
prevents accidental discovery of out-of-scope paths. It is **not** the
|
|
security mechanism. Even if a vendor hand-crafts a URL to `/Archive/` or
|
|
`/`, the cascade's default-deny rejects them before any byte is served. The
|
|
tool-rooted view exists so a vendor reading their email link doesn't have to
|
|
choose between "click something I shouldn't" and "click nothing"; the
|
|
cascade exists so that choice never matters.
|
|
|
|
This pairing — UI affordance contained, ACL enforced — is why the server
|
|
auto-serves `archive.html` at *every* directory under `ZDDC_ROOT`. If the
|
|
archive tool only worked at root, every locked-down subtree would need
|
|
either a hand-crafted entrypoint or a tool that knew how to scope itself.
|
|
Auto-serving makes "vendor's world starts at `/Archive/<their-name>/`" the
|
|
out-of-the-box behavior with no per-deployment configuration.
|
|
|
|
---
|
|
|
|
## CSS Architecture
|
|
|
|
All tools use vanilla CSS. No frameworks at build time.
|
|
|
|
**Common conventions:**
|
|
|
|
- CSS variables for theme colors and spacing in `base.css`
|
|
- Component-scoped class names
|
|
- `.hidden` class uses `display: none !important` for JavaScript show/hide
|
|
- Print styles in a separate `print.css`
|
|
|
|
---
|
|
|
|
## Testing
|
|
|
|
Tests use Playwright with Chromium only (File System Access API requires it).
|
|
|
|
### Running Tests
|
|
|
|
```bash
|
|
npm test # all tools
|
|
npx playwright test archive # single tool
|
|
npx playwright test --debug # debug mode
|
|
```
|
|
|
|
### Test Structure
|
|
|
|
Each tool has a spec file in `tests/`:
|
|
|
|
```
|
|
tests/
|
|
archive.spec.js ← 2 tests: load + directory scan
|
|
classifier.spec.js ← 2 tests: load + store injection
|
|
browse.spec.js ← load + file tree render + markdown editor mount
|
|
transmittal.spec.js ← 2 tests: paste round-trip + filesystem round-trip
|
|
fixtures/
|
|
mock-fs-api.js ← Reusable File System Access API mock
|
|
transmittal-data.js
|
|
zddc-filenames.js
|
|
```
|
|
|
|
### Mock File System API
|
|
|
|
`MOCK_FS_INIT_SCRIPT` (from `tests/fixtures/mock-fs-api.js`) overrides `showDirectoryPicker`, `showOpenFilePicker`, and `showSaveFilePicker`. Inject it via `page.addInitScript` before navigating.
|
|
|
|
```javascript
|
|
// Flat directory
|
|
window.__setMockDirectory('name', [{ name: 'file.pdf', content: '...', size: 100 }]);
|
|
|
|
// Nested directory tree
|
|
window.__setMockDirectoryTree('name', {
|
|
'subfolder': { 'file.pdf': 'content' },
|
|
'root-file.md': 'content',
|
|
});
|
|
```
|
|
|
|
### Writing Tests
|
|
|
|
Follow the pattern in `tests/transmittal.spec.js`:
|
|
|
|
- Use ESM `import` syntax
|
|
- Inject `MOCK_FS_INIT_SCRIPT` in `test.beforeEach` for any test that navigates to a tool page
|
|
- Use `waitUntil: 'domcontentloaded'` or `'load'` (not `'networkidle'` — the bundled scripts may keep the network active)
|
|
- Prefer `page.waitForFunction` over `page.waitForSelector` for app-state readiness
|
|
- Assert through the store/module API for tests that don't need visible DOM
|
|
|
|
---
|
|
|
|
## Code Standards
|
|
|
|
| Rule | Rationale |
|
|
|------|-----------|
|
|
| No `</script>` or any `</tag>` in JS string literals | Breaks inline HTML embedding — escape with `'<' + '/tag>'` or use `<\/` in `sed` at build time |
|
|
| No external dependencies at runtime | Self-contained output requirement |
|
|
| No TypeScript, no bundlers | Keeps the build system auditable and simple |
|
|
| Only `window.app` and `window.zddc` are global | Keeps the global namespace clean; expose only what's needed for debugging |
|
|
| Defensive input validation | File System API handles and user-pasted data are untrusted |
|
|
| Update README.md when features ship | Documentation parity is a delivery requirement, not optional |
|
|
|
|
---
|
|
|
|
## Git Workflow
|
|
|
|
**Branching:** short-lived feature branches (`feature/<name>`, `bugfix/<name>`, `hotfix/<name>`), squash-merged to `main` and immediately deleted. Quick fixes (typos, one-liners) go direct to `main`.
|
|
|
|
**Commit messages:** Conventional Commits — `<type>(<scope>): <description>`. Types: `feat`, `fix`, `docs`, `style`, `refactor`, `perf`, `test`, `chore`. See `AGENTS.md` for the full table and examples.
|
|
|
|
**Releases:** Tag the commit after confirming `dist/` is current. Format: `{project}-v{version}` (e.g. `archive-v1.0.0`). Semantic versioning applies. There is no CI/CD — the built `.html` file is already committed to the repo.
|
|
|
|
```bash
|
|
bash tool/build.sh # rebuild dist/
|
|
git add -f tool/dist/tool.html # stage if needed
|
|
git commit -m "chore(tool): rebuild for vX.Y.Z"
|
|
git tag tool-vX.Y.Z
|
|
git push origin main --tags
|
|
|
|
git tag -l "archive-v*" # list releases
|
|
git push origin :refs/tags/tag-name # delete a remote tag
|
|
```
|
|
|
|
---
|
|
|
|
## Adding a New Tool
|
|
|
|
1. Create `tool/` with the standard directory layout
|
|
2. Write `template.html` with `{{CSS_PLACEHOLDER}}` and `{{JS_PLACEHOLDER}}` markers
|
|
3. Write `tool/build.sh` following the pattern of an existing tool
|
|
4. Add `bash "$SCRIPT_DIR/tool/build.sh"` to the root `build.sh`
|
|
5. Add a test project entry to `playwright.config.js`
|
|
6. Create a stub `tests/tool.spec.js`
|
|
7. Force-add the dist output: `git add -f tool/dist/tool.html`
|
|
|
|
If the tool requires vendor dependencies, download them to `tool/vendor/`, add them to `.gitignore` exclusions if appropriate, and update `build.sh` to inline them (with the `</` escaping step).
|