diff --git a/AGENTS.md b/AGENTS.md index 8bec6e9..de76885 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -382,11 +382,15 @@ A schema-driven form renderer used to collect structured data into YAML files in - `GET //.yaml.html` — render form pre-filled from `.yaml` - `POST //.yaml.html` — overwrite that submission → 200 -**Storage**: spec at `/form.yaml`, submissions at `/-.yaml` (siblings of the spec). Copying `` elsewhere copies the spec plus every submission together. ACL applies via the existing `.zddc` cascade. +**Storage**: spec at `/form.yaml`. Submission filenames depend on whether the directory has a cascade-declared `records:` rule (see "Records, audit, and history" below): +- **No matching `records:` rule** — submissions land at `/-.yaml` (the legacy date+email scheme; still the path for ad-hoc operator-defined forms). +- **Matching `records:` rule** (mdl/rsk/ssr and operator-declared records) — filename is composed from body fields via the rule's `filename_format`; for rsk-style rules the server also auto-assigns a per-row sequence within the table-scope group. + +Copying `` elsewhere copies the spec plus every submission together. ACL applies via the existing `.zddc` cascade. **Round-trip**: v0 is form-as-truth — submission YAML is regenerated from form state on every save; comments in submissions are not preserved. File-as-truth mode (lossless YAML round-trip via the eemeli/yaml Document API) is a v1 feature, needed for hand-edited files like `.zddc` itself. -**Validator subset** (`zddc/internal/jsonschema/`): `type` (string/number/integer/boolean/array/object), `enum`, `minimum`, `maximum`, `minLength`, `maxLength`, `required`, `additionalProperties: false`, `properties`, `items`, `format` (`date`, `email`). NOT supported in v0: `$ref`, `$defs`, `if/then/else`, `oneOf`/`anyOf`/`allOf`, conditional visibility. The form-spec meta-schema enforces that authors stay in the supported subset. +**Validator subset** (`zddc/internal/jsonschema/`): `type` (string/number/integer/boolean/array/object), `enum`, `minimum`, `maximum`, `minLength`, `maxLength`, `pattern`, `required`, `additionalProperties: false`, `properties`, `items`, `format` (`date`, `email`). Schema also carries three client-facing extensions that survive round-trip but aren't enforced by the validator (the server enforces them via cascade or strip-on-write): `readOnly: true` (UI renders disabled), `x-labels: { code → label }` (paired display text for enum dropdowns). NOT supported in v0: `$ref`, `$defs`, `if/then/else`, `oneOf`/`anyOf`/`allOf`, conditional visibility. The form-spec meta-schema enforces that authors stay in the supported subset. **Renderer subset** (`form/js/`): types listed above, enum (select / `ui:widget: radio`), `format: date|email`, textarea, nested objects, arrays of primitives, arrays of objects with add/remove rows. `ui:show-when` and reorder are v1. @@ -415,14 +419,56 @@ Read/aggregate counterpart to the form system. Renders a directory of YAML row f - **Nested sub-tables** — `/sub-list/table.yaml` is its own self-contained table at `/sub-list/table.html`. Composition, not violation. - **Per-row attachments** — `/.attachments/file.pdf`. Natural sidecar pattern; the row YAML can reference its attachments by relative path. - **Drafts / staging** — `/.drafts/.yaml` (dot-prefix → hidden from listings as well as from the table). -- **Future per-row history** — `/.history//.yaml` if/when version sidecars are added. +- **Per-row history** — `/.history//-.yaml`. Server-managed; one directory per record, one file per archived revision. See "Records, audit, and history" below. **Default-MDL fallback at `archive//mdl/`**: when no `table.yaml` (or `form.yaml`) exists on disk in this exact location, the server serves embedded default bytes. The `mdl/` directory itself doesn't even need to exist — the URL renders the default MDL view fully virtually so a fresh archive surfaces the master document list without operator setup. Outside `archive//mdl/`, presence-based discovery is the rule. -**Default-MDL columns mirror the tracking-number components** documented at `zddc.varasys.io/reference.html#tracking-numbers`: `originator`, `phase`, `project`, `area`, `discipline`, `type`, `sequence`, `suffix` — each one a slot of the deliverable's permanent identifier — plus `title`, `plannedRevision`, `plannedDate`, `status`, `owner`. The form schema accepts free-text on every component (no enums or regex constraints) so projects pick their own conventions. Operators customize by dropping their own `table.yaml` + `form.yaml` into `archive//mdl/`; both files override the embedded defaults atomically (no merge — operator-supplied wins entirely). Source: `zddc/internal/handler/default-mdl.{table,form}.yaml`. +**Default-MDL columns mirror the tracking-number components** documented at `zddc.varasys.io/reference.html#tracking-numbers`: `originator`, `phase`, `project`, `area`, `discipline`, `type`, `sequence`, `suffix` — each one a slot of the deliverable's permanent identifier — plus `title`, `plannedRevision`, `plannedDate`, `status`, `owner`. The form schema accepts free-text on every component by default. Projects narrow the vocabulary via the cascade's `field_codes:` (see below) without rewriting the schema — operator overrides at `archive//mdl/{table,form}.yaml` still win atomically over the embedded defaults. Source: `zddc/internal/handler/default-mdl.{table,form}.yaml`. **Adding a new table**: create a directory `/` and drop `table.yaml` (and optionally `form.yaml` for row editing) into it. No code change required. Visit `/table.html`. +## Records, audit, and history + +The "records" subset of the tables system carries three guarantees the generic form/table flow doesn't: server-stamped audit fields, immutable per-record history, and cascade-driven filename composition. The mechanism lives in `zddc/internal/handler/history.go` (`WriteWithHistory`) and `zddc/internal/zddc/field_codes.go`. Three record types ship out of the box: + +| Type | Folder | Filename | Identity carrier | +|---|---|---|---| +| **MDL** (deliverables) | `archive//mdl/` (many siblings) | Composed tracking number, e.g. `ACM-PRJ-EL-SPC-0001.yaml` | Body's component fields | +| **RSK** (risk register) | `archive//rsk/` (many siblings, multiple tables) | `-.yaml`, e.g. `ACM-PRJ-EL-RSK-0001-001.yaml` | Body's components + server-assigned row sequence | +| **SSR** (parties register) | `archive//ssr.yaml` (one per party folder) | Always literal `ssr.yaml` | Parent folder name (existing `name` strip/inject in `ssrhandler.go`) | + +**Two new `.zddc` keys** carry the rules (see `zddc/internal/zddc/file.go` + `field_codes.go`): + +- `field_codes:` — vocabulary for the components used in filename composition and constrained body fields. Each entry is a discriminated union over `kind: enum|pattern|free` (`{kind: enum, codes: {ACM: Acme Inc, …}}` / `{kind: pattern, pattern: "^[0-9]{4}$"}` / `{kind: free, description: "..."}`). Map-merge across the cascade (mirror of `apps:`) — a deeper level can narrow or replace a single code's vocabulary without dropping unrelated codes. +- `records:` — per-pattern rules keyed by filename basename (literal `ssr.yaml` or glob `*.yaml`). Each entry carries `filename_format` (composition template with `{field}` and `{field?}` placeholders), `field_defaults`, `locked`, plus `row_field` + `row_scope_fields` for RSK-style tables-of-rows. Filename-pattern scoping is what lets the SSR rule live at the party-folder level without affecting `mdl/`, `rsk/`, `received/`, etc., siblings. + +Defaults are baked into `defaults.zddc.yaml`; `field_codes:` ships empty (every deployment writes its own vocabulary). + +**Six server-managed audit fields** are injected on every write and stripped from incoming bodies before validation (snake_case to match `.zddc`'s existing `created_by:`): +- `created_at`, `created_by` — stamped on create; preserved untouched on every update +- `updated_at`, `updated_by` — refreshed on every write +- `revision` — `1` on create, `+1` per update +- `previous_sha` — first 8 hex chars of SHA-256 of the prior revision's bytes; absent on create. Forms a hash chain for tamper evidence + +**History layout**: for any record at `/.`, the prior version is archived at `/.history//-.` before the live file is overwritten. Per-record subfolder under `.history/` keeps `readdir` cheap and makes party-folder rename move SSR history along atomically (the dot-folder is inside the party folder, so `os.Rename` carries it). + +**Write ordering**: history first, then live. A crash between the two leaves the prior version safely archived; the retry is idempotent because the history filename is deterministic (timestamp + sha of prior bytes). + +**Strip-and-stamp policy**: clients can't forge audit fields. `WriteWithHistory` strips all six keys from the incoming body BEFORE schema validation runs, then injects authoritative values from request context. A client that sends `created_by: eve@evil` finds it silently overwritten with the request principal. + +**Wire surface**: +- `PUT /.yaml` — routed through `WriteWithHistory` automatically when the basename matches a `records:` rule. Response echoes the stamped YAML as the body (Content-Type: application/yaml) so the tables client can update its row state without a re-GET. +- `GET /.yaml?history=1` — JSON list of prior revisions: `[{revision, ts, by, sha, path}, …]`. ACL gates against the live record (read it → read its history). + +**Record-vs-config distinction**: `WriteWithHistory` fires only for genuine record paths. The gate (`isRecordPath` in `fileapi.go`) excludes `table.yaml`, `form.yaml`, `.zddc`, and the spec naming variants `*.table.yaml` / `*.form.yaml`. Those bypass audit stamping (they're configuration, not data) and go through plain `WriteAtomic`. + +**Operator customization**: +- To narrow a deployment's originator codes: write `field_codes: originator: {kind: enum, codes: {ACM: …, BET: …}}` at the project root `.zddc`. +- To add a new table type: declare a `records:` entry under the appropriate `paths:` level (or a sibling `.zddc` in the folder) with a `filename_format` referencing fields the body carries. +- To inspect a record's revision history: `curl https:///.yaml?history=1 -H 'Authorization: Bearer …'`. + +Source: `zddc/internal/handler/history.go`, `zddc/internal/zddc/field_codes.go`, `zddc/internal/zddc/walker.go`, `zddc/internal/zddc/cascade.go`, `zddc/internal/zddc/defaults.zddc.yaml`. Tests: `zddc/internal/handler/history_test.go`. + ## Implementation-vs-dependency policy Match implementation cost to actual surface used. Reimplement focused subsets when a dep's surface area is much larger than what we consume; adopt for genuinely large specs (YAML parsing, etc.) where reimplementing is foolish. Examples in this codebase: diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index 3a40473..443f2ce 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -474,7 +474,9 @@ app.state.subscribe((property, newValue) => { - `js/post.js` — POST + handle 200/201/422/403/409 responses - `js/main.js` — boot: load context, mount root widget, wire submit -**Server-side counterpart:** `zddc/internal/handler/formhandler.go` recognizes `*.form.html` and `*.yaml.html` URLs, parses the spec, validates submissions via `zddc/internal/jsonschema/`, writes via `zddc.WriteAtomic`. Existence of `.form.yaml` is the trigger; without it, the URL falls through to static-file serving. +**Server-side counterpart:** `zddc/internal/handler/formhandler.go` recognizes `*.form.html` and `*.yaml.html` URLs, parses the spec, validates submissions via `zddc/internal/jsonschema/`, writes via `zddc.WriteAtomic` (plain submissions) or `zddc/internal/handler/history.go` `WriteWithHistory` (record-typed YAML — mdl rows, rsk rows, ssr.yaml). Existence of `.form.yaml` is the trigger; without it, the URL falls through to static-file serving. + +**Record-vs-submission distinction.** "Records" are the three table-store types (mdl/rsk/ssr); everything else is a "submission." Records get server-stamped audit fields (`created_at`/`_by`, `updated_at`/`_by`, `revision`, `previous_sha`), an immutable per-record history at `/.history//-.`, cascade-driven filename composition (via the `records:` + `field_codes:` `.zddc` keys), and per-folder field locking (e.g. type=RSK in rsk/). The mechanism intercepts at the file-API write path (`serveFilePut`): if `isRecordPath` matches, the call routes through `WriteWithHistory`; otherwise the historical `WriteAtomic` path is used. See AGENTS.md "Records, audit, and history" for the operator surface; `zddc/internal/handler/history.go` for the orchestration. **Round-trip philosophy:** v0 is "form-as-truth" — submission YAML is regenerated from form state on every save. Hand-edits to submission files are not preserved across re-edit→re-submit. v1 will add an opt-in "file-as-truth" mode (eemeli/yaml Document API) for forms like `.zddc` itself where users hand-edit and comments must survive.