ZDDC/zddc/internal/zddc/file.go
ZDDC 882d5e4c86 feat(zddc-server): server-stamped audit + history for record YAMLs
Adds cascade-driven schema + immutable audit history for the three table-style
record stores (mdl, rsk, ssr). Two new .zddc top-level keys carry the rules:

- field_codes: discriminated-union vocabulary (kind: enum|pattern|free) for
  the components used to compose tracking-number filenames and constrain
  record bodies. Map-merge across the cascade, mirror of apps: semantics.
- records: per-pattern rules (filename_format, field_defaults, locked,
  row_field, row_scope_fields). Filename-pattern scoping lets the SSR rule
  live at the party-folder level without bleeding onto mdl/rsk siblings.

PUTs to record YAML files route through a new WriteWithHistory orchestrator
(internal/handler/history.go) which:
- strips six client-supplied audit fields (created_at/by, updated_at/by,
  revision, previous_sha) so the client can't forge them
- validates body values against the cascade-resolved field_codes
- enforces filename_format composition (URL basename must match body fields)
- checks locked: defaults (422 mismatch)
- archives prior bytes to <dir>/.history/<base>/<RFC3339Nano>-<sha8>.<ext>
- stamps server-managed audit fields and writes the live file

History-before-live ordering preserves the prior version even on mid-write
crash. previous_sha forms a hash chain across revisions for tamper evidence.

The embedded defaults.zddc.yaml now declares records: entries for mdl, rsk,
and ssr.yaml. RSK rows carry the table-tracking components + row sequence
(filename = <table-tracking>-<row>); MDL rows compose to their own
tracking number; SSR records' identity is the party folder name.

GET <record>.yaml?history=1 returns a JSON list of prior revisions, ACL
gated identically to the live record. dot-segment rejection in
resolveTargetPath protects .history/ from direct client writes.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-19 09:48:58 -05:00

443 lines
22 KiB
Go

package zddc
import (
"os"
"gopkg.in/yaml.v3"
)
// ACLRules holds the access-control rules at one cascade level.
//
// One input form, keyed by principal (Permissions):
//
// - acl.permissions: { principal → verb-set }. Principal is an email
// pattern (contains "@") or a role name (no "@"); roles are looked
// up via ZddcFile.Roles in this file or any ancestor. Verb-set is
// a string drawn from {r,w,c,d,a}; empty string is an explicit
// deny.
//
// Inherit controls whether this level imports grants and roles from
// its ancestors. The default (when the field is absent — represented
// here as a nil pointer) is "inherit normally." Setting `inherit: false`
// makes this level a fence: grants and roles defined in any ancestor
// .zddc are invisible at-and-below this point in the cascade. Useful
// for "complete reset, then add back specific principals" patterns
// (e.g. a vendor folder where only the vendor and the doc controller
// should have access regardless of broader project-level grants).
//
// Federal deployments running the bundled `access_federal.rego` get
// parent-deny-is-absolute / NIST AC-6 semantics; the directive's
// fence-style "reset" should be avoided there because it would let a
// leaf widen access an ancestor refused. The cascade tracer at
// /.profile/effective-policy reports `chain.visible_start` so an
// operator can verify which level a fence is actually cutting off.
//
// Inherit is per-level and not itself cascading: an ancestor's
// `inherit: false` does not transitively block descendants from
// adding their own grants — it only fences off ANCESTORS of the
// fenced level from the descendant subtree.
//
// JSON tags are present so this type round-trips cleanly when included
// in the external-OPA input body (see internal/policy). The canonical
// in-repo serialization is YAML; JSON is only used for OPA queries.
type ACLRules struct {
Permissions map[string]string `yaml:"permissions,omitempty" json:"permissions,omitempty"`
// Inherit *bool: nil = unset (inherit normally), &true = same,
// &false = fence ancestors. Using a pointer so the default is
// distinguishable from an explicit `inherit: true`.
Inherit *bool `yaml:"inherit,omitempty" json:"inherit,omitempty"`
}
// InheritsAncestors reports whether this level imports grants and
// roles from ancestors. True when Inherit is unset or explicitly true;
// false only when explicitly set to false.
func (r ACLRules) InheritsAncestors() bool {
return r.Inherit == nil || *r.Inherit
}
// Role is the named principal-grouping primitive. Members are email
// patterns (see MatchesPattern). A role defined at level L is in scope
// at L and all descendants.
//
// Role membership UNIONS across the cascade: if the same role name is
// defined at multiple levels, the effective member set is the union
// of all those definitions. So a deeper .zddc that lists one extra
// member ADDS it to the inherited role rather than replacing the
// whole list.
//
// Reset breaks the union: when true, this level's definition is
// authoritative for the role — ancestor (shallower) definitions are
// ignored. Descendants that also define the role (without reset)
// still union on top. Use reset to start a role's membership fresh at
// a subtree boundary (e.g. a project that wants its own project_team
// independent of the deployment-wide default).
type Role struct {
Members []string `yaml:"members,omitempty" json:"members,omitempty"`
Reset bool `yaml:"reset,omitempty" json:"reset,omitempty"`
}
// OnPlanReviewConfig is the cascade block enabling the doc-controller
// "Plan Review" composite endpoint. ReviewingRoot and StagingRoot are
// paths relative to the master root (e.g. "<project>/reviewing/" or
// "archive/<project>/reviewing/"). Both must be non-empty for the
// feature to enable; either being empty disables Plan Review for this
// subtree (the right-click menu item hides client-side via
// /.profile/access exposure of this config).
type OnPlanReviewConfig struct {
ReviewingRoot string `yaml:"reviewing_root,omitempty" json:"reviewing_root,omitempty"`
StagingRoot string `yaml:"staging_root,omitempty" json:"staging_root,omitempty"`
}
// ConvertMetadata supplies per-project template variables for the
// server-side MD→{docx,html,pdf} conversion endpoint. The handler
// resolves the effective set by walking the .zddc cascade leaf→root
// with per-key latest-wins (an empty deeper value does NOT clear an
// ancestor value — operators write the explicit string they want).
//
// Variables are passed to pandoc as -V key=value flags and consumed by
// pandoc/viewer-template.html's $if(client)$ / $if(project)$ /
// $if(contractor)$ / $if(project_number)$ blocks.
type ConvertMetadata struct {
Client string `yaml:"client,omitempty" json:"client,omitempty"`
Project string `yaml:"project,omitempty" json:"project,omitempty"`
Contractor string `yaml:"contractor,omitempty" json:"contractor,omitempty"`
ProjectNumber string `yaml:"project_number,omitempty" json:"project_number,omitempty"`
}
// ZddcFile represents the parsed contents of a .zddc configuration file.
//
// Admins is honored only in the root .zddc file (<ZDDC_ROOT>/.zddc); subdir
// .zddc files have their Admins entry ignored by IsAdmin so that someone who
// can write into a subtree cannot grant themselves admin access. ACL on the
// other hand cascades — see EffectivePolicy / AllowedWithChain.
//
// Title is read only from per-project .zddc files (the file directly inside
// each project root) by ServeProjectList; it surfaces a human-readable name
// for the project on the landing-page picker. Optional — projects without a
// title fall back to displaying the directory name.
//
// Apps is a per-directory cascade override mapping app name → source spec.
// The spec is one of: "stable" / "beta" / "alpha" (channel on the canonical
// upstream), "v0.0.4" / "v0.0" / "v0" (version pin on the canonical
// upstream), an absolute "https://..." URL (custom mirror), or a relative
// or absolute filesystem path (./local.html, /opt/zddc/foo.html).
//
// On a request for a tool HTML, zddc-server walks .zddc files leaf→root
// looking for an Apps entry; first match wins. With no entry anywhere, the
// server serves the version baked into the binary at compile time (//go:embed).
// Fetched URL sources are cached in <ZDDC_ROOT>/_app/; the cache is fetch-once
// and never re-validates — operators delete the file to force a refetch.
//
// AppsPubKey is the inline PEM of the Ed25519 public key used to verify
// signatures on URL-fetched apps artifacts. Honored only at the root
// .zddc file (same root-only treatment as Admins, for the same reason:
// it's a trust anchor; subtree write authority must not be able to
// re-anchor it). Lower priority than --apps-pubkey / ZDDC_APPS_PUBKEY:
// when both are set, the env/flag (file path) wins. Empty in either
// place = URL-fetched apps refused (only embedded + local-path apps
// work). See zddc-server's setupApps.
type ZddcFile struct {
ACL ACLRules `yaml:"acl" json:"acl"`
Admins []string `yaml:"admins" json:"admins,omitempty"`
Title string `yaml:"title" json:"title,omitempty"`
Apps map[string]string `yaml:"apps,omitempty" json:"apps,omitempty"`
AppsPubKey string `yaml:"apps_pubkey,omitempty" json:"apps_pubkey,omitempty"`
// Tables declares directory-of-YAML table views available at this
// directory. The map key becomes the URL stem: tables[MDL] is served
// at <dir>/MDL.table.html. The value is a path (relative to this
// .zddc) to a *.table.yaml spec describing columns and the rows
// directory. There is no upward cascade for tables in v1 — each
// directory that hosts a table declares it directly.
Tables map[string]string `yaml:"tables,omitempty" json:"tables,omitempty"`
// Display maps a child entry's on-disk name to a human-friendly
// label rendered by browse / archive / landing in place of the raw
// folder name. The on-disk name remains canonical (lowercase for
// the project-root folders); only the rendered string changes.
//
// Match is case-insensitive on the key. Example, on Project-3/.zddc:
//
// display:
// archive: "Records"
// working: "In-Progress"
//
// Effect: project-3 listings show "Records" and "In-Progress" in
// the tree, but URLs still resolve at /Project-3/archive/ and
// /Project-3/working/. No upward cascade in v1 — a parent .zddc
// doesn't relabel grand-children. Operators set display: on the
// directory whose entries they want renamed.
Display map[string]string `yaml:"display,omitempty" json:"display,omitempty"`
// Convert supplies template variables for the server-side
// MD→{docx,html,pdf} conversion endpoint (see internal/convert).
// Cascades leaf→root with per-key latest-wins. Pointer-to-struct
// so unset is distinguishable from "explicitly empty" — relevant
// because the cascade merger needs to know whether a deeper .zddc
// is contributing a value or just inheriting.
//
// Filename-derived variables (title, tracking_number, revision,
// status) come from zddc.ParseFilename and are NOT in this struct.
Convert *ConvertMetadata `yaml:"convert,omitempty" json:"convert,omitempty"`
// Roles are named principal groups available at this level and below.
// See Role for member syntax.
Roles map[string]Role `yaml:"roles,omitempty" json:"roles,omitempty"`
// CreatedBy records the email of the user who triggered the .zddc's
// creation via the file API's mkdir post-hook (Incoming/Working/Staging
// only). It is an audit field; the cascade evaluator does not consult
// it. The auto-generated .zddc grants the creator's email directly via
// ACL.Permissions, the same way operators grant access to anyone else.
CreatedBy string `yaml:"created_by,omitempty" json:"created_by,omitempty"`
// Inherit controls whether the cascade walker descends below this
// .zddc into lower (ancestor) levels — including the embedded
// defaults that sit at the bottom. Defaults to true (cascade walks
// to the root + embedded layer).
//
// Set to false on a .zddc to stop the descent: that level becomes
// the bottom of the effective cascade. Use this at the on-disk
// root /.zddc to fully ignore the embedded defaults (the operator
// takes full responsibility for spelling out every rule from
// scratch). Useful at deeper levels too — e.g. a sandbox subtree
// that wants none of the project's policy.
//
// Pointer so an unset value (nil) is distinguishable from explicit
// false. nil == defaults to true.
Inherit *bool `yaml:"inherit,omitempty" json:"inherit,omitempty"`
// DefaultTool is the tool name served at this directory's
// no-slash URL form (e.g. /Project/staging without trailing slash
// → transmittal). Empty means "no default" — the no-slash form
// 302s to the slash form, which serves DirTool (browse by default).
// Cascades through Paths: an ancestor's Paths entry can set
// DefaultTool for a virtual descendant without anyone creating
// that dir. This is the "specialized app" half of the slash/no-
// slash convention; see DirTool for the other half.
DefaultTool string `yaml:"default_tool,omitempty" json:"default_tool,omitempty"`
// DirTool is the tool name served at this directory's TRAILING-
// SLASH URL form (e.g. /Project/working/ → the directory view).
// Empty resolves to "browse" — the file-tree navigator — which is
// the site-wide default. An operator can override it per subtree
// (rare; e.g. a folder whose slash form should render some other
// directory-oriented view). JSON listing requests are unaffected:
// they always get the raw listing regardless of DirTool, so the
// browse SPA (or any client) can still enumerate entries.
// Cascades leaf→root like DefaultTool.
DirTool string `yaml:"dir_tool,omitempty" json:"dir_tool,omitempty"`
// AutoOwn controls whether the file API's mkdir post-hook writes
// an auto-owned .zddc granting the creator rwcda at the new
// directory. Useful for working/staging/incoming-style drafting
// surfaces where the first creator should "own" what they
// created. Empty (nil) inherits via cascade.
AutoOwn *bool `yaml:"auto_own,omitempty" json:"auto_own,omitempty"`
// AutoOwnFenced augments AutoOwn: when true, the generated .zddc
// is written with `inherit: false` so the new directory is
// private to its creator (ancestor ACL grants don't apply). Used
// for per-user home folders under working/<email>/. Default
// (nil/false) writes a non-fenced auto-own .zddc — ancestor
// admin grants still apply.
AutoOwnFenced *bool `yaml:"auto_own_fenced,omitempty" json:"auto_own_fenced,omitempty"`
// Virtual marks a directory as never-materialise-on-disk. The
// server treats requests under such a path as virtual routes
// rather than triggering EnsureCanonicalAncestors. The reviewing
// aggregator is the canonical example. Empty (nil) inherits via
// cascade.
Virtual *bool `yaml:"virtual,omitempty" json:"virtual,omitempty"`
// DropTarget marks this directory as a destination for drag-drop
// uploads in the browse client. The directory listing's response
// header (X-ZDDC-Drop-Target) surfaces this to the SPA, which
// shows the drop-zone overlay only at scopes where the cascade
// permits uploads. Leaf-only — the property describes THIS dir,
// not its descendants. Defaults (nil): no drop zone.
DropTarget *bool `yaml:"drop_target,omitempty" json:"drop_target,omitempty"`
// Worm marks this directory (and its descendants) as
// write-once-read-many. A non-nil Worm list — even an empty one —
// puts the path into a WORM zone with these effects, applied AFTER
// the normal cascade ACL and BEFORE any admin escape hatch:
//
// - write (w) and delete (d) are stripped for everyone
// - create (c) is stripped for everyone EXCEPT the principals
// listed here — they get read + write-once-create ("cr")
// - read (r) for non-listed principals is whatever the normal
// cascade ACL granted (the WORM list does not itself confer
// read to outsiders, only to its own members)
//
// Each entry is an email-glob pattern (or @role:<name> / a bare
// role name). An empty list ([]) is a WORM zone with no
// create-capable principals — the embedded baseline ships this
// on received/ and issued/ with the `document_controller` role
// named but member-empty, so a deployment enables filing simply
// by populating that role. Worm lists UNION across the cascade —
// a deeper .zddc adds more controllers.
//
// Admins (root or subtree) bypass the WORM constraint entirely;
// the handler does the IsAdmin / IsSubtreeAdmin check before
// invoking the policy evaluator. WORM is a normal-user
// constraint, not an absolute one — mis-filed documents still
// need a human escape.
Worm []string `yaml:"worm,omitempty" json:"worm,omitempty"`
// AvailableTools restricts which tools the server will auto-serve
// at this directory and its descendants. The effective list is the
// concat-dedupe union of all AvailableTools across the cascade
// (leaf → root → embedded); a tool not in that union is denied
// auto-route at this path.
//
// Empty list at every level means "no tools available" (effectively
// blocks all auto-serving); the embedded defaults seed the
// universal baseline of archive/browse/landing at root. Operators
// can add tools at deeper levels (working/ adds classifier,
// staging/ adds transmittal + classifier, etc.). browse hosts the
// markdown editor as a plugin so no extra tool is needed under
// working/ or reviewing/.
//
// This does NOT gate explicit static files: an on-disk
// <dir>/transmittal.html is always served. It gates only the
// apps-subsystem auto-route.
AvailableTools []string `yaml:"available_tools,omitempty" json:"available_tools,omitempty"`
// ReceivedPath links a workflow folder (under reviewing/ or staging/)
// back to its canonical submittal in received/. Populated by the
// Plan Review composite endpoint at scaffold time and travels with
// the folder through the reviewing/ → staging/ → issued/ lifecycle.
// The path is relative to the master root (e.g. "archive/Acme/
// received/Acme-0042"), so it survives the workflow folder being
// moved between parents.
//
// When this field is non-empty, the listing handler synthesises
// a virtual `received/` child whose contents come from this path,
// and serveFilePut rewrites writes through that virtual prefix to
// `<workflow>/<base>+C<n><suffix>` comment files in the workflow
// folder itself (the canonical submittal is WORM).
ReceivedPath string `yaml:"received_path,omitempty" json:"received_path,omitempty"`
// PlannedReviewDate / PlannedResponseDate are the doc-controller's
// committed dates for this submittal's review-completion and
// response-issuance, set by Plan Review and stored on the
// CANONICAL submittal's .zddc (received/<tracking>/.zddc) — NOT on
// the workflow folders' .zddc files. The sub-admins (review lead,
// approver) manage ACLs in their respective workflow folders but
// cannot edit these dates, since the cascade does not grant them
// admin authority over received/.
//
// Both fields are ISO date strings (YYYY-MM-DD). Distinct from the
// workflow folder names' date prefixes, which are *forecast* dates
// — mutable via direct folder rename as estimates shift. Folder
// name = live forecast; .zddc planned date = original commitment.
// Comparing the issued/ folder's actual date against these planned
// dates after publish yields planned-vs-actual on-time analysis.
PlannedReviewDate string `yaml:"planned_review_date,omitempty" json:"planned_review_date,omitempty"`
PlannedResponseDate string `yaml:"planned_response_date,omitempty" json:"planned_response_date,omitempty"`
// OnPlanReview is the cascade-declared configuration for the
// "Plan Review" composite endpoint. Empty (nil) means Plan Review
// is not enabled at this subtree — the browse client hides the
// menu item. Set in an ancestor .zddc to enable.
OnPlanReview *OnPlanReviewConfig `yaml:"on_plan_review,omitempty" json:"on_plan_review,omitempty"`
// FieldCodes declares the vocabulary of "field codes" used as
// components of tracking numbers and as constrained body fields
// on record YAMLs (mdl rows, rsk rows, ssr rows). The map key is
// the field name (originator, project, discipline, type,
// sequence, phase, area, suffix, row, …); the value declares the
// allowed shape via a discriminated union — see FieldCode.
//
// Cascade semantics: map-merge per top-level key (child entirely
// replaces parent's entry for the same code). This mirrors the
// Apps: cascade: a sub-tree can narrow or override a single
// field-code's vocabulary without dropping unrelated codes.
//
// Operators define field_codes at the project root (or higher),
// and lower-level .zddc files inherit. Empty in the embedded
// defaults — every deployment populates this per project.
FieldCodes map[string]FieldCode `yaml:"field_codes,omitempty" json:"field_codes,omitempty"`
// Records declares per-record-type rules keyed by filename
// pattern (literal basename like "ssr.yaml" or a glob like
// "*.yaml"). The entry describes the rules that apply to files
// matching that pattern in the directory the .zddc controls.
//
// See RecordRule for the structure: FilenameFormat (composition
// template), FieldDefaults (per-folder defaults), Locked (fields
// that must not be overridden), RowField + RowScopeFields (for
// table-with-rows record types like RSK).
//
// Cascade semantics: map-merge keyed by pattern; each RecordRule
// merges per-field (scalars overwrite, FieldDefaults map-merge,
// Locked concat-dedupe — see mergeRecordRule).
//
// Filename-pattern scoping is what lets SSR rules live at the
// party-folder level without bleeding onto the mdl/ or rsk/
// subfolders in the same folder: the SSR entry keys on
// "ssr.yaml", which only that one file matches.
Records map[string]RecordRule `yaml:"records,omitempty" json:"records,omitempty"`
// Paths declares virtual sub-directory rules without those
// directories needing to exist on disk. Each key is a single path
// segment — either a literal name or `*` (matches any segment).
// The value is a nested ZddcFile that applies at the matching
// child directory.
//
// Recursive: a Paths entry's value may itself have a Paths map,
// matching further-down segments.
//
// Example, for a tree where every project should treat archive/
// as the workflow archive and a party's incoming/ as the
// classifier landing zone — without anyone creating those folders
// on disk:
//
// paths:
// "*": # project name (any)
// paths:
// archive:
// paths:
// "*": # party name
// paths:
// incoming:
// default_tool: classifier
//
// Match: literal-segment key first (case-insensitive); fall back
// to a "*" key if present. Multi-segment keys (e.g. "a/b") are
// NOT supported in v1 — express depth via nested Paths blocks.
//
// Virtual contributions from ancestor Paths are merged into the
// effective ZddcFile at each level by EffectivePolicy. On-disk
// .zddc at the matching directory wins per-field (most specific
// overrides). An inherit:false on any level drops the ancestor
// contributions from that level and below.
Paths map[string]ZddcFile `yaml:"paths,omitempty" json:"paths,omitempty"`
}
// ParseFile reads and parses a .zddc YAML file.
// Returns an empty ZddcFile (no rules) if the file does not exist.
func ParseFile(path string) (ZddcFile, error) {
data, err := os.ReadFile(path)
if os.IsNotExist(err) {
return ZddcFile{}, nil
}
if err != nil {
return ZddcFile{}, err
}
return parseBytes(data)
}
// parseBytes is the shared YAML→ZddcFile path used by ParseFile and
// EmbeddedDefaults. Returns a zero ZddcFile if data is empty.
func parseBytes(data []byte) (ZddcFile, error) {
if len(data) == 0 {
return ZddcFile{}, nil
}
var zf ZddcFile
if err := yaml.Unmarshal(data, &zf); err != nil {
return ZddcFile{}, err
}
return zf, nil
}