diff --git a/AGENTS.md b/AGENTS.md index 26581f5..9ff8194 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -289,13 +289,13 @@ No install script. Two paths: - **Local** — download a tool `.html` from `https://zddc.varasys.io/releases/` and open it. Done. - **Server** (`zddc-server`) — every tool is `//go:embed`'d into the binary at compile time (the current-stable build). Which tool a directory URL serves is driven by the `.zddc` cascade, not hardcoded: the baked-in baseline (`zddc/internal/zddc/defaults.zddc.yaml`, dumpable via `zddc-server show-defaults`) declares, via a recursive `paths:` tree, a `default_tool` (the no-slash form: `archive` at `archive/`, `transmittal` at `archive//staging/`, `browse` at `archive//{working,reviewing}/` (browse hosts the markdown editor), `classifier` at `archive//incoming/`, `tables` at `archive//{mdl,rsk}` and at the project-level `ssr/mdl/rsk` virtual rollups, `landing` at the deployment root) and `available_tools` (which tools may be auto-served / offered) per folder. **Project shape (May 2026 reshape):** `archive/` is the only physical project-root directory. Six top-level URLs are virtual aggregators: `ssr/mdl/rsk` (tables row-rollups across parties, with a synthesised `$party` source-party column the tables tool renders read-only and strips before write) and `working/staging/reviewing` (browse folder-nav listings of parties with non-empty content; per-party URLs 302-redirect to `archive///`). Mkdir directly at the project root is restricted to `archive` and `_`/`.`-prefixed system names — virtual aggregator names and ad-hoc folders return 409. The trailing-slash form serves `dir_tool` (defaults to `browse`). See `internal/apps/availability.go` (`DefaultAppAt`, `AppAvailableAt`) and `internal/zddc/lookups.go` (`DefaultToolAt`, `DirToolAt`, `AvailableToolsAt`); the dispatcher chokepoint is `serveSpecializedNoSlash` in `cmd/zddc-server/main.go`. Where the cascade declares no tool, requesting `.html` returns 404 like any other missing file. **The full canonical-folder convention (auto-own, WORM, virtual folders, standard roles) is documented in ARCHITECTURE.md § "Canonical folders, URL routing & the `.zddc` cascade".** -To override at any level, either: +To override a tool's HTML (local-only — no fetch, no channels/versions): 1. Drop a real `.html` file at the path → static handler serves it (highest priority). -2. Write an `apps:` entry in any `.zddc` along the path. Spec is one of `stable` (canonical "latest stable"), `v0.0.4` (exact-version pin), full URL, or local path. Closer-to-leaf entries win. (Or change `default_tool` / `dir_tool` / `available_tools` to route a different tool entirely.) +2. Add an `.html` member to the site bundle `/.zddc.zip` (a local zip read server-side via `internal/zipfs`; overrides that tool everywhere, and lets you add new `.html` tools). To route a *different* tool at a path, change `default_tool` / `dir_tool` / `available_tools`. -URL sources fetch once and cache forever in `/_app//`. To force a re-fetch, delete the cache file. No background refresh, no SHA-256 verification, no admin UI. If a configured URL fetch fails, the server falls back to the embedded copy and emits a one-time WARN log. +Otherwise the embedded build-time copy is served. There is no `apps:` `.zddc` key, no upstream fetch, and no signature verification (all removed). `.zddc.zip` is config, not content: a direct `GET /.zddc.zip` is 404 for everyone; the server reads its members from the filesystem internally. -Operators audit by reading the `X-ZDDC-Source` response header: `fetch:URL` / `cache:URL` / `path:/abs` / `embedded:@`. Direct URL access to `/_app/...` is blocked at the dispatch layer. +Operators audit by reading the `X-ZDDC-Source` response header: `bundle:.html` / `embedded:@` (an on-disk override is served by the static handler with its own headers). **Runtime mode detection** in archive is independent of install: it auto-detects multi-project / project-root / in-archive from `?projects=` plus folder shape. The other tools don't care where they live. @@ -421,7 +421,7 @@ The "records" subset of the tables system carries three guarantees the generic f **Two new `.zddc` keys** carry the rules (see `zddc/internal/zddc/file.go` + `field_codes.go`): -- `field_codes:` — vocabulary for the components used in filename composition and constrained body fields. Each entry is a discriminated union over `kind: enum|pattern|free` (`{kind: enum, codes: {ACM: Acme Inc, …}}` / `{kind: pattern, pattern: "^[0-9]{4}$"}` / `{kind: free, description: "..."}`). Map-merge across the cascade (mirror of `apps:`) — a deeper level can narrow or replace a single code's vocabulary without dropping unrelated codes. +- `field_codes:` — vocabulary for the components used in filename composition and constrained body fields. Each entry is a discriminated union over `kind: enum|pattern|free` (`{kind: enum, codes: {ACM: Acme Inc, …}}` / `{kind: pattern, pattern: "^[0-9]{4}$"}` / `{kind: free, description: "..."}`). Map-merge across the cascade (like `display:`/`tables:`) — a deeper level can narrow or replace a single code's vocabulary without dropping unrelated codes. - `records:` — per-pattern rules keyed by filename basename (literal `ssr.yaml` or glob `*.yaml`). Each entry carries `filename_format` (composition template with `{field}` and `{field?}` placeholders), `field_defaults`, `locked`, `folder_fields`, plus `row_field` + `row_scope_fields` for RSK-style tables-of-rows. Filename-pattern scoping is what lets the SSR rule live at the party-folder level without affecting `mdl/`, `rsk/`, `received/`, etc., siblings. - `folder_fields:` — map of `field → parent-distance` that binds a body field to an ancestor folder name (the folder is the sole source of truth). The map value is how many directories ABOVE the record's own directory the source folder sits (`originator: 1` under `archive//mdl/` resolves to the `` folder). The server overwrites the body field with the derived name before validation + composition (so a client value can never disagree; a mismatched URL still trips the `filename_format` check), and the form renderer marks the field read-only and pre-fills it. @@ -538,7 +538,7 @@ Pick a role per persona: These are NOT interchangeable. A note about which one operators want lives in `cascade.go:13-21` (the `PolicyChain` doc) and the relevant struct fields in `file.go`. -Run `zddc-server show-defaults` to dump the embedded `defaults.zddc.yaml` with annotated comments — that's the full schema with all the cascade keys (`worm:`, `auto_own:`, `drop_target:`, `apps:`, `convert:`, `on_plan_review:`, `records:`, `available_tools:`, `default_tool:`, `dir_tool:`, etc.). +Run `zddc-server show-defaults` to dump the embedded `defaults.zddc.yaml` with annotated comments — that's the full schema with all the cascade keys (`worm:`, `auto_own:`, `drop_target:`, `convert:`, `on_plan_review:`, `records:`, `available_tools:`, `default_tool:`, `dir_tool:`, etc.). ### Build @@ -634,14 +634,13 @@ ZDDC_ROOT=/path/to/your/archive ZDDC_TLS_CERT=none ZDDC_ADDR=:8080 \ | `ZDDC_OPA_URL` | `internal` | Policy decider endpoint. `internal` (default) = in-process Go evaluator (same `.zddc` cascade we always had). `http(s)://...` or `unix:///...` = external OPA — every access decision becomes a `POST /v1/data/zddc/access/allow` to the configured endpoint. Federal customers with their own audited Rego use this; commercial deployments leave it `internal`. | | `ZDDC_OPA_FAIL_OPEN` | *(empty)* | External OPA only. `1` = allow on transport error; default = fail closed (deny). | | `ZDDC_OPA_CACHE_TTL` | `1s` | External OPA only. Per-decision cache TTL — amortizes round-trips on bursty patterns (e.g. `.archive` listings hit the same `(email, dir)` tuple many times). `0` disables. Format is Go `time.ParseDuration`. | -| `ZDDC_APPS_PUBKEY` | *(empty)* | Path to PEM Ed25519 pubkey for verifying signatures on URL-fetched `apps:` artifacts. Empty = URL apps refused. Download from `zddc.varasys.io/pubkey.pem` (canonical channels) or supply your own. No baked-in default — same posture as TLS certs. Alternative inline form: `apps_pubkey:` in root `.zddc` (root-only, env/flag wins). | | `ZDDC_ACCESS_LOG` | `/.zddc.d/logs/access-.log` | JSON-line audit log (lumberjack-rotated, 100 MB / 10 backups / 90 days, gzipped). Server auto-mkdirs the parent. Set explicitly to empty (`--access-log=`) to disable. Per-host filename + `host` field in every record so multi-replica deployments writing to the same `.zddc.d/` dir disambiguate cleanly. | ### URL handling **URLs are case-insensitive.** The dispatcher canonicalizes `r.URL.Path` against on-disk casing before any handler runs (`zddc/internal/fs/resolve.go ResolveCanonical`). Per segment: lowercase variant wins if it exists on disk; otherwise exact-case wins; otherwise readdir+CI scan with the lowercase variant winning the tiebreak when multiple case variants are siblings on disk. Walk stops at the first segment that doesn't exist so virtual prefixes (`.archive`, `.profile`, `.tokens`, `.api`, `.auth`) and 404 paths flow through with their tail preserved verbatim. -**File and folder names preserve case on disk.** The canonicalization is purely a URL→filesystem-name mapping; nothing renames anything. Lowercase is the *project-wide canonical* convention, and auto-created folders in `internal/zddc/ensure.go` (the per-party `archive//{working,staging,reviewing,incoming}/`) and the server's own state dirs (`_app/`, `.zddc.d/tokens/`, `.zddc.d/outbox/`, `.zddc.d/logs/`) are all lowercase by string literal. Operators can drop a `Mixed-Case-Folder/` and it stays mixed-case. +**File and folder names preserve case on disk.** The canonicalization is purely a URL→filesystem-name mapping; nothing renames anything. Lowercase is the *project-wide canonical* convention, and auto-created folders in `internal/zddc/ensure.go` (the per-party `archive//{working,staging,reviewing,incoming}/`) and the server's own state dirs (`.zddc.d/tokens/`, `.zddc.d/outbox/`, `.zddc.d/logs/`) are all lowercase by string literal. Operators can drop a `Mixed-Case-Folder/` and it stays mixed-case. **Audit log captures the as-typed path.** `AccessLogMiddleware` snapshots `r.URL.Path` before dispatch rewrites it; the audit record's `path` field is what the client sent. When canonicalization changed it, a `resolved_path` field is added. diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index a014eff..e353c32 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -83,7 +83,7 @@ Each topic has exactly one authoritative home; everything else links to it. | What ZDDC is + tool channel links + dual-mode (local/server) overview + install snippets | `~/src/zddc-website/index.html` (hand-edited intro for `zddc.varasys.io/`, in the `ZDDC-website` repo) | repo `README.md`, `zddc/README.md` | | File-naming convention spec (status codes, modifiers, folder format) | `~/src/zddc-website/reference.html` | repo `README.md`, in-tool help text | | Versions + channel builds index of every tool | `dist/release-output/index.html` (regenerated by `./build`; deployed to `/srv/zddc/releases/index.html`) | website intro nav, "Browse all versions" link | -| Customer-deployment install (`zddc-server` binary embeds current-stable tools; `.zddc apps:` cascade overrides; cache at `/_app/`) | `zddc/README.md` "Apps: virtual tool HTMLs" section | website intro, `AGENTS.md` | +| Customer-deployment install (`zddc-server` binary embeds current-stable tools; local override via an on-disk `.html` or the site `/.zddc.zip` bundle — no fetch) | `zddc/README.md` "Apps: virtual tool HTMLs" section | website intro, `AGENTS.md` | | zddc-server operations: env vars, ACL syntax, `.archive` URLs, container vs binary | `zddc/README.md` | `AGENTS.md`, website intro | | Build / release / channel commands | `AGENTS.md` | repo `README.md` ("see AGENTS.md") | | Architecture & internal patterns | `ARCHITECTURE.md` (this file) | `AGENTS.md` | @@ -154,13 +154,13 @@ Two orthogonal axes: how the bytes get there (this section), and what runtime mo Resolution order at a request to `/.html` where the app is available: -1. **Override** — real `.html` file at the path → static handler. -2. **`.zddc apps:` cascade** — walk leaf→root for an `apps.` entry. Spec is `stable` (canonical "current stable"), `v0.0.4` (exact-version pin), full URL (custom mirror), or local path. Closer-to-leaf wins. +1. **On-disk override** — real `.html` file at the path → static handler. +2. **Site bundle** — an `.html` member of `/.zddc.zip`, read server-side via `internal/zipfs` (see `internal/apps/bundle.go`). Local file, no fetch, no signature; re-stat'd each request for free hot-reload. 3. **Embedded** — the build-time HTML compiled into the binary. -URL sources fetch once on first request and cache forever in `/_app//`. There is no background refresh, no SHA-256 verification, no admin UI. To pull a new build, delete the cache file. Concurrent misses for the same URL share one outbound fetch (hand-rolled singleflight). Failed fetches fall through to embedded with a one-time WARN log per source URL. Direct URL access to `/_app/...` is blocked at dispatch. +Resolution is LOCAL-ONLY — no network fetch, no signatures, no channels/versions, and no `apps:` `.zddc` key (all removed in favour of this model). `.zddc.zip` is config, not content: a direct `GET /.zddc.zip` is 404 for everyone, while the server reads its members from the filesystem internally. To change a tool's HTML: drop a file at the path, add `.html` to `.zddc.zip`, or rebuild the binary. -The `X-ZDDC-Source` response header always reports what was served: `fetch:URL`, `cache:URL`, `path:/abs`, or `embedded:@`. +The `X-ZDDC-Source` response header always reports what was served: `bundle:.html`, `embedded:@`, or (for an on-disk override) the static handler's own headers. ### Runtime mode detection diff --git a/browse/js/preview-yaml.js b/browse/js/preview-yaml.js index f45979d..95bb540 100644 --- a/browse/js/preview-yaml.js +++ b/browse/js/preview-yaml.js @@ -106,8 +106,6 @@ worm: 'string[]', paths: 'pathmap', display: 'stringmap', - apps: 'appsmap', - apps_pubkey: 'string', tables: 'stringmap', convert: 'convert', created_by: 'string', @@ -225,22 +223,6 @@ walkObject(v, TOP_KEYS, path.concat([seg]), issues); } return; - case 'appsmap': - if (t === 'null') return; - if (t !== 'object') { addTypeErr(path, kind, t, issues); return; } - for (var app in val) { - if (!Object.prototype.hasOwnProperty.call(val, app)) continue; - if (!ALLOWED_TOOLS[app]) { - issues.push({ keyPath: path.concat([app]), severity: 'warning', - message: 'Unknown tool "' + app + '" in apps:.' }); - } - if (typeOf(val[app]) !== 'string') { - issues.push({ keyPath: path.concat([app]), severity: 'error', - message: 'apps.' + app + ' must be a spec string ' - + '(channel | v | URL | path).' }); - } - } - return; case 'rolemap': if (t === 'null') return; if (t !== 'object') { addTypeErr(path, kind, t, issues); return; } diff --git a/zddc/cmd/zddc-server/main.go b/zddc/cmd/zddc-server/main.go index 936124f..f0b977b 100644 --- a/zddc/cmd/zddc-server/main.go +++ b/zddc/cmd/zddc-server/main.go @@ -507,52 +507,12 @@ func newGzipWrapper() (func(http.Handler) http.HandlerFunc, error) { return gzhttp.NewWrapper(gzhttp.MinSize(1024)) } -// setupApps creates the cache + fetcher + server. No seeding, no refresh, -// no admin UI — the server fetches once on first request, caches forever -// in /.zddc.d/apps/, and falls back to the embedded HTML on any failure. +// setupApps builds the tool-HTML server. Resolution is LOCAL-ONLY: a real +// file on disk at the request path (handled upstream by dispatch) → a +// ".html" member of the site-root /.zddc.zip bundle → the +// embedded default. No fetch, no cache, no signatures. func setupApps(cfg config.Config) (*apps.Server, error) { - cache, err := apps.NewCache(filepath.Join(cfg.Root, handler.ReservedSidecar, apps.CacheDirName)) - if err != nil { - return nil, fmt.Errorf("create cache: %w", err) - } - fetcher := apps.NewFetcher(cache, slog.Default()) - - // Apps signing pubkey. Resolution order, highest priority first: - // 1. --apps-pubkey / ZDDC_APPS_PUBKEY (path to PEM file) - // 2. apps_pubkey: inline PEM in the root /.zddc file - // (root-only — same trust-anchor treatment as admins:) - // 3. nothing → URL-fetched apps refuse-by-default; only embedded - // + local-path apps work - // - // Same posture as TLS certificates: zddc-server bakes nothing in. - // Operators using zddc.varasys.io's canonical channels download - // pubkey.pem from there and either configure the path via env/flag - // or paste the PEM contents inline into root .zddc. - switch { - case cfg.AppsPubKey != "": - pub, err := apps.LoadPubKey(cfg.AppsPubKey) - if err != nil { - return nil, fmt.Errorf("apps-pubkey: %w", err) - } - fetcher.VerifyKey = pub - slog.Info("apps signing pubkey loaded", "source", "env/flag", "path", cfg.AppsPubKey) - default: - // Fall back to apps_pubkey: in root .zddc. - rootZddc, err := zddc.ParseFile(filepath.Join(cfg.Root, ".zddc")) - if err == nil && rootZddc.AppsPubKey != "" { - pub, err := apps.ParsePubKeyPEM([]byte(rootZddc.AppsPubKey)) - if err != nil { - return nil, fmt.Errorf("root .zddc apps_pubkey: %w", err) - } - fetcher.VerifyKey = pub - slog.Info("apps signing pubkey loaded", "source", "root .zddc apps_pubkey") - } else { - slog.Warn("apps-pubkey not configured; URL-fetched apps will be refused (only embedded + local-path apps will work). " + - "Set --apps-pubkey, ZDDC_APPS_PUBKEY, or apps_pubkey: in the root .zddc file to a PEM Ed25519 public key you trust.") - } - } - - return apps.NewServer(cfg.Root, cache, fetcher, version), nil + return apps.NewServer(cfg.Root, version), nil } // warnIfNoBootstrap fires a startup slog.Warn when the root .zddc grants @@ -838,6 +798,15 @@ func dispatch(cfg config.Config, idx *archive.Index, ring *handler.LogRing, apps return } + // The site-root config bundle /.zddc.zip is config, not + // browsable content — existence-hidden over HTTP for everyone. The + // server reads its members from the filesystem internally (apps.Bundle) + // to resolve tool HTML, so the 404 here doesn't affect resolution. + if strings.EqualFold(filepath.Base(strings.TrimRight(urlPath, "/")), apps.BundleName) { + http.NotFound(w, r) + return + } + // Raw .zddc YAML view: /.zddc is reachable at every depth // and returns the on-disk file's bytes (Content-Type: application/yaml) // or — when no file exists — a synthetic placeholder body with a diff --git a/zddc/cmd/zddc-server/main_test.go b/zddc/cmd/zddc-server/main_test.go index 7b9743e..53032e2 100644 --- a/zddc/cmd/zddc-server/main_test.go +++ b/zddc/cmd/zddc-server/main_test.go @@ -4,18 +4,14 @@ import ( "archive/zip" "bytes" "context" - "crypto/ed25519" - "crypto/rand" "encoding/json" "net/http" "net/http/httptest" - "net/url" "os" "path/filepath" "strings" "testing" - "codeberg.org/VARASYS/ZDDC/zddc/internal/apps" "codeberg.org/VARASYS/ZDDC/zddc/internal/archive" "codeberg.org/VARASYS/ZDDC/zddc/internal/config" "codeberg.org/VARASYS/ZDDC/zddc/internal/handler" @@ -100,60 +96,22 @@ func TestDispatchReservesZddcD(t *testing.T) { } } -// TestDispatchAppsResolution drives the full apps fetch+cache flow through -// dispatch() with a fake upstream. Confirms that: -// - GET / serves the landing app from the apps subsystem -// - GET /archive.html serves the archive app via fetch+cache -// - second GET /archive.html serves from cache (X-ZDDC-Source: cache:) -// - direct URL access to the reserved cache (/.zddc.d/apps/...) is rejected +// TestDispatchAppsResolution drives local tool-HTML resolution through +// dispatch(): the site .zddc.zip member overrides the embedded default, the +// embedded default is served when no bundle member exists, GET / serves +// landing, the bundle itself is 404 over HTTP, and folder-availability rules +// still gate which tools are served where. func TestDispatchAppsResolution(t *testing.T) { root := t.TempDir() - body := []byte("archive content") - pub, priv, err := ed25519.GenerateKey(rand.Reader) - if err != nil { - t.Fatalf("GenerateKey: %v", err) - } - sig := ed25519.Sign(priv, body) - upstream := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { - // Same body for every artifact; same signature for every .sig - // (since the body is identical across the five tools in this - // fixture). Real deployments publish a distinct .sig per - // artifact; the test only cares that the verify gate passes. - if strings.HasSuffix(r.URL.Path, ".sig") { - _, _ = w.Write(sig) - return - } - w.Header().Set("ETag", `"v1"`) - _, _ = w.Write(body) - })) - defer upstream.Close() - upstreamURL, _ := url.Parse(upstream.URL) - upstreamHost := upstreamURL.Host - if i := strings.Index(upstreamHost, ":"); i >= 0 { - upstreamHost = upstreamHost[:i] - } - - _ = upstreamHost // referenced below - - // Seed root .zddc with subdir-cascade Apps entries pointing at the - // fake upstream. Allow all email patterns (anonymous) so the test - // doesn't have to set up email headers. - zf := zddc.ZddcFile{ - ACL: zddc.ACLRules{Permissions: map[string]string{"*": "rwcd"}}, - Apps: map[string]string{ - "archive": upstream.URL + "/archive_stable.html", - "transmittal": upstream.URL + "/transmittal_stable.html", - "classifier": upstream.URL + "/classifier_stable.html", - "landing": upstream.URL + "/landing_stable.html", - "browse": upstream.URL + "/browse_stable.html", - }, - } + // Allow-all ACL so the test doesn't need email headers. + zf := zddc.ZddcFile{ACL: zddc.ACLRules{Permissions: map[string]string{"*": "rwcd"}}} if err := zddc.WriteFile(root, zf); err != nil { t.Fatalf("WriteFile: %v", err) } - // Create folder convention dirs so classifier/browse/transmittal - // availability rules pass for the test paths used below. + // Site config bundle overriding archive.html. + writeRootBundle(t, root, map[string]string{"archive.html": "BUNDLE archive"}) + // Folder-convention dir so classifier availability passes below. mustMkdir(t, filepath.Join(root, "Project-A", "working", "Acme")) idx, err := archive.BuildIndex(root) @@ -166,47 +124,33 @@ func TestDispatchAppsResolution(t *testing.T) { EmailHeader: "X-Auth-Request-Email", } ring := handler.NewLogRing(10) - appsSrv, err := setupApps(cfg) if err != nil { t.Fatalf("setupApps: %v", err) } - // Override the production embedded public key with the test fixture's - // pubkey so signature verification of upstream.Sign'd bodies succeeds. - appsSrv.Fetcher.VerifyKey = pub - // GET /archive.html → fetched from upstream (archive is available everywhere) + // GET /archive.html → served from the bundle member (overrides embedded). rec := httptest.NewRecorder() - req := httptest.NewRequest(http.MethodGet, "/archive.html", nil) - dispatch(cfg, idx, ring, appsSrv, nil, rec, req) - if rec.Code != http.StatusOK { - t.Fatalf("first /archive.html: status=%d body=%s", rec.Code, rec.Body.String()) + dispatch(cfg, idx, ring, appsSrv, nil, rec, httptest.NewRequest(http.MethodGet, "/archive.html", nil)) + if rec.Code != http.StatusOK || !strings.Contains(rec.Body.String(), "BUNDLE archive") { + t.Fatalf("/archive.html: status=%d body=%s (want bundle override)", rec.Code, rec.Body.String()) } - if rec.Body.String() != string(body) { - t.Errorf("first /archive.html: body mismatch") + if rec.Header().Get("X-ZDDC-Source") != "bundle:archive.html" { + t.Errorf("X-ZDDC-Source=%q, want bundle:archive.html", rec.Header().Get("X-ZDDC-Source")) } - // GET /archive.html again → cache hit (no new upstream fetch) - rec2 := httptest.NewRecorder() - dispatch(cfg, idx, ring, appsSrv, nil, rec2, httptest.NewRequest(http.MethodGet, "/archive.html", nil)) - if rec2.Code != http.StatusOK { - t.Errorf("second /archive.html: status=%d", rec2.Code) - } - - // GET / → landing + // GET / → landing (no bundle member → embedded). rec3 := httptest.NewRecorder() dispatch(cfg, idx, ring, appsSrv, nil, rec3, httptest.NewRequest(http.MethodGet, "/", nil)) if rec3.Code != http.StatusOK { t.Errorf("GET /: status=%d", rec3.Code) } - // The apps cache lives under the reserved sidecar (.zddc.d/apps/); direct - // URL access by a non-admin is 404'd by the sidecar gate, so cached HTML - // can only ever be served through the apps resolver (proper headers/ACL). + // The site bundle is config, not content: a direct GET is 404 for everyone. rec4 := httptest.NewRecorder() - dispatch(cfg, idx, ring, appsSrv, nil, rec4, httptest.NewRequest(http.MethodGet, "/.zddc.d/apps/foo.html", nil)) + dispatch(cfg, idx, ring, appsSrv, nil, rec4, httptest.NewRequest(http.MethodGet, "/.zddc.zip", nil)) if rec4.Code != http.StatusNotFound { - t.Errorf("/.zddc.d/apps/ direct: status=%d, want 404", rec4.Code) + t.Errorf("GET /.zddc.zip: status=%d, want 404", rec4.Code) } // Folder availability rules: classifier should NOT be served at root @@ -277,10 +221,6 @@ func TestDispatchRootAppShellPublicButDataGated(t *testing.T) { } } -// silence "imported and not used" if apps not referenced elsewhere — keep -// import even when we trim test cases later. -var _ = apps.DefaultUpstream - // TestDispatchRoutesWritesToFileAPI verifies dispatch sends PUT/DELETE/POST // to the file API rather than to the read pipeline. func TestDispatchRoutesWritesToFileAPI(t *testing.T) { @@ -1094,3 +1034,26 @@ func TestGzhttpWrapper_CompressesLargeResponses(t *testing.T) { // dot-prefix guard, like any bookkeeping, and surfaced only through the // history endpoints. Raw-block coverage is in TestDispatchHidesDotPrefixedSegments; // the viewer is covered in mdhistory_test.go.) + +// writeRootBundle writes /.zddc.zip containing the given members. +// Used by dispatch tests exercising the local tool-HTML bundle override. +func writeRootBundle(t *testing.T, root string, members map[string]string) { + t.Helper() + var buf bytes.Buffer + zw := zip.NewWriter(&buf) + for name, body := range members { + w, err := zw.Create(name) + if err != nil { + t.Fatalf("zip create %s: %v", name, err) + } + if _, err := w.Write([]byte(body)); err != nil { + t.Fatalf("zip write %s: %v", name, err) + } + } + if err := zw.Close(); err != nil { + t.Fatalf("zip close: %v", err) + } + if err := os.WriteFile(filepath.Join(root, ".zddc.zip"), buf.Bytes(), 0o644); err != nil { + t.Fatalf("write bundle: %v", err) + } +} diff --git a/zddc/internal/apps/apps.go b/zddc/internal/apps/apps.go index 9ab35bf..b1ceca5 100644 --- a/zddc/internal/apps/apps.go +++ b/zddc/internal/apps/apps.go @@ -1,415 +1,22 @@ // Package apps serves the ZDDC tool HTML files (archive, transmittal, -// classifier, landing, browse, form, tables) on virtual paths in the -// file tree. Each tool is "available" only at directories whose name -// matches a folder convention (Incoming/Working/Staging) — see -// availability.go. The markdown editor lives as a plugin inside browse. +// classifier, landing, browse, form, tables) on virtual paths in the file +// tree. Each tool is "available" only at directories whose cascade selects +// it (default_tool / dir_tool / available_tools) — see availability.go and +// the .zddc cascade. The markdown editor lives as a plugin inside browse. // -// Resolution priority for an enabled /.html request: +// Tool HTML resolution is LOCAL-ONLY — no network fetch, no signatures, no +// channels/versions. For an enabled /.html request the bytes come +// from, in precedence: // -// 1. Real file at the request path → static handler (operator override). -// 2. Subdir cascade — walk .zddc files root→leaf, accumulating URL prefix -// and channel/version components from the special `apps.default` key -// and the per-app `apps.` key. Either component can be set, -// overridden, or left to inherit at any level. Path or full-`.html`-URL -// entries are *terminal* — they short-circuit composition and a deeper -// non-terminal entry overrides a parent terminal. -// 3. Embedded fallback — bytes baked into the binary at compile time via -// //go:embed. Used when no `apps:` entry was found anywhere up the chain. +// 1. A real file on disk at the request path → static handler (operator +// override; handled by the dispatcher BEFORE Serve is ever reached, so +// by the time Serve runs no such file exists). +// 2. A member of the site-root config bundle /.zddc.zip, named +// ".html", read server-side via internal/zipfs (see bundle.go). +// 3. The embedded default baked into the binary at compile time via +// //go:embed (see embed.go). // -// Spec forms (each is a string value in `.zddc apps:`): -// -// :stable / :v0.0.4 — channel-only -// stable / v0.0.4 / 0.0.4 — channel-only (no leading colon) -// https://host/path — URL-prefix only (combines with cascade channel) -// https://host/path:stable — URL-prefix + channel (composes) -// https://host/path/file.html — terminal full URL (used as-is) -// ./local.html / /abs/local.html — terminal local path -// -// No background refresh, no SHA-256 verification. To pick up new upstream -// bytes, delete the cache file (or the whole .zddc.d/apps/ tree). +// To change a tool's HTML, drop a file at the path, drop ".html" into +// .zddc.zip, or rebuild the binary. There is no `apps:` .zddc key and no +// upstream fetch — both were removed in favour of this local model. package apps - -import ( - "fmt" - "net/url" - "path/filepath" - "strings" - - "codeberg.org/VARASYS/ZDDC/zddc/internal/zddc" -) - -// DefaultUpstream is where channel and version shorthand specs resolve when -// no `apps.default` URL prefix is configured anywhere up the chain. -const DefaultUpstream = "https://zddc.varasys.io" - -// DefaultUpstreamReleases is the prefix appended to DefaultUpstream when -// composing the canonical upstream URL. -const DefaultUpstreamReleases = DefaultUpstream + "/releases" - -// DefaultChannel is the channel shorthand used when nothing in the chain -// specifies one. -const DefaultChannel = "stable" - -// CacheDirName is the directory under /.zddc.d/ where fetched URL -// sources are cached. Living under the reserved .zddc.d/ sidecar means the -// cache is hidden from listings and admin-gated for direct URL access like all -// other server bookkeeping (see handler.ReservedSidecar); the resolver itself -// reads/writes it via the filesystem, not over HTTP. -const CacheDirName = "apps" - -// DefaultAppsKey is the special key in `apps:` that provides the baseline -// URL prefix and channel for any app not overridden per-name. Cascades -// through .zddc files like everything else. -const DefaultAppsKey = "default" - -// Source is a fully-resolved app source (output of Resolve). -type Source struct { - App string // canonical app name - URL string // upstream URL (mutually exclusive with Path) - Path string // resolved local file path -} - -// IsURL reports whether the source is fetched (vs read from disk). -func (s Source) IsURL() bool { return s.URL != "" } - -// SpecComponents is the parsed shape of a single `.zddc apps:` value. -// Terminal forms (Path or FullURL) are mutually exclusive with the -// composable URLPrefix/Channel forms. Resolve() turns one or more -// SpecComponents (one per applicable level in the cascade) into a final -// Source. -type SpecComponents struct { - // Terminal forms — exactly one set means the spec is terminal and - // short-circuits composition. - Path string // local file path (resolved + bounded to ZDDC_ROOT) - FullURL string // full URL ending in `.html` (used as-is) - - // Composable forms — either or both may be set, both may be empty - // (caller should treat empty-everything as a no-op). - URLPrefix string // "https://host/path" (no trailing /) - Channel string // "stable" (latest), "v0.0.4" (exact version pin) -} - -// IsTerminal reports whether this spec terminates composition. -func (c SpecComponents) IsTerminal() bool { - return c.Path != "" || c.FullURL != "" -} - -// IsEmpty reports whether the spec contributes nothing to composition. -func (c SpecComponents) IsEmpty() bool { - return c.Path == "" && c.FullURL == "" && c.URLPrefix == "" && c.Channel == "" -} - -// ParseSpec parses one `.zddc apps:` value into components. -// zddcDir anchors relative paths; root bounds path-traversal. -func ParseSpec(spec, zddcDir, root string) (SpecComponents, error) { - spec = strings.TrimSpace(spec) - if spec == "" { - return SpecComponents{}, fmt.Errorf("source spec is empty") - } - - // Path forms — terminal. - if strings.HasPrefix(spec, "/") || - strings.HasPrefix(spec, "./") || - strings.HasPrefix(spec, "../") { - var abs string - if filepath.IsAbs(spec) { - abs = filepath.Clean(spec) - } else { - abs = filepath.Clean(filepath.Join(zddcDir, spec)) - } - rootClean := filepath.Clean(root) - if abs != rootClean && !strings.HasPrefix(abs, rootClean+string(filepath.Separator)) { - return SpecComponents{}, fmt.Errorf("path %q escapes ZDDC_ROOT", spec) - } - return SpecComponents{Path: abs}, nil - } - - // URL forms. - if strings.HasPrefix(spec, "https://") || strings.HasPrefix(spec, "http://") { - return parseURLSpec(spec) - } - - // Channel-only forms: ":channel" or bare "channel". - chanPart := strings.TrimPrefix(spec, ":") - if chanPart == "" { - return SpecComponents{}, fmt.Errorf("empty channel after ':'") - } - if !isValidChannelOrVersion(chanPart) { - return SpecComponents{}, fmt.Errorf("unrecognized source spec %q (expected channel, version, URL, or path)", spec) - } - return SpecComponents{Channel: normalizeChannel(chanPart)}, nil -} - -// parseURLSpec splits a URL spec into prefix vs full-URL based on the -// last `:` after the last `/`. Examples: -// -// https://host:8080/path:stable → URLPrefix=https://host:8080/path, Channel=stable -// https://host:8080/path → URLPrefix=https://host:8080/path -// https://host/path/file.html → FullURL=https://host/path/file.html (terminal) -// https://host/path/file.html:stable → error (terminal URL with extra suffix) -func parseURLSpec(spec string) (SpecComponents, error) { - // Locate the channel separator: last `:` that comes after the last `/`. - lastSlash := strings.LastIndex(spec, "/") - if lastSlash < 0 { - return SpecComponents{}, fmt.Errorf("invalid URL %q: missing path separator", spec) - } - afterSlash := spec[lastSlash+1:] - colonInTail := strings.LastIndex(afterSlash, ":") - - urlPart, suffixPart := spec, "" - if colonInTail >= 0 { - urlPart = spec[:lastSlash+1+colonInTail] - suffixPart = afterSlash[colonInTail+1:] - } - - // Validate the URL portion. - u, err := url.Parse(urlPart) - if err != nil { - return SpecComponents{}, fmt.Errorf("invalid URL %q: %w", urlPart, err) - } - if u.Host == "" { - return SpecComponents{}, fmt.Errorf("URL %q is missing host", urlPart) - } - - // Terminal full URL: ends in `.html`. A `:suffix` on a `.html` URL is - // rejected to prevent silent misinterpretation. - if strings.HasSuffix(urlPart, ".html") { - if suffixPart != "" { - return SpecComponents{}, fmt.Errorf("URL ends in .html but has %q suffix", ":"+suffixPart) - } - return SpecComponents{FullURL: urlPart}, nil - } - - // URL-prefix form. Strip trailing slash for normalization. - prefix := strings.TrimRight(urlPart, "/") - - out := SpecComponents{URLPrefix: prefix} - if suffixPart != "" { - if !isValidChannelOrVersion(suffixPart) { - return SpecComponents{}, fmt.Errorf("invalid channel/version suffix %q", suffixPart) - } - out.Channel = normalizeChannel(suffixPart) - } - return out, nil -} - -// isValidChannelOrVersion reports whether s is `stable` (the canonical -// "current stable" alias) or an exact-version pin like `v0.0.4` / `0.0.4`. -// Partial pins (`v0.0`, `v0`) and the legacy `beta`/`alpha` channels -// are no longer accepted — the upstream publishes only stable + exact. -func isValidChannelOrVersion(s string) bool { - if s == "stable" { - return true - } - rest := strings.TrimPrefix(s, "v") - if rest == "" { - return false - } - parts := strings.Split(rest, ".") - if len(parts) != 3 { - return false - } - for _, p := range parts { - if p == "" { - return false - } - for _, r := range p { - if r < '0' || r > '9' { - return false - } - } - } - return true -} - -// normalizeChannel ensures versions carry the `v` prefix (so the resulting -// filename is `_v.html` per upstream convention). -func normalizeChannel(s string) string { - if s == "stable" { - return s - } - if !strings.HasPrefix(s, "v") { - return "v" + s - } - return s -} - -// Resolve walks the .zddc chain root→leaf, applying `apps.default` and -// `apps.` at each level. Returns the resolved Source and true if any -// entry contributed; (Source{}, false, nil) means no override (caller -// serves embedded). On malformed spec, returns an error. -func Resolve(chain zddc.PolicyChain, app, root, requestDir string) (Source, bool, error) { - return ResolveWithOverride(chain, app, root, requestDir, "") -} - -// ResolveWithOverride is Resolve with an additional per-request override -// applied as one final cascade level after the .zddc chain. Used to honor -// the `?v=` query parameter on tool HTML requests. -// -// vSpec accepts the same syntax as `.zddc apps:` values (channel/version, -// `:channel`, URL prefix, `url:channel`, full `.html` URL). Path sources -// are rejected (security: `?v=` must resolve to a URL whose bytes the -// caller can fetch from cache only). -// -// Empty vSpec is equivalent to plain Resolve. -func ResolveWithOverride(chain zddc.PolicyChain, app, root, requestDir, vSpec string) (Source, bool, error) { - app = strings.ToLower(strings.TrimSpace(app)) - if !zddc.IsKnownApp(app) { - return Source{}, false, fmt.Errorf("unknown app %q", app) - } - - dirs := walkDirs(root, requestDir) - - st := newAppsState(app) - - // Walk root → leaf. - for i := 0; i < len(chain.Levels); i++ { - lvl := chain.Levels[i] - dir := root - if i < len(dirs) { - dir = dirs[i] - } - // `default` first, then per-app override at the same level. - if spec, ok := lvl.Apps[DefaultAppsKey]; ok && spec != "" { - if err := st.apply(spec, dir, root, "apps.default"); err != nil { - return Source{}, false, err - } - } - if spec, ok := lvl.Apps[app]; ok && spec != "" { - if err := st.apply(spec, dir, root, "apps."+app); err != nil { - return Source{}, false, err - } - } - } - - // Per-request override (`?v=`): one final layer. - if vSpec = strings.TrimSpace(vSpec); vSpec != "" { - comp, err := ParseSpec(vSpec, requestDir, root) - if err != nil { - return Source{}, false, fmt.Errorf("?v=%s: %w", vSpec, err) - } - // Reject path sources from per-request override — security: we serve - // only what the cache (populated by .zddc-controlled fetches) holds. - if comp.Path != "" { - return Source{}, false, fmt.Errorf("?v= cannot specify a local path source") - } - if err := st.applyComponents(comp); err != nil { - return Source{}, false, fmt.Errorf("?v=%s: %w", vSpec, err) - } - } - - return st.finalize() -} - -// appsState accumulates URL-prefix and channel components across cascade -// levels, with terminal-source short-circuit semantics. -type appsState struct { - app string - haveAny bool - urlPrefix string - channel string - terminalSrc *Source -} - -func newAppsState(app string) *appsState { - return &appsState{app: app} -} - -func (s *appsState) apply(spec, zddcDir, root, label string) error { - comp, err := ParseSpec(spec, zddcDir, root) - if err != nil { - return fmt.Errorf("%s: %w", label, err) - } - return s.applyComponents(comp) -} - -func (s *appsState) applyComponents(comp SpecComponents) error { - if comp.IsEmpty() { - return nil - } - s.haveAny = true - switch { - case comp.Path != "": - s.terminalSrc = &Source{App: s.app, Path: comp.Path} - s.urlPrefix, s.channel = "", "" - case comp.FullURL != "": - s.terminalSrc = &Source{App: s.app, URL: comp.FullURL} - s.urlPrefix, s.channel = "", "" - default: - // Non-terminal: deeper non-terminal entries override a parent terminal. - s.terminalSrc = nil - if comp.URLPrefix != "" { - s.urlPrefix = comp.URLPrefix - } - if comp.Channel != "" { - s.channel = comp.Channel - } - } - return nil -} - -func (s *appsState) finalize() (Source, bool, error) { - if !s.haveAny { - return Source{}, false, nil - } - if s.terminalSrc != nil { - return *s.terminalSrc, true, nil - } - urlPrefix := s.urlPrefix - if urlPrefix == "" { - urlPrefix = DefaultUpstreamReleases - } - channel := s.channel - if channel == "" { - channel = DefaultChannel - } - // channel == "stable" → canonical URL /.html (a - // symlink that always follows the latest stable cut). - // channel == "v" → immutable per-version URL. - var name string - if channel == "stable" { - name = s.app + ".html" - } else { - name = s.app + "_" + channel + ".html" - } - return Source{ - App: s.app, - URL: urlPrefix + "/" + name, - }, true, nil -} - -// PreviewLine returns a short human-readable description of how an app -// resolves at requestDir given the chain. Used by the .zddc editor to -// render a "Resolves to: ..." line beside each apps input. -func PreviewLine(chain zddc.PolicyChain, app, root, requestDir string) string { - src, has, err := Resolve(chain, app, root, requestDir) - if err != nil { - return "error: " + err.Error() - } - if !has { - return "embedded (build-time default)" - } - if src.Path != "" { - return "local file: " + src.Path - } - return src.URL -} - -func walkDirs(root, requestDir string) []string { - root = filepath.Clean(root) - requestDir = filepath.Clean(requestDir) - if requestDir == root { - return []string{root} - } - rel, err := filepath.Rel(root, requestDir) - if err != nil { - return []string{root} - } - dirs := []string{root} - cur := root - for _, part := range strings.Split(rel, string(filepath.Separator)) { - cur = filepath.Join(cur, part) - dirs = append(dirs, cur) - } - return dirs -} diff --git a/zddc/internal/apps/apps_test.go b/zddc/internal/apps/apps_test.go deleted file mode 100644 index 012293e..0000000 --- a/zddc/internal/apps/apps_test.go +++ /dev/null @@ -1,438 +0,0 @@ -package apps - -import ( - "path/filepath" - "strings" - "testing" - - "codeberg.org/VARASYS/ZDDC/zddc/internal/zddc" -) - -// ── ParseSpec ──────────────────────────────────────────────────────────── - -func TestParseSpec_Channels(t *testing.T) { - // "stable" is the only channel alias (latest stable). beta and alpha - // channels no longer exist as public concepts. - cases := []struct { - spec, wantChan string - }{ - {"stable", "stable"}, - {":stable", "stable"}, - } - for _, tc := range cases { - t.Run(tc.spec, func(t *testing.T) { - c, err := ParseSpec(tc.spec, "/root", "/root") - if err != nil { - t.Fatalf("ParseSpec error: %v", err) - } - if c.Channel != tc.wantChan { - t.Errorf("got Channel=%q, want %q", c.Channel, tc.wantChan) - } - if c.URLPrefix != "" || c.Path != "" || c.FullURL != "" { - t.Errorf("expected channel-only, got %+v", c) - } - }) - } -} - -func TestParseSpec_Versions(t *testing.T) { - // Exact-version pins only. Partial pins (v0.0, v0) no longer exist - // — the upstream publishes .html (current stable) and - // _v.html (exact-version immutable). Bare "0.0.4" - // (no v prefix) is normalized to "v0.0.4". - cases := []struct { - spec, wantChan string - }{ - {"v0.0.4", "v0.0.4"}, - {"0.0.4", "v0.0.4"}, - {":v0.0.4", "v0.0.4"}, - {":0.0.4", "v0.0.4"}, - } - for _, tc := range cases { - t.Run(tc.spec, func(t *testing.T) { - c, err := ParseSpec(tc.spec, "/root", "/root") - if err != nil { - t.Fatalf("ParseSpec error: %v", err) - } - if c.Channel != tc.wantChan { - t.Errorf("got Channel=%q, want %q", c.Channel, tc.wantChan) - } - }) - } -} - -func TestParseSpec_RejectsLegacyChannelsAndPartialPins(t *testing.T) { - // alpha/beta channels and partial-version pins are no longer valid. - rejected := []string{"alpha", "beta", ":alpha", ":beta", "v0.0", "v0", "0.0", "0", ":v0.0"} - for _, spec := range rejected { - t.Run(spec, func(t *testing.T) { - _, err := ParseSpec(spec, "/root", "/root") - if err == nil { - t.Errorf("expected error for %q, got none", spec) - } - }) - } -} - -func TestParseSpec_URLPrefix(t *testing.T) { - cases := []struct { - spec, wantPrefix, wantChan string - }{ - {"https://my-mirror.example/releases", "https://my-mirror.example/releases", ""}, - {"https://my-mirror.example/releases/", "https://my-mirror.example/releases", ""}, // trailing slash stripped - {"https://my-mirror.example/releases:stable", "https://my-mirror.example/releases", "stable"}, - {"https://my-mirror.example/releases:v0.0.4", "https://my-mirror.example/releases", "v0.0.4"}, - // Port colon must NOT be confused with channel separator. - {"https://my-mirror.example:8080/releases", "https://my-mirror.example:8080/releases", ""}, - {"https://my-mirror.example:8080/releases:stable", "https://my-mirror.example:8080/releases", "stable"}, - // Colon embedded in path before final slash — treated as part of path. - {"https://host/some:thing/releases", "https://host/some:thing/releases", ""}, - {"https://host/some:thing/releases:v0.0.4", "https://host/some:thing/releases", "v0.0.4"}, - } - for _, tc := range cases { - t.Run(tc.spec, func(t *testing.T) { - c, err := ParseSpec(tc.spec, "/root", "/root") - if err != nil { - t.Fatalf("ParseSpec error: %v", err) - } - if c.URLPrefix != tc.wantPrefix { - t.Errorf("got URLPrefix=%q, want %q", c.URLPrefix, tc.wantPrefix) - } - if c.Channel != tc.wantChan { - t.Errorf("got Channel=%q, want %q", c.Channel, tc.wantChan) - } - if c.Path != "" || c.FullURL != "" { - t.Errorf("expected non-terminal, got %+v", c) - } - }) - } -} - -func TestParseSpec_FullURL(t *testing.T) { - c, err := ParseSpec("https://my-fork.example/archive.html", "/root", "/root") - if err != nil { - t.Fatalf("ParseSpec error: %v", err) - } - if c.FullURL != "https://my-fork.example/archive.html" { - t.Errorf("got FullURL=%q", c.FullURL) - } - if !c.IsTerminal() { - t.Errorf("expected terminal") - } -} - -func TestParseSpec_FullURLWithChannelSuffixRejected(t *testing.T) { - _, err := ParseSpec("https://my-fork.example/archive.html:stable", "/root", "/root") - if err == nil { - t.Errorf("expected error for .html URL with :suffix") - } -} - -func TestParseSpec_Paths(t *testing.T) { - root := t.TempDir() - zddcDir := filepath.Join(root, "Project-X") - cases := []struct { - spec string - wantOK bool - wantPath string - }{ - {"./local.html", true, filepath.Join(zddcDir, "local.html")}, - {"../sibling.html", true, filepath.Join(root, "sibling.html")}, - {filepath.Join(root, "abs.html"), true, filepath.Join(root, "abs.html")}, - {"/etc/passwd", false, ""}, - {"../../../escape.html", false, ""}, - } - for _, tc := range cases { - t.Run(tc.spec, func(t *testing.T) { - c, err := ParseSpec(tc.spec, zddcDir, root) - if tc.wantOK { - if err != nil { - t.Fatalf("want success, got error: %v", err) - } - if c.Path != tc.wantPath { - t.Errorf("got Path=%q, want %q", c.Path, tc.wantPath) - } - if !c.IsTerminal() { - t.Errorf("expected terminal") - } - } else { - if err == nil { - t.Errorf("want error, got %+v", c) - } - } - }) - } -} - -func TestParseSpec_Errors(t *testing.T) { - cases := []string{ - "", - "weird-thing", - ":", - ":weird", - "v", - "v0.0.0.0", - "v0.a.0", - "https://", // missing host - } - for _, tc := range cases { - t.Run(tc, func(t *testing.T) { - _, err := ParseSpec(tc, "/root", "/root") - if err == nil { - t.Errorf("ParseSpec(%q) = nil, want error", tc) - } - }) - } -} - -// ── Resolve ────────────────────────────────────────────────────────────── - -func TestResolve_NoEntries(t *testing.T) { - chain := zddc.PolicyChain{Levels: []zddc.ZddcFile{{}}} - _, has, err := Resolve(chain, "archive", t.TempDir(), t.TempDir()) - if err != nil { - t.Fatalf("Resolve: %v", err) - } - if has { - t.Errorf("got override=true, want false") - } -} - -func TestResolve_PerAppChannelOnly(t *testing.T) { - root := t.TempDir() - chain := zddc.PolicyChain{Levels: []zddc.ZddcFile{{ - Apps: map[string]string{"archive": "stable"}, - }}} - src, has, err := Resolve(chain, "archive", root, root) - if err != nil || !has { - t.Fatalf("has=%v err=%v", has, err) - } - // stable channel → canonical URL (no _stable_ suffix); the upstream - // publishes a symlink at this URL pointing at the latest version. - want := DefaultUpstreamReleases + "/archive.html" - if src.URL != want { - t.Errorf("got URL=%q, want %q", src.URL, want) - } -} - -func TestResolve_PerAppVersionOnly(t *testing.T) { - root := t.TempDir() - chain := zddc.PolicyChain{Levels: []zddc.ZddcFile{{ - Apps: map[string]string{"archive": "v0.0.4"}, - }}} - src, _, err := Resolve(chain, "archive", root, root) - if err != nil { - t.Fatal(err) - } - want := DefaultUpstreamReleases + "/archive_v0.0.4.html" - if src.URL != want { - t.Errorf("got URL=%q, want %q", src.URL, want) - } -} - -func TestResolve_DefaultProvidesURLAndChannel(t *testing.T) { - root := t.TempDir() - chain := zddc.PolicyChain{Levels: []zddc.ZddcFile{{ - Apps: map[string]string{ - "default": "https://mirror.example/releases:v0.0.4", - }, - }}} - src, has, err := Resolve(chain, "archive", root, root) - if err != nil || !has { - t.Fatalf("has=%v err=%v", has, err) - } - if src.URL != "https://mirror.example/releases/archive_v0.0.4.html" { - t.Errorf("got URL=%q", src.URL) - } -} - -func TestResolve_DefaultPlusPerAppChannelOverride(t *testing.T) { - // default=https://zddc.varasys.io/releases:stable, classifier=:v0.0.4 - // → classifier pinned to v0.0.4 on the same mirror. - root := t.TempDir() - chain := zddc.PolicyChain{Levels: []zddc.ZddcFile{{ - Apps: map[string]string{ - "default": "https://zddc.varasys.io/releases:stable", - "classifier": ":v0.0.4", - }, - }}} - src, _, err := Resolve(chain, "classifier", root, root) - if err != nil { - t.Fatal(err) - } - if src.URL != "https://zddc.varasys.io/releases/classifier_v0.0.4.html" { - t.Errorf("got URL=%q", src.URL) - } -} - -func TestResolve_DefaultPlusPerAppURLPrefixOverride(t *testing.T) { - // default=...:stable, archive=https://my.local.stuff/releases - // → custom URL + default channel (stable, canonical filename). - root := t.TempDir() - chain := zddc.PolicyChain{Levels: []zddc.ZddcFile{{ - Apps: map[string]string{ - "default": "https://zddc.varasys.io/releases:stable", - "archive": "https://my.local.stuff/releases", - }, - }}} - src, _, err := Resolve(chain, "archive", root, root) - if err != nil { - t.Fatal(err) - } - if src.URL != "https://my.local.stuff/releases/archive.html" { - t.Errorf("got URL=%q", src.URL) - } -} - -func TestResolve_DeeperLevelOverridesParentChannel(t *testing.T) { - root := t.TempDir() - requestDir := filepath.Join(root, "Project-A") - chain := zddc.PolicyChain{Levels: []zddc.ZddcFile{ - {Apps: map[string]string{"default": ":stable"}}, - {Apps: map[string]string{"default": ":v0.0.4"}}, - }} - src, _, err := Resolve(chain, "archive", root, requestDir) - if err != nil { - t.Fatal(err) - } - want := DefaultUpstreamReleases + "/archive_v0.0.4.html" - if src.URL != want { - t.Errorf("got URL=%q, want %q", src.URL, want) - } -} - -func TestResolve_DeeperLevelOverridesParentURL(t *testing.T) { - root := t.TempDir() - requestDir := filepath.Join(root, "Project-A") - chain := zddc.PolicyChain{Levels: []zddc.ZddcFile{ - {Apps: map[string]string{"default": "https://a.example/releases:stable"}}, - {Apps: map[string]string{"default": "https://b.example/releases"}}, - }} - src, _, err := Resolve(chain, "archive", root, requestDir) - if err != nil { - t.Fatal(err) - } - // b.example URL prefix wins; channel inherited (stable → canonical - // filename, no _stable_ suffix). - want := "https://b.example/releases/archive.html" - if src.URL != want { - t.Errorf("got URL=%q, want %q", src.URL, want) - } -} - -func TestResolve_TerminalAtLeafBeatsParentDefault(t *testing.T) { - root := t.TempDir() - requestDir := filepath.Join(root, "Project-A") - chain := zddc.PolicyChain{Levels: []zddc.ZddcFile{ - {Apps: map[string]string{"default": "https://a.example/releases:stable"}}, - {Apps: map[string]string{"archive": "https://my-fork.example/archive.html"}}, - }} - src, _, err := Resolve(chain, "archive", root, requestDir) - if err != nil { - t.Fatal(err) - } - if src.URL != "https://my-fork.example/archive.html" { - t.Errorf("got URL=%q (want terminal full URL)", src.URL) - } -} - -func TestResolve_DeeperNonTerminalOverridesParentTerminal(t *testing.T) { - root := t.TempDir() - requestDir := filepath.Join(root, "Project-A") - chain := zddc.PolicyChain{Levels: []zddc.ZddcFile{ - {Apps: map[string]string{"archive": "https://a.example/archive.html"}}, // terminal - {Apps: map[string]string{"archive": "v0.0.4"}}, // non-terminal - }} - src, _, err := Resolve(chain, "archive", root, requestDir) - if err != nil { - t.Fatal(err) - } - want := DefaultUpstreamReleases + "/archive_v0.0.4.html" - if src.URL != want { - t.Errorf("got URL=%q, want %q", src.URL, want) - } -} - -func TestResolve_PathSourceTerminal(t *testing.T) { - root := t.TempDir() - projDir := filepath.Join(root, "Project-X") - chain := zddc.PolicyChain{Levels: []zddc.ZddcFile{ - {}, - {Apps: map[string]string{"archive": "./our-archive.html"}}, - }} - src, _, err := Resolve(chain, "archive", root, projDir) - if err != nil { - t.Fatal(err) - } - if src.URL != "" { - t.Errorf("got URL=%q, want empty", src.URL) - } - want := filepath.Join(projDir, "our-archive.html") - if src.Path != want { - t.Errorf("got Path=%q, want %q", src.Path, want) - } -} - -func TestResolve_PerAppOverridesDefaultAtSameLevel(t *testing.T) { - root := t.TempDir() - chain := zddc.PolicyChain{Levels: []zddc.ZddcFile{{ - Apps: map[string]string{ - "default": "https://a.example/releases:stable", - "archive": "https://b.example/archive.html", // terminal — wins for archive only - }, - }}} - src, _, err := Resolve(chain, "archive", root, root) - if err != nil { - t.Fatal(err) - } - if src.URL != "https://b.example/archive.html" { - t.Errorf("got URL=%q (want b.example terminal)", src.URL) - } - - // Other apps still use the default. - src2, _, err := Resolve(chain, "classifier", root, root) - if err != nil { - t.Fatal(err) - } - if src2.URL != "https://a.example/releases/classifier.html" { - t.Errorf("got classifier URL=%q (want a.example default)", src2.URL) - } -} - -func TestResolve_BadSpecBubblesError(t *testing.T) { - root := t.TempDir() - chain := zddc.PolicyChain{Levels: []zddc.ZddcFile{{ - Apps: map[string]string{"archive": "this is garbage"}, - }}} - _, _, err := Resolve(chain, "archive", root, root) - if err == nil { - t.Errorf("expected error") - } -} - -func TestResolve_UnknownAppRejected(t *testing.T) { - root := t.TempDir() - _, _, err := Resolve(zddc.PolicyChain{}, "unknown", root, root) - if err == nil { - t.Errorf("expected error") - } -} - -// ── PreviewLine ────────────────────────────────────────────────────────── - -func TestPreviewLine(t *testing.T) { - root := t.TempDir() - t.Run("no entries → embedded", func(t *testing.T) { - got := PreviewLine(zddc.PolicyChain{Levels: []zddc.ZddcFile{{}}}, "archive", root, root) - if !strings.Contains(got, "embedded") { - t.Errorf("got %q", got) - } - }) - t.Run("default channel → URL", func(t *testing.T) { - chain := zddc.PolicyChain{Levels: []zddc.ZddcFile{{Apps: map[string]string{"default": ":v0.0.4"}}}} - got := PreviewLine(chain, "archive", root, root) - if !strings.Contains(got, "archive_v0.0.4.html") { - t.Errorf("got %q", got) - } - }) -} diff --git a/zddc/internal/apps/bundle.go b/zddc/internal/apps/bundle.go new file mode 100644 index 0000000..c3436a5 --- /dev/null +++ b/zddc/internal/apps/bundle.go @@ -0,0 +1,115 @@ +package apps + +import ( + "archive/zip" + "bytes" + "io" + "log/slog" + "os" + "path/filepath" + "sync" + "time" + + "codeberg.org/VARASYS/ZDDC/zddc/internal/zipfs" +) + +// BundleName is the site-root config bundle that supplies local tool-HTML +// overrides (and, in future, templated config). It lives at +// /.zddc.zip. It is dot-hidden and 404-gated over HTTP (it's +// config, not browsable content); the server reads it from the filesystem +// internally, so members resolve for any user regardless of the HTTP gate. +const BundleName = ".zddc.zip" + +// maxBundleBytes caps the whole .zddc.zip read into memory. The bundle is +// small config (a handful of HTML files), so a generous cap is fine. +const maxBundleBytes = 64 << 20 // 64 MiB + +// maxBundleMemberBytes caps a single extracted member. +const maxBundleMemberBytes = 32 << 20 // 32 MiB + +// Bundle is the cached parsed view of /.zddc.zip. A nil *Bundle +// is valid and behaves as "no bundle present" for all methods. Member() +// re-stats the file each call (cheap, and gives free hot-reload when an +// operator drops in a new bundle), reparsing only when mtime or size change. +type Bundle struct { + path string + logger *slog.Logger + + mu sync.Mutex + data []byte + reader *zip.Reader + modTime time.Time + size int64 + loaded bool // a valid zip is parsed into reader +} + +// NewBundle returns a Bundle bound to /.zddc.zip. The file need +// not exist; Member returns (nil,false) until it does. +func NewBundle(zddcRoot string, logger *slog.Logger) *Bundle { + if logger == nil { + logger = slog.Default() + } + return &Bundle{path: filepath.Join(zddcRoot, BundleName), logger: logger} +} + +// Member returns the bytes of the named member (e.g. "browse.html") from the +// bundle, or (nil,false) when the bundle is absent, unreadable, corrupt, or +// has no such member. Lookup is case-insensitive (via zipfs), matching the +// rest of the server's URL case-folding. +func (b *Bundle) Member(name string) ([]byte, bool) { + if b == nil { + return nil, false + } + b.mu.Lock() + defer b.mu.Unlock() + + info, err := os.Stat(b.path) + if err != nil || info.IsDir() { + // Absent (or replaced by a dir) → no bundle. Drop any stale parse. + b.data, b.reader, b.loaded = nil, nil, false + return nil, false + } + if !b.loaded || info.ModTime() != b.modTime || info.Size() != b.size { + if !b.reparse(info) { + return nil, false + } + } + rc, _, _, _, ok := zipfs.OpenMember(b.reader, name) + if !ok { + return nil, false + } + defer rc.Close() + body, err := io.ReadAll(io.LimitReader(rc, maxBundleMemberBytes+1)) + if err != nil || int64(len(body)) > maxBundleMemberBytes { + b.logger.Warn("zddc.zip member unreadable or too large", "member", name) + return nil, false + } + return body, true +} + +// reparse re-reads + re-parses the bundle. Caller holds b.mu. On any error +// the bundle is treated as absent (loaded=false) and the server falls back +// to embedded. Returns true when a valid reader is in place. +func (b *Bundle) reparse(info os.FileInfo) bool { + b.data, b.reader, b.loaded = nil, nil, false + if info.Size() > maxBundleBytes { + b.logger.Warn("zddc.zip too large; ignoring", "size", info.Size(), "cap", maxBundleBytes) + return false + } + data, err := os.ReadFile(b.path) + if err != nil { + b.logger.Warn("zddc.zip unreadable; ignoring", "err", err) + return false + } + zr, err := zip.NewReader(bytes.NewReader(data), int64(len(data))) + if err != nil { + b.logger.Warn("zddc.zip is not a valid zip; ignoring", "err", err) + return false + } + b.data = data + b.reader = zr + b.modTime = info.ModTime() + b.size = info.Size() + b.loaded = true + return true +} diff --git a/zddc/internal/apps/bundle_test.go b/zddc/internal/apps/bundle_test.go new file mode 100644 index 0000000..f873c02 --- /dev/null +++ b/zddc/internal/apps/bundle_test.go @@ -0,0 +1,96 @@ +package apps + +import ( + "archive/zip" + "bytes" + "os" + "path/filepath" + "testing" + "time" +) + +// writeTestBundle writes a /.zddc.zip containing the given members. +// Shared by bundle + handler precedence tests. +func writeTestBundle(t *testing.T, dir string, members map[string]string) string { + t.Helper() + var buf bytes.Buffer + zw := zip.NewWriter(&buf) + for name, body := range members { + w, err := zw.Create(name) + if err != nil { + t.Fatalf("zip create %s: %v", name, err) + } + if _, err := w.Write([]byte(body)); err != nil { + t.Fatalf("zip write %s: %v", name, err) + } + } + if err := zw.Close(); err != nil { + t.Fatalf("zip close: %v", err) + } + p := filepath.Join(dir, BundleName) + if err := os.WriteFile(p, buf.Bytes(), 0o644); err != nil { + t.Fatalf("write bundle: %v", err) + } + return p +} + +func TestBundle_Member_Hit(t *testing.T) { + root := t.TempDir() + writeTestBundle(t, root, map[string]string{"browse.html": "BUNDLE browse"}) + b := NewBundle(root, nil) + got, ok := b.Member("browse.html") + if !ok || string(got) != "BUNDLE browse" { + t.Fatalf("Member = (%q,%v), want (BUNDLE browse,true)", got, ok) + } + // Case-insensitive lookup (matches URL folding). + if _, ok := b.Member("BROWSE.HTML"); !ok { + t.Errorf("case-insensitive member lookup failed") + } +} + +func TestBundle_Member_Absent(t *testing.T) { + root := t.TempDir() + writeTestBundle(t, root, map[string]string{"browse.html": "x"}) + b := NewBundle(root, nil) + if _, ok := b.Member("archive.html"); ok { + t.Errorf("absent member reported present") + } +} + +func TestBundle_NoFile(t *testing.T) { + b := NewBundle(t.TempDir(), nil) + if _, ok := b.Member("browse.html"); ok { + t.Errorf("no bundle file but member reported present") + } + // nil bundle is safe. + var nb *Bundle + if _, ok := nb.Member("browse.html"); ok { + t.Errorf("nil bundle reported a member") + } +} + +func TestBundle_HotReload(t *testing.T) { + root := t.TempDir() + p := writeTestBundle(t, root, map[string]string{"browse.html": "v1"}) + b := NewBundle(root, nil) + if got, _ := b.Member("browse.html"); string(got) != "v1" { + t.Fatalf("first read = %q, want v1", got) + } + // Rewrite with new bytes + a bumped mtime so the stat-based cache reparses. + writeTestBundle(t, root, map[string]string{"browse.html": "v2"}) + _ = os.Chtimes(p, time.Now().Add(2*time.Second), time.Now().Add(2*time.Second)) + if got, _ := b.Member("browse.html"); string(got) != "v2" { + t.Errorf("after reload = %q, want v2", got) + } +} + +func TestBundle_CorruptZip(t *testing.T) { + root := t.TempDir() + if err := os.WriteFile(filepath.Join(root, BundleName), []byte("not a zip"), 0o644); err != nil { + t.Fatal(err) + } + b := NewBundle(root, nil) + if _, ok := b.Member("browse.html"); ok { + t.Errorf("corrupt zip should yield no members") + } +} diff --git a/zddc/internal/apps/cache.go b/zddc/internal/apps/cache.go deleted file mode 100644 index 50a4ccb..0000000 --- a/zddc/internal/apps/cache.go +++ /dev/null @@ -1,187 +0,0 @@ -package apps - -import ( - "fmt" - "io/fs" - "net/url" - "os" - "path/filepath" - "strings" -) - -// Cache stores fetched URL responses on disk under /_app/. -// Files are name-keyed by upstream host + path so operators can list -// and inspect them by hand. There is no metadata, no SHA-256, no -// expiration — fetch-once-and-keep-forever. To force a refetch, -// delete the cache file. -type Cache struct { - root string -} - -// NewCache creates a Cache rooted at the given path. The directory is -// created if missing. Stale *.tmp files left over from interrupted -// writes are swept on construction. -func NewCache(root string) (*Cache, error) { - root = filepath.Clean(root) - if err := os.MkdirAll(root, 0o755); err != nil { - return nil, fmt.Errorf("create cache root: %w", err) - } - c := &Cache{root: root} - if err := c.sweepTemps(); err != nil { - return nil, fmt.Errorf("sweep temps: %w", err) - } - return c, nil -} - -// Root returns the cache directory absolute path. -func (c *Cache) Root() string { return c.root } - -// keyForURL converts a URL into a relative filesystem path under the -// cache root. -// -// Layout: /[:]/. The full origin tuple is in -// the key so two URLs that resolve different content cannot collide: -// -// https://example.com/x.html → https/example.com/x.html -// http://example.com/x.html → http/example.com/x.html -// https://example.com:8443/x.html → https/example.com:8443/x.html -// -// No port stripping. The previous behavior — collapsing :443 onto bare -// host for https (and :80 for http) — was a defensible HTTP convention -// but conflated "the operator wrote a URL with the default port" with -// "the operator wrote a bare-host URL". With explicit port preserved, -// every URL maps to exactly one filesystem path; operators can still -// `ls _app/https/example.com/` to inspect what's cached. Scheme -// segregation prevents an http:// hit from masquerading as an https:// -// hit when both are deliberately distinct (rare, but real on -// reverse-proxied stacks where http and https serve different bytes). -// -// Host is lowercased so the canonical-host normalization survives -// case-insensitive DNS. Port is preserved verbatim. -func keyForURL(rawURL string) (string, error) { - u, err := url.Parse(rawURL) - if err != nil { - return "", fmt.Errorf("parse URL: %w", err) - } - if u.Scheme != "http" && u.Scheme != "https" { - return "", fmt.Errorf("unsupported scheme %q", u.Scheme) - } - if u.Host == "" { - return "", fmt.Errorf("URL is missing host") - } - if u.RawQuery != "" { - return "", fmt.Errorf("URL must not contain query string: %s", rawURL) - } - // Lowercase the host part but preserve the port verbatim. Without - // this we'd lowercase a numeric port unnecessarily, which is fine - // but pointless; with this the ASCII-cased host normalization - // works the same for both default and explicit-port URLs. - host := u.Host - if i := strings.Index(host, ":"); i >= 0 { - host = strings.ToLower(host[:i]) + host[i:] - } else { - host = strings.ToLower(host) - } - p := u.Path - for strings.Contains(p, "//") { - p = strings.ReplaceAll(p, "//", "/") - } - p = strings.TrimPrefix(p, "/") - if p == "" { - p = "index.html" - } - cleaned := filepath.Clean("/" + p) - if strings.Contains(cleaned, "..") { - return "", fmt.Errorf("URL path contains '..'") - } - return u.Scheme + "/" + host + cleaned, nil -} - -func (c *Cache) pathFor(rawURL string) (string, error) { - key, err := keyForURL(rawURL) - if err != nil { - return "", err - } - return filepath.Join(c.root, filepath.FromSlash(key)), nil -} - -// Has reports whether a cache entry exists for the URL. -func (c *Cache) Has(rawURL string) bool { - p, err := c.pathFor(rawURL) - if err != nil { - return false - } - _, err = os.Stat(p) - return err == nil -} - -// Read returns the cached body or os.ErrNotExist. -func (c *Cache) Read(rawURL string) ([]byte, error) { - p, err := c.pathFor(rawURL) - if err != nil { - return nil, err - } - return os.ReadFile(p) -} - -// Write atomically stores body for the URL. Parent directories are -// created as needed. Writes via tmp+rename so partial files are never -// observable. -func (c *Cache) Write(rawURL string, body []byte) error { - p, err := c.pathFor(rawURL) - if err != nil { - return err - } - if err := os.MkdirAll(filepath.Dir(p), 0o755); err != nil { - return err - } - return writeAtomic(p, body) -} - -func writeAtomic(path string, data []byte) error { - dir := filepath.Dir(path) - tmp, err := os.CreateTemp(dir, filepath.Base(path)+".tmp.*") - if err != nil { - return err - } - tmpName := tmp.Name() - cleanup := func() { _ = os.Remove(tmpName) } - if _, err := tmp.Write(data); err != nil { - _ = tmp.Close() - cleanup() - return err - } - if err := tmp.Sync(); err != nil { - _ = tmp.Close() - cleanup() - return err - } - if err := tmp.Close(); err != nil { - cleanup() - return err - } - if err := os.Rename(tmpName, path); err != nil { - cleanup() - return err - } - return nil -} - -func (c *Cache) sweepTemps() error { - err := filepath.WalkDir(c.root, func(p string, d fs.DirEntry, err error) error { - if err != nil { - return err - } - if d.IsDir() { - return nil - } - if strings.Contains(d.Name(), ".tmp.") { - _ = os.Remove(p) - } - return nil - }) - if err != nil && !os.IsNotExist(err) { - return err - } - return nil -} diff --git a/zddc/internal/apps/cache_test.go b/zddc/internal/apps/cache_test.go deleted file mode 100644 index 67b5e09..0000000 --- a/zddc/internal/apps/cache_test.go +++ /dev/null @@ -1,160 +0,0 @@ -package apps - -import ( - "os" - "path/filepath" - "strings" - "testing" -) - -func TestKeyForURL(t *testing.T) { - cases := []struct { - raw, want string - }{ - // Default ports are PRESERVED — no port-stripping (the previous - // behavior conflated "operator wrote :443" with "operator wrote - // bare host"; with the full origin in the key, every URL maps - // to exactly one path). - {"https://zddc.varasys.io/releases/archive_stable.html", "https/zddc.varasys.io/releases/archive_stable.html"}, - {"https://ZDDC.Varasys.IO/releases/archive_stable.html", "https/zddc.varasys.io/releases/archive_stable.html"}, - {"http://example.com/foo.html", "http/example.com/foo.html"}, - {"http://example.com:80/foo.html", "http/example.com:80/foo.html"}, - {"https://example.com/foo.html", "https/example.com/foo.html"}, - {"https://example.com:443/foo.html", "https/example.com:443/foo.html"}, - {"https://example.com:8443/foo.html", "https/example.com:8443/foo.html"}, - // Scheme segregation: same host+path under http and https map - // to different cache entries (defensive against reverse-proxy - // stacks that legitimately serve different bytes per scheme). - {"http://example.com/x.html", "http/example.com/x.html"}, - {"https://example.com/x.html", "https/example.com/x.html"}, - // Path normalization preserved. - {"https://example.com/", "https/example.com/index.html"}, - {"https://example.com", "https/example.com/index.html"}, - {"https://example.com//foo//bar.html", "https/example.com/foo/bar.html"}, - } - for _, tc := range cases { - t.Run(tc.raw, func(t *testing.T) { - got, err := keyForURL(tc.raw) - if err != nil { - t.Fatalf("keyForURL error: %v", err) - } - if got != tc.want { - t.Errorf("got %q, want %q", got, tc.want) - } - }) - } -} - -// TestKeyForURL_NoCollisions: explicit assertion that the dimensions -// previously collapsed (default-port ↔ bare-host, http ↔ https) are -// now distinct. Any future change that re-introduces collapsing will -// fail this test. -func TestKeyForURL_NoCollisions(t *testing.T) { - pairs := [][2]string{ - // Different scheme, same host+path - {"http://example.com/x.html", "https://example.com/x.html"}, - // https default port preserved (not collapsed onto bare host) - {"https://example.com/x.html", "https://example.com:443/x.html"}, - // http default port preserved - {"http://example.com/x.html", "http://example.com:80/x.html"}, - // Different non-default ports - {"https://example.com:8443/x.html", "https://example.com:9443/x.html"}, - } - for _, p := range pairs { - t.Run(p[0]+" vs "+p[1], func(t *testing.T) { - a, err := keyForURL(p[0]) - if err != nil { - t.Fatalf("keyForURL(%q): %v", p[0], err) - } - b, err := keyForURL(p[1]) - if err != nil { - t.Fatalf("keyForURL(%q): %v", p[1], err) - } - if a == b { - t.Errorf("collision: %q and %q both → %q", p[0], p[1], a) - } - }) - } -} - -func TestKeyForURL_Errors(t *testing.T) { - cases := []string{ - "", - "not-a-url", - "ftp://example.com/x.html", - "https:///x.html", - "https://example.com/x.html?v=1", - } - for _, tc := range cases { - t.Run(tc, func(t *testing.T) { - if _, err := keyForURL(tc); err == nil { - t.Errorf("keyForURL(%q) = nil, want error", tc) - } - }) - } -} - -func TestCacheRoundtrip(t *testing.T) { - c, err := NewCache(filepath.Join(t.TempDir(), "_app")) - if err != nil { - t.Fatalf("NewCache: %v", err) - } - urlStr := "https://zddc.varasys.io/releases/archive_stable.html" - body := []byte("archive content") - - if c.Has(urlStr) { - t.Fatalf("Has(empty cache) = true, want false") - } - if err := c.Write(urlStr, body); err != nil { - t.Fatalf("Write: %v", err) - } - if !c.Has(urlStr) { - t.Fatalf("Has(after write) = false, want true") - } - got, err := c.Read(urlStr) - if err != nil { - t.Fatalf("Read: %v", err) - } - if string(got) != string(body) { - t.Errorf("body mismatch") - } -} - -func TestCacheAtomicWrite_LeavesNoTempOnSuccess(t *testing.T) { - root := filepath.Join(t.TempDir(), "_app") - c, _ := NewCache(root) - urlStr := "https://zddc.varasys.io/releases/archive_stable.html" - if err := c.Write(urlStr, []byte("hello")); err != nil { - t.Fatalf("Write: %v", err) - } - count := 0 - _ = filepath.Walk(root, func(p string, info os.FileInfo, err error) error { - if err != nil { - return err - } - if !info.IsDir() && strings.Contains(info.Name(), ".tmp.") { - count++ - } - return nil - }) - if count != 0 { - t.Errorf("found %d .tmp.* leftovers, want 0", count) - } -} - -func TestCacheSweepsTempsOnNew(t *testing.T) { - root := filepath.Join(t.TempDir(), "_app") - stale := filepath.Join(root, "example.com", "releases", "archive_stable.html.tmp.123") - if err := os.MkdirAll(filepath.Dir(stale), 0o755); err != nil { - t.Fatal(err) - } - if err := os.WriteFile(stale, []byte("partial"), 0o644); err != nil { - t.Fatal(err) - } - if _, err := NewCache(root); err != nil { - t.Fatalf("NewCache: %v", err) - } - if _, err := os.Stat(stale); !os.IsNotExist(err) { - t.Errorf("stale tmp file not swept: %v", err) - } -} diff --git a/zddc/internal/apps/fetch.go b/zddc/internal/apps/fetch.go deleted file mode 100644 index 7572231..0000000 --- a/zddc/internal/apps/fetch.go +++ /dev/null @@ -1,193 +0,0 @@ -package apps - -import ( - "context" - "crypto/ed25519" - "errors" - "fmt" - "io" - "log/slog" - "net/http" - "sync" - "time" -) - -// Fetcher pulls URL sources once, caches the body forever, and serves -// from cache on subsequent calls. Path sources don't go through here — -// the handler reads the file directly. -// -// Concurrent calls for the same URL dedupe via singleflight. There is no -// background refresh, no conditional GET. -// -// Signature verification (Ed25519). Strict. On every fetch, also -// fetches .sig (raw 64-byte Ed25519 signature). The fetched body -// is rejected unless the .sig is present, well-formed, and verifies -// against the trusted public key. Rejection causes the apps resolver -// to fall through to the embedded copy. -// -// There is no "accept unsigned with a warning" mode and no embedded -// default key. The operator configures VerifyKey explicitly via -// --apps-pubkey or ZDDC_APPS_PUBKEY (same posture as TLS certificates: -// zddc-server bakes nothing in). When VerifyKey is nil, every URL fetch -// is rejected with an error noting the missing config — the resolver -// falls back to embedded and operators get a clear signal that they -// need to opt in. -// -// Every URL the resolver might fetch is expected to have a -// corresponding .sig published by whoever signed the artifact. -// Operators using custom mirrors must sign their own artifacts and -// host the .sig alongside, then configure their public key here. -type Fetcher struct { - Cache *Cache - Client *http.Client - Logger *slog.Logger - - // VerifyKey is the Ed25519 public key against which fetched - // artifacts are verified. Set at startup from the operator's - // configured --apps-pubkey path. nil = URL fetches refuse-by- - // default (caller falls back to embedded). - VerifyKey ed25519.PublicKey - - sf singleflightGroup - embeddedFails sync.Map // url → struct{} (rate-limit "fell back to embedded" warnings) -} - -// NewFetcher returns a Fetcher with sensible defaults: 10s timeout, no -// redirects (ops must point at the final URL). -func NewFetcher(cache *Cache, logger *slog.Logger) *Fetcher { - if logger == nil { - logger = slog.Default() - } - return &Fetcher{ - Cache: cache, - Logger: logger, - // VerifyKey starts nil. Operator configures it via - // cfg.AppsPubKey at server startup; main.go sets it on the - // returned Fetcher before any request is served. - Client: &http.Client{ - Timeout: 10 * time.Second, - CheckRedirect: func(*http.Request, []*http.Request) error { - return http.ErrUseLastResponse - }, - }, - } -} - -// Fetch returns the body for url. If the cache already has it, returns -// the cached bytes immediately. Otherwise fetches, caches, and returns. -// All concurrent requests for the same URL share one outbound fetch. -func (f *Fetcher) Fetch(ctx context.Context, urlStr string) ([]byte, error) { - if f.Cache != nil { - if body, err := f.Cache.Read(urlStr); err == nil { - return body, nil - } - } - val, err := f.sf.Do(urlStr, func() (any, error) { - return f.fetchOnce(ctx, urlStr) - }) - if err != nil { - return nil, err - } - return val.([]byte), nil -} - -func (f *Fetcher) fetchOnce(ctx context.Context, urlStr string) ([]byte, error) { - req, err := http.NewRequestWithContext(ctx, http.MethodGet, urlStr, nil) - if err != nil { - return nil, err - } - resp, err := f.Client.Do(req) - if err != nil { - return nil, err - } - defer resp.Body.Close() - if resp.StatusCode < 200 || resp.StatusCode >= 300 { - return nil, fmt.Errorf("upstream %s returned HTTP %d", urlStr, resp.StatusCode) - } - const maxBytes = 25 * 1024 * 1024 - body, err := io.ReadAll(io.LimitReader(resp.Body, maxBytes+1)) - if err != nil { - return nil, err - } - if int64(len(body)) > maxBytes { - return nil, fmt.Errorf("response from %s exceeds %d bytes", urlStr, maxBytes) - } - - // Signature verification gate. See Fetcher type docstring for the - // decision matrix. The transitional period accepts unsigned artifacts - // with a WARN log; flipping RequireSigs makes it strict-reject. - if err := f.verifyFetched(ctx, urlStr, body); err != nil { - return nil, fmt.Errorf("signature verification failed: %w", err) - } - - if f.Cache != nil { - if err := f.Cache.Write(urlStr, body); err != nil { - f.Logger.Warn("cache write failed; serving from response anyway", - "url", urlStr, "err", err) - } - } - return body, nil -} - -// verifyFetched fetches .sig and validates body against it. -// Returns nil only when the signature is present, well-formed, and -// verifies against f.VerifyKey. Any other outcome is a hard reject: -// the caller drops the body and the apps resolver falls through to -// the embedded copy. -// -// f.VerifyKey == nil means the operator hasn't configured an apps- -// pubkey. We reject every URL fetch in that state — the operator -// needs to opt in to a specific signing key explicitly. The reject -// error is informative so the WARN log line tells the operator -// exactly what to fix. -func (f *Fetcher) verifyFetched(ctx context.Context, urlStr string, body []byte) error { - if f.VerifyKey == nil { - return errors.New("ZDDC_APPS_PUBKEY is not configured; URL-fetched apps require an explicit signing key (see zddc.varasys.io/pubkey.pem for the canonical-channel key)") - } - sigURL := urlStr + ".sig" - req, err := http.NewRequestWithContext(ctx, http.MethodGet, sigURL, nil) - if err != nil { - return fmt.Errorf("build sig request for %s: %w", sigURL, err) - } - resp, err := f.Client.Do(req) - if err != nil { - return fmt.Errorf("fetch %s: %w", sigURL, err) - } - defer resp.Body.Close() - if resp.StatusCode != http.StatusOK { - return fmt.Errorf("%s returned HTTP %d", sigURL, resp.StatusCode) - } - - // Raw Ed25519 sig is 64 bytes; cap at a small limit so a hostile - // upstream can't flood us with a garbage "signature." - const maxSigBytes = 256 - sig, err := io.ReadAll(io.LimitReader(resp.Body, maxSigBytes+1)) - if err != nil { - return fmt.Errorf("read %s: %w", sigURL, err) - } - if len(sig) > maxSigBytes { - return fmt.Errorf("%s exceeds %d bytes", sigURL, maxSigBytes) - } - - if err := VerifyEd25519(f.VerifyKey, body, sig); err != nil { - // Verification failure is positive evidence of tampering or a - // build/key mismatch. Logged at WARN so operators see it; the - // resolver's existing embedded-fallback logging will note that - // the embedded copy is being served instead. - f.Logger.Warn("REJECTED: artifact signature does not verify", - "url", urlStr, "sig_url", sigURL, "err", err) - return err - } - f.Logger.Debug("artifact signature verified", "url", urlStr) - return nil -} - -// LogEmbeddedFallback emits a one-time warning when the embedded fallback -// is used for a particular source URL. Rate-limited per URL. -func (f *Fetcher) LogEmbeddedFallback(app, urlStr string, reason error) { - if _, loaded := f.embeddedFails.LoadOrStore(urlStr, struct{}{}); loaded { - return - } - f.Logger.Warn("serving embedded fallback for app HTML", - "app", app, "url", urlStr, "reason", reason) -} diff --git a/zddc/internal/apps/handler.go b/zddc/internal/apps/handler.go index 9ddc3ec..4ca0625 100644 --- a/zddc/internal/apps/handler.go +++ b/zddc/internal/apps/handler.go @@ -3,33 +3,36 @@ package apps import ( "crypto/sha256" "encoding/hex" - "errors" + "log/slog" "net/http" - "os" "path/filepath" "strings" "codeberg.org/VARASYS/ZDDC/zddc/internal/zddc" ) -// Server orchestrates app HTML resolution: subdir cascade override → fetch -// or path read → embedded fallback. It does NOT check whether the app is -// available at the request directory — that's AppAvailableAt's job, called -// from dispatch before invoking Serve. +// Server resolves tool HTML for a request: bundle member → embedded. The +// on-disk-at-path tier (operator override) is handled UPSTREAM by the +// dispatcher's stat-first static handler, so by the time Serve runs no real +// file exists at the path. Server does NOT decide whether the app is +// available at the directory — that's AppAvailableAt's job, called from +// dispatch before Serve. type Server struct { Root string - Cache *Cache - Fetcher *Fetcher BuildVer string // baked into X-ZDDC-Source for embedded responses + Bundle *Bundle + Logger *slog.Logger } -// NewServer constructs a Server. -func NewServer(root string, cache *Cache, fetcher *Fetcher, buildVer string) *Server { +// NewServer constructs a Server bound to the site-root config bundle. +func NewServer(root, buildVer string) *Server { + root = filepath.Clean(root) + logger := slog.Default() return &Server{ - Root: filepath.Clean(root), - Cache: cache, - Fetcher: fetcher, + Root: root, BuildVer: buildVer, + Bundle: NewBundle(root, logger), + Logger: logger, } } @@ -38,8 +41,8 @@ func NewServer(root string, cache *Cache, fetcher *Fetcher, buildVer string) *Se // directory (relative to root) the request is rooted at. The cmd/zddc- // server dispatcher calls this when stat fails on a URL: a missing file // that happens to look like `/archive.html` (or browse.html, etc.) -// resolves to the embedded app HTML for that directory — operators -// don't have to copy app HTML into every project. +// resolves to the embedded (or bundle) app HTML for that directory — +// operators don't have to copy app HTML into every project. // // Special case: GET / and GET /index.html both resolve to landing — the // only entry point that scopes ACL per-project, and the conventional @@ -72,105 +75,51 @@ func MatchAppHTML(requestPath string) (app string, requestDirRel string) { return "", "" } +// resolveBytes applies the local override precedence (tiers 2 then 3; tier 1 +// is handled upstream). Returns the HTML body, the X-ZDDC-Source tag, and +// whether to use the memoized embedded ETag (vs a body-hash ETag). +func (s *Server) resolveBytes(app string) (body []byte, sourceTag string, embedded, ok bool) { + if s.Bundle != nil { + if b, found := s.Bundle.Member(app + ".html"); found { + return b, "bundle:" + app + ".html", false, true + } + } + if b := EmbeddedBytes(app); len(b) > 0 { + return b, "embedded:" + app + "@" + s.BuildVer, true, true + } + return nil, "", false, false +} + // Serve resolves and writes the response. Caller has already verified: -// - no real file exists at the request path +// - no real file exists at the request path (so tier 1 didn't apply) // - AppAvailableAt(root, requestDir, app) is true // - ACL passes for requestDir // -// Honors a `?v=` query parameter as a per-request override on top of -// the cascade. With `?v=` set, the resolved URL must already exist in the -// cache — otherwise the response is 404. This prevents users from -// triggering arbitrary upstream fetches via URL-crafted requests; only -// versions the operator's `.zddc apps:` entries have already pulled in -// (or that the user has manually placed in `_app/`) are reachable. -func (s *Server) Serve(w http.ResponseWriter, r *http.Request, app string, chain zddc.PolicyChain, requestDir string) { - vSpec := strings.TrimSpace(r.URL.Query().Get("v")) - - src, hasOverride, err := ResolveWithOverride(chain, app, s.Root, requestDir, vSpec) - if err != nil { - // `?v=` parsing/validation errors are user input → 400. - if vSpec != "" { - http.Error(w, "400 Bad Request — invalid ?v= value: "+err.Error(), http.StatusBadRequest) - return - } - // Malformed `.zddc` spec — operator's fault. Log and serve embedded. - s.Fetcher.Logger.Warn("apps.Resolve failed; serving embedded", - "app", app, "request_dir", requestDir, "err", err) - s.serveEmbedded(w, r, app, err) +// chain and requestDir are retained in the signature for call-site stability +// and future per-directory resolution; the current local model is path- +// independent (a bundle member or the embedded default). +func (s *Server) Serve(w http.ResponseWriter, r *http.Request, app string, _ zddc.PolicyChain, _ string) { + body, tag, embedded, ok := s.resolveBytes(app) + if !ok { + w.Header().Set("Retry-After", "60") + http.Error(w, + "503 Service Unavailable\n\n"+ + "This zddc-server has no embedded fallback for "+app+" and no\n"+ + "\""+app+".html\" in the site .zddc.zip bundle.\n"+ + "Rebuild the binary against the latest tool HTMLs, or add the\n"+ + "file to .zddc.zip.\n", + http.StatusServiceUnavailable) return } - - if !hasOverride { - // No `.zddc apps:` entry anywhere up the chain and no `?v=` either → - // embedded is the authoritative default. - s.serveEmbedded(w, r, app, nil) - return + etag := bodyETag(body) + if embedded { + etag = EmbeddedETag(app) } - - // Per-request `?v=` is restricted to cache-backed URL sources. - if vSpec != "" { - if !src.IsURL() { - http.Error(w, "400 Bad Request — ?v= requires a URL-form spec", http.StatusBadRequest) - return - } - if s.Cache == nil || !s.Cache.Has(src.URL) { - http.Error(w, - "404 Not Found — version requested via ?v= is not in the local cache.\n"+ - "Only versions the deployment has already fetched (via .zddc apps: entries) are servable.\n"+ - "Asked for: "+src.URL+"\n", - http.StatusNotFound) - return - } - body, err := s.Cache.Read(src.URL) - if err != nil { - s.Fetcher.Logger.Warn("?v= cache read failed", "url", src.URL, "err", err) - http.Error(w, "500 Internal Server Error", http.StatusInternalServerError) - return - } - s.serveBody(w, r, body, "cache:"+src.URL) - return - } - - if !src.IsURL() { - // Path source: read directly, no cache. - body, err := os.ReadFile(src.Path) - if err != nil { - if errors.Is(err, os.ErrNotExist) { - s.Fetcher.Logger.Warn("path source missing; serving embedded", - "app", app, "path", src.Path) - } else { - s.Fetcher.Logger.Warn("path source unreadable; serving embedded", - "app", app, "path", src.Path, "err", err) - } - s.serveEmbedded(w, r, app, err) - return - } - s.serveBody(w, r, body, "path:"+src.Path) - return - } - - // URL source: cache hit serves immediately; cache miss fetches once. - body, err := s.Fetcher.Fetch(r.Context(), src.URL) - if err != nil { - s.Fetcher.LogEmbeddedFallback(app, src.URL, err) - s.serveEmbedded(w, r, app, err) - return - } - sourceTag := "fetch:" + src.URL - if s.Cache != nil && s.Cache.Has(src.URL) { - // Likely served from cache (Has was true when the read started). - // Distinguishing cache-hit from just-fetched is best-effort here. - sourceTag = "cache:" + src.URL - } - s.serveBody(w, r, body, sourceTag) + writeWithETag(w, r, body, etag, "text/html; charset=utf-8", tag) } -// writeWithETag writes body with a strong ETag derived from `etag`, the -// cache-friendly headers, and short-circuits to 304 Not Modified when the -// client's `If-None-Match` matches. `max-age=0, must-revalidate` means the -// browser revalidates on every load — and the matching ETag returns 304 -// with empty body, so the steady-state cost of a reload is ~200 bytes -// instead of the full HTML payload (50–920 KB depending on the tool). +// writeWithETag writes body with a strong ETag, cache-friendly headers, and +// short-circuits to 304 Not Modified when the client's If-None-Match matches. func writeWithETag(w http.ResponseWriter, r *http.Request, body []byte, etag, contentType, sourceHeader string) { quotedTag := `"` + etag + `"` w.Header().Set("ETag", quotedTag) @@ -185,30 +134,8 @@ func writeWithETag(w http.ResponseWriter, r *http.Request, body []byte, etag, co _, _ = w.Write(body) } -// bodyETag computes a stable 32-hex-char ETag for an arbitrary body. Used -// for the URL/path-sourced response path (the bytes vary per cache-fetch -// or per file read, so memoizing per-app would be wrong). +// bodyETag computes a stable 32-hex-char ETag for an arbitrary body. func bodyETag(body []byte) string { sum := sha256.Sum256(body) return hex.EncodeToString(sum[:])[:32] } - -func (s *Server) serveBody(w http.ResponseWriter, r *http.Request, body []byte, sourceHeader string) { - writeWithETag(w, r, body, bodyETag(body), "text/html; charset=utf-8", sourceHeader) -} - -func (s *Server) serveEmbedded(w http.ResponseWriter, r *http.Request, app string, _ error) { - body := EmbeddedBytes(app) - if len(body) == 0 { - w.Header().Set("Retry-After", "60") - http.Error(w, - "503 Service Unavailable\n\n"+ - "This zddc-server has no embedded fallback for "+app+".\n"+ - "Rebuild the binary against the latest tool HTMLs.\n", - http.StatusServiceUnavailable) - return - } - writeWithETag(w, r, body, EmbeddedETag(app), - "text/html; charset=utf-8", - "embedded:"+app+"@"+s.BuildVer) -} diff --git a/zddc/internal/apps/handler_test.go b/zddc/internal/apps/handler_test.go index 537e8a5..24cc1c6 100644 --- a/zddc/internal/apps/handler_test.go +++ b/zddc/internal/apps/handler_test.go @@ -1,47 +1,14 @@ package apps import ( - "crypto/ed25519" - "crypto/rand" "net/http" "net/http/httptest" - "net/url" - "os" - "path/filepath" "strings" - "sync/atomic" "testing" "codeberg.org/VARASYS/ZDDC/zddc/internal/zddc" ) -// signedFixture returns a (publicKey, handler) pair where the handler -// serves `body` for any URL ending in `.html` and the corresponding -// Ed25519 signature for the same URL with `.sig` appended. Tests use -// this to stand up upstream stubs that exercise the apps fetcher's -// strict signature-verification path. -// -// All tests share one pattern: the fetcher's VerifyKey gets overridden -// to this fixture's publicKey so verification passes against the -// fixture's signature instead of the production embedded key. -func signedFixture(t *testing.T, body []byte) (ed25519.PublicKey, http.HandlerFunc) { - t.Helper() - pub, priv, err := ed25519.GenerateKey(rand.Reader) - if err != nil { - t.Fatalf("GenerateKey: %v", err) - } - sig := ed25519.Sign(priv, body) - handler := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { - switch { - case strings.HasSuffix(r.URL.Path, ".sig"): - _, _ = w.Write(sig) - default: - _, _ = w.Write(body) - } - }) - return pub, handler -} - func TestMatchAppHTML(t *testing.T) { cases := []struct { path, wantApp, wantDir string @@ -63,34 +30,21 @@ func TestMatchAppHTML(t *testing.T) { } } -// Build a Server with a fake upstream serving body. The upstream -// also publishes a valid Ed25519 signature alongside (.sig) and the -// fetcher's VerifyKey is overridden to the matching test pubkey so -// fetched bytes pass the strict-signature gate. -func newTestServer(t *testing.T, body []byte) (*Server, *httptest.Server, string) { - t.Helper() - pub, handler := signedFixture(t, body) - upstream := httptest.NewServer(handler) - t.Cleanup(upstream.Close) - root := t.TempDir() - cache, err := NewCache(filepath.Join(root, CacheDirName)) - if err != nil { - t.Fatal(err) - } - f := NewFetcher(cache, nil) - f.VerifyKey = pub - return NewServer(root, cache, f, "test"), upstream, root +// serve runs srv.Serve for app and returns the recorder. +func serve(srv *Server, app string) *httptest.ResponseRecorder { + rec := httptest.NewRecorder() + chain := zddc.PolicyChain{Levels: []zddc.ZddcFile{{}}} + srv.Serve(rec, httptest.NewRequest(http.MethodGet, "/"+app+".html", nil), app, chain, srv.Root) + return rec } -func TestServer_NoOverride_ServesEmbedded(t *testing.T) { - srv, _, root := newTestServer(t, []byte("upstream body")) +func TestServer_NoBundle_ServesEmbedded(t *testing.T) { + srv := NewServer(t.TempDir(), "test") saved := embeddedArchive embeddedArchive = []byte("EMBEDDED archive") defer func() { embeddedArchive = saved }() - chain := zddc.PolicyChain{Levels: []zddc.ZddcFile{{}}} - rec := httptest.NewRecorder() - srv.Serve(rec, httptest.NewRequest(http.MethodGet, "/archive.html", nil), "archive", chain, root) + rec := serve(srv, "archive") if rec.Code != http.StatusOK { t.Fatalf("status=%d", rec.Code) } @@ -102,266 +56,60 @@ func TestServer_NoOverride_ServesEmbedded(t *testing.T) { } } -func TestServer_OverrideURL_FetchesAndCaches(t *testing.T) { - body := []byte("from upstream") - srv, up, root := newTestServer(t, body) - chain := zddc.PolicyChain{ - Levels: []zddc.ZddcFile{{ - Apps: map[string]string{"archive": up.URL + "/archive_stable.html"}, - }}, - } - rec := httptest.NewRecorder() - srv.Serve(rec, httptest.NewRequest(http.MethodGet, "/archive.html", nil), "archive", chain, root) - if rec.Code != http.StatusOK { - t.Fatalf("status=%d", rec.Code) - } - if rec.Body.String() != string(body) { - t.Errorf("body mismatch") - } - // Cache should be populated. - if !srv.Cache.Has(up.URL + "/archive_stable.html") { - t.Errorf("cache miss after fetch") - } -} - -func TestServer_OverrideURL_CacheHitOnSecondCall(t *testing.T) { - var hits atomic.Int64 - body := []byte("body") - pub, _, sig := func() (ed25519.PublicKey, ed25519.PrivateKey, []byte) { - p, k, err := ed25519.GenerateKey(rand.Reader) - if err != nil { - t.Fatalf("GenerateKey: %v", err) - } - return p, k, ed25519.Sign(k, body) - }() - upstream := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { - // Count only artifact fetches (not .sig fetches) so the assertion - // "1 hit means cache works" stays meaningful: cache stores the - // artifact body, signature verification re-runs each time the - // resolver hits the URL but only on the first miss does it fetch - // the artifact bytes itself. After that, cache.Read short-circuits. - if !strings.HasSuffix(r.URL.Path, ".sig") { - hits.Add(1) - _, _ = w.Write(body) - return - } - _, _ = w.Write(sig) - })) - defer upstream.Close() - +func TestServer_BundleMemberOverridesEmbedded(t *testing.T) { root := t.TempDir() - cache, _ := NewCache(filepath.Join(root, CacheDirName)) - f := NewFetcher(cache, nil) - f.VerifyKey = pub - srv := NewServer(root, cache, f, "test") - - chain := zddc.PolicyChain{Levels: []zddc.ZddcFile{{ - Apps: map[string]string{"archive": upstream.URL + "/archive_stable.html"}, - }}} - for i := 0; i < 3; i++ { - rec := httptest.NewRecorder() - srv.Serve(rec, httptest.NewRequest(http.MethodGet, "/archive.html", nil), "archive", chain, root) - if rec.Code != http.StatusOK { - t.Fatalf("call %d status=%d", i, rec.Code) - } - } - if hits.Load() != 1 { - t.Errorf("upstream fetched %d times, want exactly 1 (cache forever)", hits.Load()) - } -} - -func TestServer_PathOverride_ServedDirectly(t *testing.T) { - root := t.TempDir() - pathFile := filepath.Join(root, "local.html") - body := []byte("local archive bytes") - if err := os.WriteFile(pathFile, body, 0o644); err != nil { - t.Fatal(err) - } - cache, _ := NewCache(filepath.Join(root, CacheDirName)) - f := NewFetcher(cache, nil) - srv := NewServer(root, cache, f, "test") - - chain := zddc.PolicyChain{Levels: []zddc.ZddcFile{ - {Apps: map[string]string{"archive": "./local.html"}}, - }} - rec := httptest.NewRecorder() - srv.Serve(rec, httptest.NewRequest(http.MethodGet, "/archive.html", nil), "archive", chain, root) - if rec.Code != http.StatusOK { - t.Fatalf("status=%d", rec.Code) - } - if rec.Body.String() != string(body) { - t.Errorf("body mismatch") - } - if !strings.HasPrefix(rec.Header().Get("X-ZDDC-Source"), "path:") { - t.Errorf("X-ZDDC-Source=%q", rec.Header().Get("X-ZDDC-Source")) - } -} - -func TestServer_FetchFailFallsBackToEmbedded(t *testing.T) { - srv, _, root := newTestServer(t, []byte("ok")) + writeTestBundle(t, root, map[string]string{"archive.html": "BUNDLE archive override"}) + srv := NewServer(root, "test") saved := embeddedArchive - embeddedArchive = []byte("EMBEDDED") + embeddedArchive = []byte("EMBEDDED archive") defer func() { embeddedArchive = saved }() - chain := zddc.PolicyChain{Levels: []zddc.ZddcFile{{ - Apps: map[string]string{"archive": "https://no-such.example/archive.html"}, - }}} - rec := httptest.NewRecorder() - srv.Serve(rec, httptest.NewRequest(http.MethodGet, "/archive.html", nil), "archive", chain, root) + rec := serve(srv, "archive") if rec.Code != http.StatusOK { - t.Fatalf("status=%d (want 200 from embedded)", rec.Code) + t.Fatalf("status=%d", rec.Code) } + if !strings.Contains(rec.Body.String(), "BUNDLE archive override") { + t.Errorf("expected bundle body, got %q", rec.Body.String()) + } + if rec.Header().Get("X-ZDDC-Source") != "bundle:archive.html" { + t.Errorf("X-ZDDC-Source=%q, want bundle:archive.html", rec.Header().Get("X-ZDDC-Source")) + } +} + +func TestServer_BundlePresent_MemberAbsent_ServesEmbedded(t *testing.T) { + root := t.TempDir() + writeTestBundle(t, root, map[string]string{"browse.html": "BUNDLE browse"}) + srv := NewServer(root, "test") + saved := embeddedArchive + embeddedArchive = []byte("EMBEDDED archive") + defer func() { embeddedArchive = saved }() + + rec := serve(srv, "archive") // bundle has browse, not archive if !strings.Contains(rec.Body.String(), "EMBEDDED") { - t.Errorf("body did not come from embedded fallback: %q", rec.Body.String()) + t.Errorf("expected embedded fallback, got %q", rec.Body.String()) } } -// ── ?v= per-request override ───────────────────────────────────────────── - -func TestServer_VParam_CacheHitServesFromCache(t *testing.T) { - srv, _, root := newTestServer(t, []byte("ignored")) - // Pre-populate the cache with a known URL. - cachedURL := "https://zddc.varasys.io/releases/archive_v0.0.4.html" - cachedBody := []byte("CACHED v0.0.4 archive") - if err := srv.Cache.Write(cachedURL, cachedBody); err != nil { - t.Fatal(err) - } - - chain := zddc.PolicyChain{Levels: []zddc.ZddcFile{{}}} - rec := httptest.NewRecorder() - srv.Serve(rec, httptest.NewRequest(http.MethodGet, "/archive.html?v=v0.0.4", nil), "archive", chain, root) - if rec.Code != http.StatusOK { - t.Fatalf("status=%d body=%s", rec.Code, rec.Body.String()) - } - if rec.Body.String() != string(cachedBody) { - t.Errorf("body=%q, want CACHED bytes", rec.Body.String()) - } - if got := rec.Header().Get("X-ZDDC-Source"); got != "cache:"+cachedURL { - t.Errorf("X-ZDDC-Source=%q", got) +func TestServer_UnknownTool_503WithoutBundle(t *testing.T) { + srv := NewServer(t.TempDir(), "test") + rec := serve(srv, "nope") // not embedded, no bundle + if rec.Code != http.StatusServiceUnavailable { + t.Errorf("status=%d, want 503", rec.Code) } } -func TestServer_VParam_CacheMissReturns404(t *testing.T) { - srv, _, root := newTestServer(t, []byte("ignored")) - chain := zddc.PolicyChain{Levels: []zddc.ZddcFile{{}}} - rec := httptest.NewRecorder() - srv.Serve(rec, httptest.NewRequest(http.MethodGet, "/archive.html?v=v0.0.4", nil), "archive", chain, root) - if rec.Code != http.StatusNotFound { - t.Fatalf("status=%d (want 404)", rec.Code) - } - if !strings.Contains(rec.Body.String(), "not in the local cache") { - t.Errorf("body should explain cache miss, got %q", rec.Body.String()) - } -} - -func TestServer_VParam_RejectsPathSource(t *testing.T) { - srv, _, root := newTestServer(t, []byte("ignored")) - chain := zddc.PolicyChain{Levels: []zddc.ZddcFile{{}}} - rec := httptest.NewRecorder() - srv.Serve(rec, httptest.NewRequest(http.MethodGet, "/archive.html?v=./local.html", nil), "archive", chain, root) - if rec.Code != http.StatusBadRequest { - t.Errorf("status=%d (want 400 for path source via ?v=)", rec.Code) - } -} - -func TestServer_VParam_BadSpecReturns400(t *testing.T) { - srv, _, root := newTestServer(t, []byte("ignored")) - chain := zddc.PolicyChain{Levels: []zddc.ZddcFile{{}}} - rec := httptest.NewRecorder() - srv.Serve(rec, httptest.NewRequest(http.MethodGet, "/archive.html?v=not%20a%20spec", nil), "archive", chain, root) - if rec.Code != http.StatusBadRequest { - t.Errorf("status=%d (want 400)", rec.Code) - } -} - -func TestServer_VParam_CombinesWithCascadeURLPrefix(t *testing.T) { - // Cascade has a default URL prefix; ?v=:beta should resolve against it. - srv, _, root := newTestServer(t, []byte("ignored")) - cachedURL := "https://my-mirror.example/releases/archive_v0.0.4.html" - if err := srv.Cache.Write(cachedURL, []byte("MIRROR v0.0.4")); err != nil { - t.Fatal(err) - } - chain := zddc.PolicyChain{Levels: []zddc.ZddcFile{{ - Apps: map[string]string{"default": "https://my-mirror.example/releases:stable"}, - }}} - rec := httptest.NewRecorder() - srv.Serve(rec, httptest.NewRequest(http.MethodGet, "/archive.html?v=:v0.0.4", nil), "archive", chain, root) - if rec.Code != http.StatusOK { - t.Fatalf("status=%d body=%s", rec.Code, rec.Body.String()) - } - if rec.Body.String() != "MIRROR v0.0.4" { - t.Errorf("body=%q", rec.Body.String()) - } - if got := rec.Header().Get("X-ZDDC-Source"); got != "cache:"+cachedURL { - t.Errorf("X-ZDDC-Source=%q (expected mirror URL)", got) - } -} - -func TestServer_VParam_OverridesPathTerminalFromCascade(t *testing.T) { - // Operator's cascade specifies a path source. User passes ?v=stable. - // ?v= overrides → resolves to canonical/archive.html, then cache check. - srv, _, root := newTestServer(t, []byte("ignored")) - cachedURL := "https://zddc.varasys.io/releases/archive.html" - if err := srv.Cache.Write(cachedURL, []byte("CACHED stable")); err != nil { - t.Fatal(err) - } - pathFile := filepath.Join(root, "operator-version.html") - if err := os.WriteFile(pathFile, []byte("OPERATOR PATH"), 0o644); err != nil { - t.Fatal(err) - } - chain := zddc.PolicyChain{Levels: []zddc.ZddcFile{{ - Apps: map[string]string{"archive": "./operator-version.html"}, - }}} - rec := httptest.NewRecorder() - srv.Serve(rec, httptest.NewRequest(http.MethodGet, "/archive.html?v=stable", nil), "archive", chain, root) - if rec.Code != http.StatusOK { - t.Fatalf("status=%d body=%s", rec.Code, rec.Body.String()) - } - if rec.Body.String() != "CACHED stable" { - t.Errorf("body=%q (expected ?v= override to win)", rec.Body.String()) - } -} - -func TestServer_VParam_FullURLForm(t *testing.T) { - // `?v=https://my-fork/archive.html` — terminal full URL, must be cached. - srv, _, root := newTestServer(t, []byte("ignored")) - cachedURL := "https://my-fork.example/custom.html" - if err := srv.Cache.Write(cachedURL, []byte("FORK custom")); err != nil { - t.Fatal(err) - } - chain := zddc.PolicyChain{Levels: []zddc.ZddcFile{{}}} - target := "/archive.html?v=" + url.QueryEscape(cachedURL) - rec := httptest.NewRecorder() - srv.Serve(rec, httptest.NewRequest(http.MethodGet, target, nil), "archive", chain, root) - if rec.Code != http.StatusOK { - t.Fatalf("status=%d body=%s", rec.Code, rec.Body.String()) - } - if rec.Body.String() != "FORK custom" { - t.Errorf("body=%q", rec.Body.String()) - } -} - -// TestServer_Embedded_ConditionalGET verifies the ETag/If-None-Match dance -// for the embedded fallback path: a fresh GET returns 200 with an ETag, -// and a follow-up with a matching If-None-Match returns 304 + empty body. -// This is the cache-friendliness fix that lets a browser revalidate -// against zddc-server's embedded HTML without re-transferring the bytes. func TestServer_Embedded_ConditionalGET(t *testing.T) { - srv, _, root := newTestServer(t, []byte("upstream")) + srv := NewServer(t.TempDir(), "test") saved := embeddedArchive embeddedArchive = []byte("EMBEDDED archive bytes for ETag test") defer func() { embeddedArchive = saved - etagCacheByApp.Delete("archive") // reset memoization for sibling tests + etagCacheByApp.Delete("archive") }() - etagCacheByApp.Delete("archive") // ensure clean state for THIS test + etagCacheByApp.Delete("archive") - chain := zddc.PolicyChain{Levels: []zddc.ZddcFile{{}}} - - // First request: full body + ETag header. - rec1 := httptest.NewRecorder() - srv.Serve(rec1, httptest.NewRequest(http.MethodGet, "/archive.html", nil), "archive", chain, root) + rec1 := serve(srv, "archive") if rec1.Code != http.StatusOK { t.Fatalf("first GET: status=%d body=%s", rec1.Code, rec1.Body.String()) } @@ -370,17 +118,15 @@ func TestServer_Embedded_ConditionalGET(t *testing.T) { t.Fatalf("first GET: missing ETag header") } if cc := rec1.Header().Get("Cache-Control"); !strings.Contains(cc, "max-age=0") || !strings.Contains(cc, "must-revalidate") { - t.Errorf("first GET: Cache-Control=%q (want max-age=0 + must-revalidate)", cc) - } - if !strings.Contains(rec1.Body.String(), "EMBEDDED archive bytes") { - t.Errorf("first GET: body=%q", rec1.Body.String()) + t.Errorf("first GET: Cache-Control=%q", cc) } - // Second request with matching If-None-Match: 304, empty body. + // Matching If-None-Match → 304, empty body. rec2 := httptest.NewRecorder() req2 := httptest.NewRequest(http.MethodGet, "/archive.html", nil) req2.Header.Set("If-None-Match", etag) - srv.Serve(rec2, req2, "archive", chain, root) + chain := zddc.PolicyChain{Levels: []zddc.ZddcFile{{}}} + srv.Serve(rec2, req2, "archive", chain, srv.Root) if rec2.Code != http.StatusNotModified { t.Fatalf("If-None-Match match: status=%d (want 304)", rec2.Code) } @@ -388,16 +134,34 @@ func TestServer_Embedded_ConditionalGET(t *testing.T) { t.Errorf("304 response should have empty body; got %d bytes", rec2.Body.Len()) } - // Third request with stale If-None-Match: 200, full body. + // Stale If-None-Match → 200, full body. rec3 := httptest.NewRecorder() req3 := httptest.NewRequest(http.MethodGet, "/archive.html", nil) req3.Header.Set("If-None-Match", `"deadbeef"`) - srv.Serve(rec3, req3, "archive", chain, root) - if rec3.Code != http.StatusOK { - t.Errorf("stale If-None-Match: status=%d (want 200)", rec3.Code) + srv.Serve(rec3, req3, "archive", chain, srv.Root) + if rec3.Code != http.StatusOK || rec3.Body.Len() == 0 { + t.Errorf("stale If-None-Match: status=%d bodyLen=%d (want 200, non-empty)", rec3.Code, rec3.Body.Len()) } - if rec3.Body.Len() == 0 { - t.Errorf("stale If-None-Match: empty body; want full") +} + +// Bundle responses get a body-hash ETag and also short-circuit to 304. +func TestServer_Bundle_ConditionalGET(t *testing.T) { + root := t.TempDir() + writeTestBundle(t, root, map[string]string{"browse.html": "BUNDLE browse body"}) + srv := NewServer(root, "test") + + rec1 := serve(srv, "browse") + etag := rec1.Header().Get("ETag") + if rec1.Code != http.StatusOK || etag == "" { + t.Fatalf("first GET: status=%d etag=%q", rec1.Code, etag) + } + rec2 := httptest.NewRecorder() + req2 := httptest.NewRequest(http.MethodGet, "/browse.html", nil) + req2.Header.Set("If-None-Match", etag) + chain := zddc.PolicyChain{Levels: []zddc.ZddcFile{{}}} + srv.Serve(rec2, req2, "browse", chain, srv.Root) + if rec2.Code != http.StatusNotModified { + t.Errorf("bundle If-None-Match: status=%d (want 304)", rec2.Code) } } @@ -422,6 +186,6 @@ func TestEmbeddedETag_Stable(t *testing.T) { etagCacheByApp.Delete("archive") b := EmbeddedETag("archive") if b == a1 { - t.Errorf("EmbeddedETag should differ for different bytes; both %q", b) + t.Errorf("EmbeddedETag should change with bytes; both %q", b) } } diff --git a/zddc/internal/apps/singleflight.go b/zddc/internal/apps/singleflight.go deleted file mode 100644 index e60517e..0000000 --- a/zddc/internal/apps/singleflight.go +++ /dev/null @@ -1,43 +0,0 @@ -package apps - -import "sync" - -// singleflightGroup deduplicates concurrent calls keyed by string. If N -// goroutines call Do(key, fn) before the first one returns, fn runs once -// and all callers receive the same (val, err). -// -// Hand-rolled to avoid pulling in golang.org/x/sync — we only need the -// 30-line core, not Forget/DoChan. Pattern is the standard one. -type singleflightGroup struct { - mu sync.Mutex - m map[string]*sfCall -} - -type sfCall struct { - done chan struct{} - val any - err error -} - -func (g *singleflightGroup) Do(key string, fn func() (any, error)) (any, error) { - g.mu.Lock() - if g.m == nil { - g.m = make(map[string]*sfCall) - } - if c, ok := g.m[key]; ok { - g.mu.Unlock() - <-c.done - return c.val, c.err - } - c := &sfCall{done: make(chan struct{})} - g.m[key] = c - g.mu.Unlock() - - c.val, c.err = fn() - close(c.done) - - g.mu.Lock() - delete(g.m, key) - g.mu.Unlock() - return c.val, c.err -} diff --git a/zddc/internal/apps/singleflight_test.go b/zddc/internal/apps/singleflight_test.go deleted file mode 100644 index 42fd961..0000000 --- a/zddc/internal/apps/singleflight_test.go +++ /dev/null @@ -1,67 +0,0 @@ -package apps - -import ( - "sync" - "sync/atomic" - "testing" - "time" -) - -func TestSingleflightDedupes(t *testing.T) { - var g singleflightGroup - var calls atomic.Int64 - fn := func() (any, error) { - calls.Add(1) - time.Sleep(50 * time.Millisecond) // hold the lock long enough for races - return "result", nil - } - var wg sync.WaitGroup - const N = 50 - for i := 0; i < N; i++ { - wg.Add(1) - go func() { - defer wg.Done() - val, err := g.Do("the-key", fn) - if err != nil { - t.Errorf("Do err: %v", err) - return - } - if val.(string) != "result" { - t.Errorf("got %v, want 'result'", val) - } - }() - } - wg.Wait() - if got := calls.Load(); got != 1 { - t.Errorf("fn called %d times, want exactly 1", got) - } -} - -func TestSingleflightDifferentKeysParallel(t *testing.T) { - var g singleflightGroup - var calls atomic.Int64 - fn := func() (any, error) { - calls.Add(1) - return "ok", nil - } - for _, k := range []string{"a", "b", "c"} { - _, _ = g.Do(k, fn) - } - if got := calls.Load(); got != 3 { - t.Errorf("fn called %d times, want 3", got) - } -} - -func TestSingleflightSecondCallAfterFirstResolves(t *testing.T) { - var g singleflightGroup - var calls atomic.Int64 - fn := func() (any, error) { - calls.Add(1) - return "x", nil - } - _, _ = g.Do("k", fn) - _, _ = g.Do("k", fn) - if got := calls.Load(); got != 2 { - t.Errorf("fn called %d times, want 2 (second call sees no in-flight entry)", got) - } -} diff --git a/zddc/internal/apps/verify.go b/zddc/internal/apps/verify.go deleted file mode 100644 index 63168aa..0000000 --- a/zddc/internal/apps/verify.go +++ /dev/null @@ -1,73 +0,0 @@ -package apps - -import ( - "crypto/ed25519" - "crypto/x509" - "encoding/pem" - "errors" - "fmt" - "os" -) - -// LoadPubKey reads a PEM-encoded SubjectPublicKeyInfo (the format -// `openssl pkey -pubout` emits) from path and returns the underlying -// Ed25519 public key. -// -// Operators distribute and configure this key explicitly — same posture -// as the TLS certificate: zddc-server bakes nothing in. Customers -// running against zddc.varasys.io's release channel download the -// canonical key from zddc.varasys.io/pubkey.pem and pass the local -// path via --apps-pubkey or ZDDC_APPS_PUBKEY. Customers running their -// own signing infrastructure pass their own public key instead. -// -// Returns a descriptive error for missing files, malformed PEM, wrong -// PEM type, or non-Ed25519 keys. Callers (cmd/zddc-server's startup -// path) treat any error as fatal — refusing to start with a misconfigured -// apps-pubkey is the right posture. -func LoadPubKey(path string) (ed25519.PublicKey, error) { - data, err := os.ReadFile(path) - if err != nil { - return nil, fmt.Errorf("read apps-pubkey from %s: %w", path, err) - } - return ParsePubKeyPEM(data) -} - -// ParsePubKeyPEM is LoadPubKey's content-only variant. Useful when the -// PEM bytes come from somewhere other than disk (test fixtures, etc.). -func ParsePubKeyPEM(pemBytes []byte) (ed25519.PublicKey, error) { - block, _ := pem.Decode(pemBytes) - if block == nil { - return nil, errors.New("no PEM block found") - } - if block.Type != "PUBLIC KEY" { - return nil, fmt.Errorf("unexpected PEM type %q (want PUBLIC KEY)", block.Type) - } - pub, err := x509.ParsePKIXPublicKey(block.Bytes) - if err != nil { - return nil, fmt.Errorf("parse PKIX: %w", err) - } - edPub, ok := pub.(ed25519.PublicKey) - if !ok { - return nil, fmt.Errorf("public key is not Ed25519 (got %T)", pub) - } - return edPub, nil -} - -// VerifyEd25519 checks that sig is a valid Ed25519 signature of body -// produced with the private key matching pub. Returns nil on success -// or a descriptive error otherwise. -// -// sig must be exactly 64 bytes (the raw Ed25519 signature format -// produced by `openssl pkeyutl -sign -rawin`). -func VerifyEd25519(pub ed25519.PublicKey, body, sig []byte) error { - if pub == nil { - return errors.New("no public key configured") - } - if len(sig) != ed25519.SignatureSize { - return fmt.Errorf("signature has wrong length: %d (want %d)", len(sig), ed25519.SignatureSize) - } - if !ed25519.Verify(pub, body, sig) { - return errors.New("signature does not verify against trusted public key") - } - return nil -} diff --git a/zddc/internal/apps/verify_test.go b/zddc/internal/apps/verify_test.go deleted file mode 100644 index 1e9b5f9..0000000 --- a/zddc/internal/apps/verify_test.go +++ /dev/null @@ -1,255 +0,0 @@ -package apps - -import ( - "context" - "crypto/ed25519" - "crypto/rand" - "crypto/x509" - "encoding/pem" - "net/http" - "net/http/httptest" - "path/filepath" - "strings" - "testing" -) - -// genTestKey returns a fresh Ed25519 keypair for tests so the test -// suite never depends on the embedded production key. -func genTestKey(t *testing.T) (ed25519.PublicKey, ed25519.PrivateKey) { - t.Helper() - pub, priv, err := ed25519.GenerateKey(rand.Reader) - if err != nil { - t.Fatalf("GenerateKey: %v", err) - } - return pub, priv -} - -func TestParseEd25519PublicKeyPEM_RoundTrip(t *testing.T) { - pub, _ := genTestKey(t) - derBytes, err := x509.MarshalPKIXPublicKey(pub) - if err != nil { - t.Fatalf("MarshalPKIXPublicKey: %v", err) - } - pemBytes := pem.EncodeToMemory(&pem.Block{Type: "PUBLIC KEY", Bytes: derBytes}) - - parsed, err := ParsePubKeyPEM(pemBytes) - if err != nil { - t.Fatalf("parse: %v", err) - } - if !pub.Equal(parsed) { - t.Errorf("round-trip pubkey mismatch") - } -} - -func TestParseEd25519PublicKeyPEM_RejectsRSA(t *testing.T) { - // PEM containing a non-Ed25519 key should error rather than - // silently coerce. Use a hand-crafted bad PEM block. - bad := pem.EncodeToMemory(&pem.Block{Type: "PUBLIC KEY", Bytes: []byte("not a valid SubjectPublicKeyInfo")}) - if _, err := ParsePubKeyPEM(bad); err == nil { - t.Error("ParsePubKeyPEM accepted malformed PEM, want error") - } -} - -func TestParseEd25519PublicKeyPEM_RejectsWrongType(t *testing.T) { - pub, _ := genTestKey(t) - derBytes, _ := x509.MarshalPKIXPublicKey(pub) - wrongType := pem.EncodeToMemory(&pem.Block{Type: "RSA PUBLIC KEY", Bytes: derBytes}) - if _, err := ParsePubKeyPEM(wrongType); err == nil { - t.Error("ParsePubKeyPEM accepted wrong PEM Type, want error") - } -} - -func TestVerifyEd25519_ValidSignature(t *testing.T) { - pub, priv := genTestKey(t) - msg := []byte("the artifact bytes") - sig := ed25519.Sign(priv, msg) - if err := VerifyEd25519(pub, msg, sig); err != nil { - t.Errorf("VerifyEd25519 rejected a valid signature: %v", err) - } -} - -func TestVerifyEd25519_TamperedMessage(t *testing.T) { - pub, priv := genTestKey(t) - original := []byte("the artifact bytes") - tampered := []byte("the artifact byteX") - sig := ed25519.Sign(priv, original) - if err := VerifyEd25519(pub, tampered, sig); err == nil { - t.Error("VerifyEd25519 accepted a tampered message, want error") - } -} - -func TestVerifyEd25519_WrongKey(t *testing.T) { - _, priv := genTestKey(t) - otherPub, _ := genTestKey(t) - msg := []byte("the artifact bytes") - sig := ed25519.Sign(priv, msg) - if err := VerifyEd25519(otherPub, msg, sig); err == nil { - t.Error("VerifyEd25519 accepted a signature from the wrong key, want error") - } -} - -func TestVerifyEd25519_MalformedSignature(t *testing.T) { - pub, _ := genTestKey(t) - msg := []byte("hello") - cases := [][]byte{ - nil, // empty - make([]byte, 32), // too short - make([]byte, 100), // too long - make([]byte, 64), // right length, wrong contents - } - for i, sig := range cases { - if err := VerifyEd25519(pub, msg, sig); err == nil { - t.Errorf("case %d: VerifyEd25519 accepted malformed signature of length %d, want error", i, len(sig)) - } - } -} - -func TestVerifyEd25519_NilKey(t *testing.T) { - if err := VerifyEd25519(nil, []byte("x"), make([]byte, 64)); err == nil { - t.Error("VerifyEd25519(nil, ...) accepted, want error") - } -} - -// TestFetcher_AcceptsValidSignature: end-to-end. Server publishes -// an artifact and a valid .sig; fetcher accepts and caches. -func TestFetcher_AcceptsValidSignature(t *testing.T) { - pub, priv := genTestKey(t) - body := []byte("signed artifact") - sig := ed25519.Sign(priv, body) - - srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { - switch r.URL.Path { - case "/archive.html": - w.Header().Set("Content-Type", "text/html") - _, _ = w.Write(body) - case "/archive.html.sig": - w.Header().Set("Content-Type", "application/octet-stream") - _, _ = w.Write(sig) - default: - http.NotFound(w, r) - } - })) - defer srv.Close() - - cache, err := NewCache(filepath.Join(t.TempDir(), "_app")) - if err != nil { - t.Fatalf("NewCache: %v", err) - } - f := NewFetcher(cache, nil) - f.VerifyKey = pub // override the embedded production key - got, err := f.Fetch(context.Background(), srv.URL+"/archive.html") - if err != nil { - t.Fatalf("Fetch failed: %v", err) - } - if string(got) != string(body) { - t.Errorf("body mismatch") - } - // Cache hit on second call. - if !cache.Has(srv.URL + "/archive.html") { - t.Error("expected cache to contain artifact after successful verification") - } -} - -// TestFetcher_RejectsTamperedBody: the published .sig is valid but -// the body has been changed by a hypothetical mitm. Fetcher must -// reject and NOT cache the tampered bytes. -func TestFetcher_RejectsTamperedBody(t *testing.T) { - pub, priv := genTestKey(t) - original := []byte("genuine") - sig := ed25519.Sign(priv, original) - tampered := []byte("injected") - - srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { - switch r.URL.Path { - case "/archive.html": - _, _ = w.Write(tampered) - case "/archive.html.sig": - _, _ = w.Write(sig) - default: - http.NotFound(w, r) - } - })) - defer srv.Close() - - cache, err := NewCache(filepath.Join(t.TempDir(), "_app")) - if err != nil { - t.Fatalf("NewCache: %v", err) - } - f := NewFetcher(cache, nil) - f.VerifyKey = pub - _, err = f.Fetch(context.Background(), srv.URL+"/archive.html") - if err == nil { - t.Fatal("Fetch accepted tampered body, want error") - } - if !strings.Contains(err.Error(), "signature") { - t.Errorf("error %q does not mention signature", err) - } - if cache.Has(srv.URL + "/archive.html") { - t.Error("tampered bytes were cached; verifier must not write to cache on rejection") - } -} - -// TestFetcher_RejectsMissingSignature: artifact published but no .sig -// alongside (HTTP 404). Strict mode → reject. -func TestFetcher_RejectsMissingSignature(t *testing.T) { - pub, _ := genTestKey(t) - body := []byte("body without sig") - - srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { - switch r.URL.Path { - case "/archive.html": - _, _ = w.Write(body) - case "/archive.html.sig": - http.NotFound(w, r) - default: - http.NotFound(w, r) - } - })) - defer srv.Close() - - cache, _ := NewCache(filepath.Join(t.TempDir(), "_app")) - f := NewFetcher(cache, nil) - f.VerifyKey = pub - _, err := f.Fetch(context.Background(), srv.URL+"/archive.html") - if err == nil { - t.Fatal("Fetch accepted unsigned artifact, want error") - } - if !strings.Contains(err.Error(), "404") && !strings.Contains(err.Error(), "signature") { - t.Errorf("error %q does not mention 404 or signature", err) - } - if cache.Has(srv.URL + "/archive.html") { - t.Error("unsigned bytes were cached; verifier must reject before caching") - } -} - -// TestFetcher_RejectsWrongKeySignature: .sig present, well-formed, -// but signed by a different key than f.VerifyKey trusts. -func TestFetcher_RejectsWrongKeySignature(t *testing.T) { - trustedPub, _ := genTestKey(t) - _, attackerPriv := genTestKey(t) - body := []byte("body signed by an untrusted key") - sig := ed25519.Sign(attackerPriv, body) - - srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { - switch r.URL.Path { - case "/archive.html": - _, _ = w.Write(body) - case "/archive.html.sig": - _, _ = w.Write(sig) - default: - http.NotFound(w, r) - } - })) - defer srv.Close() - - cache, _ := NewCache(filepath.Join(t.TempDir(), "_app")) - f := NewFetcher(cache, nil) - f.VerifyKey = trustedPub - _, err := f.Fetch(context.Background(), srv.URL+"/archive.html") - if err == nil { - t.Fatal("Fetch accepted wrong-key-signed artifact, want error") - } - if cache.Has(srv.URL + "/archive.html") { - t.Error("wrong-key-signed bytes were cached") - } -} diff --git a/zddc/internal/config/config.go b/zddc/internal/config/config.go index c2f8668..185889b 100644 --- a/zddc/internal/config/config.go +++ b/zddc/internal/config/config.go @@ -34,17 +34,16 @@ type Config struct { // Root then becomes the cache directory rather than the served // data root. Master-mode flags (apps, archive, opa, etc.) are // ignored in client mode — see cmd/zddc-server/main.go. - Upstream string // --upstream / ZDDC_UPSTREAM — master URL (https://master.example.com); empty = run as master - Mode string // --mode / ZDDC_MODE — "proxy" (no disk persistence), "cache" (default; persist on access), "mirror" (cache + access-triggered subtree warmer) - BearerFile string // --bearer-file / ZDDC_BEARER_FILE — path to a 0600 file containing the master-issued token to forward upstream - SkipTLSVerify bool // --skip-tls-verify / ZDDC_SKIP_TLS_VERIFY=1 — accept self-signed / untrusted upstream certs. Distinct from --no-auth; intended for dev/internal CA scenarios only. - MirrorSubtree []string // --mirror-subtree / ZDDC_MIRROR_SUBTREE — comma-separated subtree URL paths the access-triggered walker keeps current. Default `/` when --mode=mirror and unset; ignored otherwise. - MirrorMinInterval time.Duration // --mirror-min-interval / ZDDC_MIRROR_MIN_INTERVAL — minimum gap between walks of the same subtree. Idle subtrees stay quiet; bumping this reduces upstream load on busy mirrors. Default 1h. - OPAURL string // --opa-url / ZDDC_OPA_URL — policy decider endpoint: "internal" (default), "http(s)://..." (real OPA via HTTP), or "unix:///..." (OPA via Unix socket) - OPAFailOpen bool // --opa-fail-open / ZDDC_OPA_FAIL_OPEN=1 — when external OPA is unreachable, allow instead of deny (default: fail closed) - OPACacheTTL time.Duration // --opa-cache-ttl / ZDDC_OPA_CACHE_TTL — external mode only: per-decision cache TTL. Default 1s. Set 0s to disable. - AppsPubKey string // --apps-pubkey / ZDDC_APPS_PUBKEY — path to the Ed25519 public key (PEM) used to verify Ed25519 signatures on URL-fetched apps: artifacts. Empty = URL apps disabled (only embedded + local-path apps work). Operators using zddc.varasys.io's canonical channels download pubkey.pem from there. - MaxWriteBytes int64 // --max-write-bytes / ZDDC_MAX_WRITE_BYTES — upper bound on PUT body size. Default 256 MiB. Per-request limit; rejected with 413. + Upstream string // --upstream / ZDDC_UPSTREAM — master URL (https://master.example.com); empty = run as master + Mode string // --mode / ZDDC_MODE — "proxy" (no disk persistence), "cache" (default; persist on access), "mirror" (cache + access-triggered subtree warmer) + BearerFile string // --bearer-file / ZDDC_BEARER_FILE — path to a 0600 file containing the master-issued token to forward upstream + SkipTLSVerify bool // --skip-tls-verify / ZDDC_SKIP_TLS_VERIFY=1 — accept self-signed / untrusted upstream certs. Distinct from --no-auth; intended for dev/internal CA scenarios only. + MirrorSubtree []string // --mirror-subtree / ZDDC_MIRROR_SUBTREE — comma-separated subtree URL paths the access-triggered walker keeps current. Default `/` when --mode=mirror and unset; ignored otherwise. + MirrorMinInterval time.Duration // --mirror-min-interval / ZDDC_MIRROR_MIN_INTERVAL — minimum gap between walks of the same subtree. Idle subtrees stay quiet; bumping this reduces upstream load on busy mirrors. Default 1h. + OPAURL string // --opa-url / ZDDC_OPA_URL — policy decider endpoint: "internal" (default), "http(s)://..." (real OPA via HTTP), or "unix:///..." (OPA via Unix socket) + OPAFailOpen bool // --opa-fail-open / ZDDC_OPA_FAIL_OPEN=1 — when external OPA is unreachable, allow instead of deny (default: fail closed) + OPACacheTTL time.Duration // --opa-cache-ttl / ZDDC_OPA_CACHE_TTL — external mode only: per-decision cache TTL. Default 1s. Set 0s to disable. + MaxWriteBytes int64 // --max-write-bytes / ZDDC_MAX_WRITE_BYTES — upper bound on PUT body size. Default 256 MiB. Per-request limit; rejected with 413. ArchiveRescanInterval time.Duration // --archive-rescan-interval / ZDDC_ARCHIVE_RESCAN_INTERVAL — periodic full re-walk of the archive index. Covers SMB/CIFS where inotify misses cross-client writes. Default 60s; 0 to disable. // MD→{docx,html,pdf} conversion endpoint (see internal/convert). @@ -132,8 +131,6 @@ func Load(args []string) (Config, error) { "External OPA only: on unreachable / non-2xx / malformed response, allow the request instead of denying. Default: fail closed.") opaCacheTTLFlag := fs.Duration("opa-cache-ttl", parseDurationOrDefault(os.Getenv("ZDDC_OPA_CACHE_TTL"), time.Second), "External OPA only: per-decision cache TTL. Amortizes round-trips on bursts of identical queries (e.g. .archive listing). Default 1s; set 0 to disable.") - appsPubKeyFlag := fs.String("apps-pubkey", os.Getenv("ZDDC_APPS_PUBKEY"), - "Path to the Ed25519 public key (PEM) used to verify signatures on URL-fetched apps: artifacts. Empty (default) = URL-fetched apps refused; only embedded + local-path apps work. Download zddc.varasys.io/pubkey.pem if you use the canonical channels.") maxWriteBytesFlag := fs.Int64("max-write-bytes", parseInt64OrDefault(os.Getenv("ZDDC_MAX_WRITE_BYTES"), 256*1024*1024), "Maximum PUT body size in bytes for the file API. Default 256 MiB. Larger requests are rejected with 413.") archiveRescanIntervalFlag := fs.Duration("archive-rescan-interval", parseDurationOrDefault(os.Getenv("ZDDC_ARCHIVE_RESCAN_INTERVAL"), 60*time.Second), @@ -198,28 +195,27 @@ func Load(args []string) (Config, error) { addrExplicit := addrFlagSet || addrEnvSet cfg := Config{ - Root: *rootFlag, - Addr: *addrFlag, - TLSCert: *tlsCertFlag, - TLSKey: *tlsKeyFlag, - LogLevel: *logLevelFlag, - IndexPath: *indexPathFlag, - EmailHeader: *emailHeaderFlag, - CORSOrigins: resolveCORS(corsFlagSet, *corsOriginFlag), - AccessLog: *accessLogFlag, - Insecure: *insecureFlag, - NoAuth: *noAuthFlag, - Upstream: *upstreamFlag, - Mode: *modeFlag, - BearerFile: *bearerFileFlag, - SkipTLSVerify: *skipTLSVerifyFlag, - MirrorSubtree: parseCSV(*mirrorSubtreeFlag), - MirrorMinInterval: *mirrorMinIntervalFlag, - OPAURL: *opaURLFlag, - OPAFailOpen: *opaFailOpenFlag, - OPACacheTTL: *opaCacheTTLFlag, - AppsPubKey: *appsPubKeyFlag, - MaxWriteBytes: *maxWriteBytesFlag, + Root: *rootFlag, + Addr: *addrFlag, + TLSCert: *tlsCertFlag, + TLSKey: *tlsKeyFlag, + LogLevel: *logLevelFlag, + IndexPath: *indexPathFlag, + EmailHeader: *emailHeaderFlag, + CORSOrigins: resolveCORS(corsFlagSet, *corsOriginFlag), + AccessLog: *accessLogFlag, + Insecure: *insecureFlag, + NoAuth: *noAuthFlag, + Upstream: *upstreamFlag, + Mode: *modeFlag, + BearerFile: *bearerFileFlag, + SkipTLSVerify: *skipTLSVerifyFlag, + MirrorSubtree: parseCSV(*mirrorSubtreeFlag), + MirrorMinInterval: *mirrorMinIntervalFlag, + OPAURL: *opaURLFlag, + OPAFailOpen: *opaFailOpenFlag, + OPACacheTTL: *opaCacheTTLFlag, + MaxWriteBytes: *maxWriteBytesFlag, ArchiveRescanInterval: *archiveRescanIntervalFlag, ConvertPandocBinary: *convertPandocBinaryFlag, ConvertChromiumBinary: *convertChromiumBinaryFlag, @@ -416,7 +412,6 @@ func Usage(w io.Writer) { fs.String("opa-url", "internal", "Policy decider: \"internal\", \"http(s)://...\", or \"unix:///...\".") fs.Bool("opa-fail-open", false, "External OPA: allow on transport error (default: deny / fail closed).") fs.Duration("opa-cache-ttl", time.Second, "External OPA: per-decision cache TTL (default 1s; 0 disables).") - fs.String("apps-pubkey", "", "Path to PEM Ed25519 pubkey for verifying signed URL-fetched apps. Empty = URL apps refused.") fs.String("access-log", "", "Tee structured access logs to this file (JSON, size-rotated). Default /.zddc.d/logs/access-.log; --access-log= disables.") fs.Duration("archive-rescan-interval", 60*time.Second, "Periodic full re-walk of the archive index (covers SMB inotify gap). Default 60s; 0 disables.") fs.Bool("help", false, "Print this help and exit.") diff --git a/zddc/internal/handler/zddcfile.go b/zddc/internal/handler/zddcfile.go index 5685334..4f76f02 100644 --- a/zddc/internal/handler/zddcfile.go +++ b/zddc/internal/handler/zddcfile.go @@ -31,22 +31,29 @@ func IsZddcFileRequest(urlPath string) bool { // ServeZddcFile serves a directory's .zddc as a plain YAML view. // // Method: GET / HEAD only — the dispatcher routes writes -// (PUT/DELETE/POST) directly to ServeFileAPI. +// +// (PUT/DELETE/POST) directly to ServeFileAPI. +// // ACL: the parent directory's read permission gates access. A -// user who can read the directory can read its .zddc. +// +// user who can read the directory can read its .zddc. +// // On-disk: if /.zddc exists, its bytes are returned verbatim -// with Content-Type: application/yaml. +// +// with Content-Type: application/yaml. +// // Virtual: if it does not exist, the body is the cascade's -// leaf-level ZddcFile (what defaults.zddc.yaml's paths: -// tree declares for THIS exact directory, plus any -// virtual contributions threaded through by the walker) -// marshalled as YAML. A header comment names the source -// and points at ?effective=1 for the composed view. The -// virtual body is itself valid YAML — PUT-saving it back -// (with or without edits) through the file API -// materialises a real on-disk override carrying exactly -// the bytes the user saved. The response sets -// X-ZDDC-Source: virtual:zddc so clients can distinguish. +// +// leaf-level ZddcFile (what defaults.zddc.yaml's paths: +// tree declares for THIS exact directory, plus any +// virtual contributions threaded through by the walker) +// marshalled as YAML. A header comment names the source +// and points at ?effective=1 for the composed view. The +// virtual body is itself valid YAML — PUT-saving it back +// (with or without edits) through the file API +// materialises a real on-disk override carrying exactly +// the bytes the user saved. The response sets +// X-ZDDC-Source: virtual:zddc so clients can distinguish. func ServeZddcFile(cfg config.Config, w http.ResponseWriter, r *http.Request) { decider := DeciderFromContext(r) @@ -274,7 +281,6 @@ func levelURLsFor(_, dirURL string, n int) []string { // surfacing. func isZeroZddcFile(zf zddc.ZddcFile) bool { return zf.Title == "" && - zf.AppsPubKey == "" && zf.CreatedBy == "" && zf.DefaultTool == "" && zf.DirTool == "" && @@ -289,7 +295,6 @@ func isZeroZddcFile(zf zddc.ZddcFile) bool { zf.Convert == nil && len(zf.ACL.Permissions) == 0 && len(zf.Admins) == 0 && - len(zf.Apps) == 0 && len(zf.Tables) == 0 && len(zf.Display) == 0 && len(zf.Roles) == 0 && diff --git a/zddc/internal/zddc/cascade.go b/zddc/internal/zddc/cascade.go index 2d362f7..c717277 100644 --- a/zddc/internal/zddc/cascade.go +++ b/zddc/internal/zddc/cascade.go @@ -393,8 +393,6 @@ func nonZeroZddcFields(zf ZddcFile) []string { add("title", zf.Title != "") add("acl", len(zf.ACL.Permissions) > 0 || zf.ACL.Inherit != nil) add("admins", len(zf.Admins) > 0) - add("apps", len(zf.Apps) > 0) - add("apps_pubkey", zf.AppsPubKey != "") add("tables", len(zf.Tables) > 0) add("display", len(zf.Display) > 0) add("convert", zf.Convert != nil) diff --git a/zddc/internal/zddc/file.go b/zddc/internal/zddc/file.go index f855a3c..47b32c1 100644 --- a/zddc/internal/zddc/file.go +++ b/zddc/internal/zddc/file.go @@ -104,32 +104,13 @@ type ConvertMetadata struct { // for the project on the landing-page picker. Optional — projects without a // title fall back to displaying the directory name. // -// Apps is a per-directory cascade override mapping app name → source spec. -// The spec is one of: "stable" / "beta" / "alpha" (channel on the canonical -// upstream), "v0.0.4" / "v0.0" / "v0" (version pin on the canonical -// upstream), an absolute "https://..." URL (custom mirror), or a relative -// or absolute filesystem path (./local.html, /opt/zddc/foo.html). -// -// On a request for a tool HTML, zddc-server walks .zddc files leaf→root -// looking for an Apps entry; first match wins. With no entry anywhere, the -// server serves the version baked into the binary at compile time (//go:embed). -// Fetched URL sources are cached in /_app/; the cache is fetch-once -// and never re-validates — operators delete the file to force a refetch. -// -// AppsPubKey is the inline PEM of the Ed25519 public key used to verify -// signatures on URL-fetched apps artifacts. Honored only at the root -// .zddc file (same root-only treatment as Admins, for the same reason: -// it's a trust anchor; subtree write authority must not be able to -// re-anchor it). Lower priority than --apps-pubkey / ZDDC_APPS_PUBKEY: -// when both are set, the env/flag (file path) wins. Empty in either -// place = URL-fetched apps refused (only embedded + local-path apps -// work). See zddc-server's setupApps. +// Tool HTML is resolved LOCALLY (no .zddc key): a real file on disk at the +// path → an ".html" member of /.zddc.zip → the embedded +// default. There is no `apps:` / `apps_pubkey:` key and no upstream fetch. type ZddcFile struct { - ACL ACLRules `yaml:"acl,omitempty" json:"acl,omitempty"` - Admins []string `yaml:"admins,omitempty" json:"admins,omitempty"` - Title string `yaml:"title,omitempty" json:"title,omitempty"` - Apps map[string]string `yaml:"apps,omitempty" json:"apps,omitempty"` - AppsPubKey string `yaml:"apps_pubkey,omitempty" json:"apps_pubkey,omitempty"` + ACL ACLRules `yaml:"acl,omitempty" json:"acl,omitempty"` + Admins []string `yaml:"admins,omitempty" json:"admins,omitempty"` + Title string `yaml:"title,omitempty" json:"title,omitempty"` // Tables declares directory-of-YAML table views available at this // directory. The map key becomes the URL stem: tables[MDL] is served diff --git a/zddc/internal/zddc/file_test.go b/zddc/internal/zddc/file_test.go index 3291244..ac5d2fe 100644 --- a/zddc/internal/zddc/file_test.go +++ b/zddc/internal/zddc/file_test.go @@ -42,9 +42,8 @@ roles: if zf.Title != "Demo" { t.Errorf("Title = %q want %q", zf.Title, "Demo") } - if got := zf.Apps["archive"]; got != "stable" { - t.Errorf("Apps[archive] = %q want %q", got, "stable") - } + // A stale `apps:` key in the fixture is ignored (the key was removed), + // not a parse error — back-compat for existing .zddc files. if r, ok := zf.Roles["reviewers"]; !ok || len(r.Members) != 1 { t.Errorf("Roles[reviewers] = %+v want one member", r) } diff --git a/zddc/internal/zddc/lookups.go b/zddc/internal/zddc/lookups.go index d069b0d..aac026f 100644 --- a/zddc/internal/zddc/lookups.go +++ b/zddc/internal/zddc/lookups.go @@ -375,7 +375,7 @@ func isZeroZddcFile(zf ZddcFile) bool { if len(zf.AvailableTools) > 0 { return false } - if zf.AppsPubKey != "" || zf.CreatedBy != "" { + if zf.CreatedBy != "" { return false } if zf.Worm != nil { // non-nil even when empty — marks a WORM zone @@ -390,7 +390,7 @@ func isZeroZddcFile(zf ZddcFile) bool { if zf.ACL.Inherit != nil { return false } - if len(zf.Apps) > 0 || len(zf.Tables) > 0 || len(zf.Display) > 0 || len(zf.Paths) > 0 { + if len(zf.Tables) > 0 || len(zf.Display) > 0 || len(zf.Paths) > 0 { return false } if len(zf.Roles) > 0 { diff --git a/zddc/internal/zddc/validate.go b/zddc/internal/zddc/validate.go index 0ca55e5..daeb8e4 100644 --- a/zddc/internal/zddc/validate.go +++ b/zddc/internal/zddc/validate.go @@ -5,24 +5,16 @@ import ( "strings" ) -// AppNames is the canonical set of app HTML files the server resolves -// via the apps fetch+cache subsystem. Order is stable for reproducible -// admin-UI rendering. +// AppNames is the canonical set of app HTML files the server can serve +// (from disk, the site .zddc.zip bundle, or the embedded default). Order +// is stable for reproducible rendering. // // All seven HTML tools belong here — including browse, form, and tables. -// Omitting any of them means the apps cascade (.zddc apps:) silently -// short-circuits to embedded for that name, defeating live-dev -// path-source overrides. // // Markdown editing used to be a dedicated tool ("mdedit"); it now // lives as a plugin inside browse (browse/js/preview-markdown.js). var AppNames = []string{"archive", "transmittal", "classifier", "landing", "browse", "form", "tables"} -// AppsDefaultKey is the special apps-map key that provides the baseline -// URL prefix and channel for any app not overridden per-name. Cascades -// through .zddc files like a per-app entry. -const AppsDefaultKey = "default" - // IsKnownApp reports whether name is one of the canonical apps. func IsKnownApp(name string) bool { for _, n := range AppNames { @@ -33,12 +25,6 @@ func IsKnownApp(name string) bool { return false } -// IsValidAppsKey reports whether name is acceptable as a key in the -// `apps:` map — either a canonical app or the special "default" key. -func IsValidAppsKey(name string) bool { - return name == AppsDefaultKey || IsKnownApp(name) -} - // ValidatePattern returns an error if pattern is not a syntactically // well-formed email-glob. The matcher in MatchesPattern is forgiving and // will silently fail to match malformed patterns (e.g., "alice@@x" or @@ -120,101 +106,6 @@ func ValidateProjectName(name string) error { return nil } -// ValidateAppSourceSpec returns nil if spec is a syntactically well-formed -// source spec accepted by apps.ParseSpec. It checks the string shape only — -// it does not verify URLs are reachable or paths exist. -// -// Accepted forms: -// - "stable" / "beta" / "alpha" / ":stable" / ":beta" / ":alpha" (channel) -// - "v0.0.4" / "0.0.4" / "v0.0" / "0.0" / "v0" / "0" / ":v0.0.4" (version) -// - "https://host/path" (URL prefix) -// - "https://host/path:stable" (URL prefix + channel) -// - "https://host/path/file.html" (terminal full URL) -// - "/abs/path.html" / "./rel/path.html" / "../sibling.html" (path) -func ValidateAppSourceSpec(spec string) error { - if spec == "" { - return fmt.Errorf("source spec is empty") - } - if strings.ContainsAny(spec, " \t\n\r") { - return fmt.Errorf("source spec contains whitespace") - } - - // Path forms. - if strings.HasPrefix(spec, "/") || - strings.HasPrefix(spec, "./") || - strings.HasPrefix(spec, "../") { - return nil - } - - // URL forms. - if strings.HasPrefix(spec, "https://") || strings.HasPrefix(spec, "http://") { - return validateURLSpec(spec) - } - - // Channel-or-version (with optional leading colon). - chanPart := strings.TrimPrefix(spec, ":") - if chanPart == "" { - return fmt.Errorf("empty channel after ':'") - } - return validateChannelOrVersion(chanPart) -} - -// validateURLSpec checks the URL-prefix or full-URL form. Splits on the -// last `:` after the last `/` (matching apps.parseURLSpec behavior). -func validateURLSpec(spec string) error { - // Minimal sanity check on URL shape. - if len(spec) <= len("https://") { - return fmt.Errorf("URL is missing host") - } - lastSlash := strings.LastIndex(spec, "/") - if lastSlash < 0 { - return fmt.Errorf("invalid URL %q: missing path separator", spec) - } - afterSlash := spec[lastSlash+1:] - colonInTail := strings.LastIndex(afterSlash, ":") - urlPart, suffixPart := spec, "" - if colonInTail >= 0 { - urlPart = spec[:lastSlash+1+colonInTail] - suffixPart = afterSlash[colonInTail+1:] - } - if strings.HasSuffix(urlPart, ".html") { - if suffixPart != "" { - return fmt.Errorf("URL ends in .html but has %q suffix", ":"+suffixPart) - } - return nil // terminal full URL - } - if suffixPart != "" { - return validateChannelOrVersion(suffixPart) - } - return nil // URL-prefix only -} - -// validateChannelOrVersion enforces the channel/version shape. -func validateChannelOrVersion(s string) error { - if s == "stable" || s == "beta" || s == "alpha" { - return nil - } - rest := strings.TrimPrefix(s, "v") - if rest == "" { - return fmt.Errorf("unrecognized source spec %q", s) - } - parts := strings.Split(rest, ".") - if len(parts) > 3 { - return fmt.Errorf("version has too many dots: %q", s) - } - for _, p := range parts { - if p == "" { - return fmt.Errorf("version has empty component: %q", s) - } - for _, r := range p { - if r < '0' || r > '9' { - return fmt.Errorf("unrecognized source spec %q", s) - } - } - } - return nil -} - func ValidateFile(zf ZddcFile) []FieldError { var errs []FieldError check := func(field string, vals []string) { @@ -242,21 +133,6 @@ func ValidateFile(zf ZddcFile) []FieldError { Message: "title exceeds 200 characters", }) } - for app, spec := range zf.Apps { - if !IsValidAppsKey(app) { - errs = append(errs, FieldError{ - Field: fmt.Sprintf("apps.%s", app), - Message: fmt.Sprintf("unknown app %q (known: default, archive, transmittal, classifier, landing, browse, form, tables)", app), - }) - continue - } - if err := ValidateAppSourceSpec(spec); err != nil { - errs = append(errs, FieldError{ - Field: fmt.Sprintf("apps.%s", app), - Message: err.Error(), - }) - } - } // worm: is a list of principal patterns (email-globs, @role:name, // or bare role names) that get write-once-create inside the WORM // zone. Validate each as an email-glob unless it's a role diff --git a/zddc/internal/zddc/validate_test.go b/zddc/internal/zddc/validate_test.go index 056976d..841b8b8 100644 --- a/zddc/internal/zddc/validate_test.go +++ b/zddc/internal/zddc/validate_test.go @@ -70,125 +70,6 @@ func TestValidateFile(t *testing.T) { } } -func TestValidateAppSourceSpec(t *testing.T) { - cases := []struct { - spec string - ok bool - }{ - // Channel shorthand (with and without leading colon) - {"stable", true}, - {"beta", true}, - {"alpha", true}, - {":stable", true}, - {":beta", true}, - {":alpha", true}, - // Version pin shorthand (full, partial, with/without leading 'v') - {"v0.0.4", true}, - {"0.0.4", true}, - {"v0.0", true}, - {"0.0", true}, - {"v0", true}, - {"0", true}, - {"v1.2.3", true}, - {":v0.0.4", true}, - {":0.0.4", true}, - // URLs - {"https://zddc.varasys.io/releases/archive_stable.html", true}, - {"http://my-fork.example.com/archive.html", true}, - {"https://my-mirror.example/releases", true}, // URL-prefix only - {"https://my-mirror.example/releases:stable", true}, // URL-prefix + channel - {"https://my-mirror.example/releases:v0.0.4", true}, // URL-prefix + version - {"https://my-mirror.example:8080/releases", true}, // URL with port - {"https://my-mirror.example:8080/releases:stable", true}, // URL with port + channel - // Paths - {"/abs/path.html", true}, - {"./local.html", true}, - {"../sibling.html", true}, - // Errors - {"", false}, - {" stable", false}, - {"stable ", false}, - {"with space", false}, - {"https://", false}, - {"https://host/path/file.html:stable", false}, // .html URL with suffix - {"random-thing", false}, - {":", false}, - {":random", false}, - {"v", false}, - {"v0.", false}, - {".0.0", false}, - {"v0.0.0.0", false}, - {"v0.a.0", false}, - {"https://my-mirror.example/releases:bogus", false}, // bad channel suffix - } - for _, tc := range cases { - t.Run(tc.spec, func(t *testing.T) { - err := ValidateAppSourceSpec(tc.spec) - if tc.ok && err != nil { - t.Errorf("ValidateAppSourceSpec(%q) = %v, want nil", tc.spec, err) - } - if !tc.ok && err == nil { - t.Errorf("ValidateAppSourceSpec(%q) = nil, want error", tc.spec) - } - }) - } -} - -func TestIsValidAppsKey(t *testing.T) { - cases := []struct { - key string - ok bool - }{ - {"default", true}, - {"archive", true}, - {"transmittal", true}, - {"classifier", true}, - {"browse", true}, - {"landing", true}, - {"unknown", false}, - {"", false}, - {"DEFAULT", false}, // case-sensitive - } - for _, tc := range cases { - t.Run(tc.key, func(t *testing.T) { - if got := IsValidAppsKey(tc.key); got != tc.ok { - t.Errorf("IsValidAppsKey(%q) = %v, want %v", tc.key, got, tc.ok) - } - }) - } -} - -func TestValidateFile_Apps(t *testing.T) { - zf := ZddcFile{ - Apps: map[string]string{ - "archive": "stable", // ok - "classifier": "v0.0.4", // ok - "default": "https://zddc.varasys.io/releases:stable", // ok (default key + URL+channel) - "transmittal": ":beta", // ok (channel-only) - "browse": "https://my-mirror.example/releases", // ok (URL-prefix only) - "unknown": "stable", // unknown app - "landing": "what is this", // bad spec - }, - } - errs := ValidateFile(zf) - want := map[string]bool{ - "apps.unknown": false, - "apps.landing": false, - } - for _, e := range errs { - if _, ok := want[e.Field]; ok { - want[e.Field] = true - } else { - t.Errorf("unexpected error field: %q (%s)", e.Field, e.Message) - } - } - for f, seen := range want { - if !seen { - t.Errorf("missing error for field %q (got: %+v)", f, errs) - } - } -} - func TestValidateProjectName(t *testing.T) { cases := []struct { name string diff --git a/zddc/internal/zddc/walker.go b/zddc/internal/zddc/walker.go index 28f0e5e..da106df 100644 --- a/zddc/internal/zddc/walker.go +++ b/zddc/internal/zddc/walker.go @@ -57,9 +57,6 @@ func mergeOverlay(base, top ZddcFile) ZddcFile { if top.Title != "" { out.Title = top.Title } - if top.AppsPubKey != "" { - out.AppsPubKey = top.AppsPubKey - } if top.CreatedBy != "" { out.CreatedBy = top.CreatedBy } @@ -124,7 +121,6 @@ func mergeOverlay(base, top ZddcFile) ZddcFile { } out.ACL.Permissions = mergeStringMap(out.ACL.Permissions, top.ACL.Permissions) - out.Apps = mergeStringMap(out.Apps, top.Apps) out.Tables = mergeStringMap(out.Tables, top.Tables) out.Display = mergeStringMap(out.Display, top.Display)