--mode mirror layers an access-triggered walker on top of the cache
pipeline. When an incoming request's URL falls under one of the
configured --mirror-subtree paths, the scheduler kicks off a recursive
walk of that subtree iff (a) no walk for that subtree is in flight and
(b) now - last_walk_at >= --mirror-min-interval (default 1h). Walks
run in a goroutine; the user's request never blocks on scheduling.
Why access-triggered: a naive "walk on a fixed timer" would produce
thundering-herd polls on a master from many vendor mirrors most of
which are idle most of the time. Demand-triggering means idle mirrors
generate zero upstream traffic until someone hits them; active
mirrors stay current as a side effect of normal use.
The walk:
1. Recursively fetches JSON listings under the subtree, persisting
each at <dir>/.zddc-listing.json so directory browsing works
offline for walked subtrees.
2. For each file, fires a conditional If-Modified-Since GET (bounded
parallelism; default 4 concurrent) — 304 no-op, 200 overwrites,
403/404 purges the local cache.
3. After enumeration, per-directory orphan purge: local files absent
from upstream's filtered listing are removed (handles upstream
deletes + ACL revocations).
State persists at <root>/.zddc-mirror-state.json as
{subtrees: {<path>: {last_walk_at}}}. In-flight tracking is in-memory
only — a crash mid-walk lets the next access retry without manual
cleanup. Subtree path matching is longest-prefix-wins; "/" is a
catch-all (full mirror, the default when --mode=mirror is set without
explicit --mirror-subtree).
The cache layer also gained directory-listing caching (independent of
mirror mode but enabled by it). Directories are now stored at
<dir>/.zddc-listing.<html|json> sidecars, varied by Accept header.
Hit/miss/offline semantics mirror the file pipeline. Phase 2's
limitation that directories always proxied live (no offline browse)
is now resolved for any directory the user has visited or that mirror
mode has walked.
Mirror scope falls out of auth: the walker uses the local instance's
bearer, so it sees exactly what the user can see at upstream. Admin
bearer → full mirror; vendor bearer → vendor's permitted subtree;
no code distinguishes the cases.
New flags (also as ZDDC_* env vars), ignored when --mode != mirror:
- --mirror-subtree <csv> — repeatable subtrees (comma-separated);
empty + --mode=mirror = "/" (full mirror)
- --mirror-min-interval <duration> — default 1h
Tests (15 new in walker_test.go, 3 new in cache_test.go): subtree
normalization, longest-prefix matching, root-as-catch-all, walk
fetches all files in scope, out-of-scope URLs are no-op, rate-
limiting prevents double-walks within min-interval, walks re-fire
after interval elapses, orphan purge removes local-only files,
state file survives restart, concurrent triggers don't double-walk,
end-to-end ServeHTTP-kicks-mirror-on-access, listing format varies
by Accept, listing offline serves stale, persisted state atomic
write + corrupt-input handling. Full suite + go vet clean.
Doc updates: zddc/README.md flags table gains the two new entries
plus a "Mirror mode (access-triggered subtree walker)" subsection
with trigger semantics and properties; the "What client mode is NOT,
yet" list shrinks accordingly. AGENTS.md env-var table gains the
two new entries. ARCHITECTURE.md "Master + proxy/cache/mirror"
section now documents the walker scheduler / walk algorithm / state
file in a "Mirror walker (access-triggered)" subsection.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
459 lines
24 KiB
Go
459 lines
24 KiB
Go
package config
|
|
|
|
import (
|
|
"errors"
|
|
"flag"
|
|
"fmt"
|
|
"io"
|
|
"net"
|
|
"os"
|
|
"path/filepath"
|
|
"strings"
|
|
"time"
|
|
)
|
|
|
|
// Config holds all runtime configuration. Each field can be set via a
|
|
// command-line flag (--<name>) or environment variable (ZDDC_<NAME>);
|
|
// the flag takes precedence when both are present.
|
|
type Config struct {
|
|
Root string // --root / ZDDC_ROOT — served file tree (default: CWD)
|
|
Addr string // --addr / ZDDC_ADDR — bind address (default :8443)
|
|
TLSCert string // --tls-cert / ZDDC_TLS_CERT — PEM cert path; "none" = plain HTTP; empty = self-signed
|
|
TLSKey string // --tls-key / ZDDC_TLS_KEY — PEM key path
|
|
TLSMode string // computed: none/selfsigned/provided
|
|
LogLevel string // --log-level / ZDDC_LOG_LEVEL — debug/info/warn/error (default info)
|
|
IndexPath string // --index-path / ZDDC_INDEX_PATH — virtual archive prefix (default .archive)
|
|
EmailHeader string // --email-header / ZDDC_EMAIL_HEADER — auth header name (default X-Auth-Request-Email)
|
|
CORSOrigins []string // --cors-origin / ZDDC_CORS_ORIGIN — comma-separated allowlist; default empty (CORS disabled); explicit value enables
|
|
AccessLog string // --access-log / ZDDC_ACCESS_LOG — file path for tee'd JSON access log; empty = stderr only
|
|
Insecure bool // --insecure / ZDDC_INSECURE=1 — opt out of safety checks (currently: allow start without a root .zddc, leaving the tree publicly accessible)
|
|
NoAuth bool // --no-auth / ZDDC_NO_AUTH=1 — skip ACL enforcement entirely. This instance is NOT the security boundary; on master = "open" (anyone reads everything), on a client = "trust upstream's filtering, don't re-evaluate ACLs locally."
|
|
|
|
// Client-mode flags. When Upstream is non-empty, this binary runs
|
|
// as a downstream proxy/cache/mirror against the named master.
|
|
// Root then becomes the cache directory rather than the served
|
|
// data root. Master-mode flags (apps, archive, opa, etc.) are
|
|
// ignored in client mode — see cmd/zddc-server/main.go.
|
|
Upstream string // --upstream / ZDDC_UPSTREAM — master URL (https://master.example.com); empty = run as master
|
|
Mode string // --mode / ZDDC_MODE — "proxy" (no disk persistence), "cache" (default; persist on access), "mirror" (cache + access-triggered subtree warmer)
|
|
BearerFile string // --bearer-file / ZDDC_BEARER_FILE — path to a 0600 file containing the master-issued token to forward upstream
|
|
SkipTLSVerify bool // --skip-tls-verify / ZDDC_SKIP_TLS_VERIFY=1 — accept self-signed / untrusted upstream certs. Distinct from --no-auth; intended for dev/internal CA scenarios only.
|
|
MirrorSubtree []string // --mirror-subtree / ZDDC_MIRROR_SUBTREE — comma-separated subtree URL paths the access-triggered walker keeps current. Default `/` when --mode=mirror and unset; ignored otherwise.
|
|
MirrorMinInterval time.Duration // --mirror-min-interval / ZDDC_MIRROR_MIN_INTERVAL — minimum gap between walks of the same subtree. Idle subtrees stay quiet; bumping this reduces upstream load on busy mirrors. Default 1h.
|
|
OPAURL string // --opa-url / ZDDC_OPA_URL — policy decider endpoint: "internal" (default), "http(s)://..." (real OPA via HTTP), or "unix:///..." (OPA via Unix socket)
|
|
OPAFailOpen bool // --opa-fail-open / ZDDC_OPA_FAIL_OPEN=1 — when external OPA is unreachable, allow instead of deny (default: fail closed)
|
|
OPACacheTTL time.Duration // --opa-cache-ttl / ZDDC_OPA_CACHE_TTL — external mode only: per-decision cache TTL. Default 1s. Set 0s to disable.
|
|
AppsPubKey string // --apps-pubkey / ZDDC_APPS_PUBKEY — path to the Ed25519 public key (PEM) used to verify Ed25519 signatures on URL-fetched apps: artifacts. Empty = URL apps disabled (only embedded + local-path apps work). Operators using zddc.varasys.io's canonical channels download pubkey.pem from there.
|
|
MaxWriteBytes int64 // --max-write-bytes / ZDDC_MAX_WRITE_BYTES — upper bound on PUT body size. Default 256 MiB. Per-request limit; rejected with 413.
|
|
CascadeMode string // --cascade-mode / ZDDC_CASCADE_MODE — "delegated" (default; leaf grants override ancestor denies) or "strict" (ancestor explicit-denies are absolute, NIST AC-6).
|
|
ArchiveRescanInterval time.Duration // --archive-rescan-interval / ZDDC_ARCHIVE_RESCAN_INTERVAL — periodic full re-walk of the archive index. Covers SMB/CIFS where inotify misses cross-client writes. Default 60s; 0 to disable.
|
|
}
|
|
|
|
// ErrHelpRequested is returned by Load when --help is passed; the caller
|
|
// should print Usage() to stderr and exit 0.
|
|
var ErrHelpRequested = errors.New("help requested")
|
|
|
|
// ErrVersionRequested is returned by Load when --version is passed; the
|
|
// caller should print version info and exit 0.
|
|
var ErrVersionRequested = errors.New("version requested")
|
|
|
|
// Load reads configuration from CLI flags + environment variables.
|
|
//
|
|
// Precedence (highest → lowest): command-line flag, environment variable,
|
|
// hard-coded default. Special-cases:
|
|
// - --root / ZDDC_ROOT default to the current working directory if both
|
|
// are unset, so an operator can `cd /srv/zddc && zddc-server` with
|
|
// zero config.
|
|
// - --version and --help return distinguished sentinel errors; the caller
|
|
// handles printing and exit. Pass nil for args to skip flag parsing
|
|
// entirely (used by tests that set state via env vars only).
|
|
//
|
|
// Standard usage from main.go:
|
|
//
|
|
// cfg, err := config.Load(os.Args[1:])
|
|
// if errors.Is(err, config.ErrHelpRequested) { os.Exit(0) }
|
|
// if errors.Is(err, config.ErrVersionRequested) { /* print versions */ ; os.Exit(0) }
|
|
// if err != nil { ... }
|
|
func Load(args []string) (Config, error) {
|
|
fs := flag.NewFlagSet("zddc-server", flag.ContinueOnError)
|
|
// Discard flag's own error output; we wrap and return our own.
|
|
fs.SetOutput(io.Discard)
|
|
|
|
rootFlag := fs.String("root", os.Getenv("ZDDC_ROOT"),
|
|
"Path to the served file tree. Default: ZDDC_ROOT or the current directory.")
|
|
addrFlag := fs.String("addr", getEnv("ZDDC_ADDR", ":8443"),
|
|
"Listen address (host:port). Default: ZDDC_ADDR or :8443.")
|
|
tlsCertFlag := fs.String("tls-cert", os.Getenv("ZDDC_TLS_CERT"),
|
|
"Path to a PEM TLS certificate. \"none\" disables TLS (plain HTTP). Empty means self-signed.")
|
|
tlsKeyFlag := fs.String("tls-key", os.Getenv("ZDDC_TLS_KEY"),
|
|
"Path to the matching PEM TLS private key.")
|
|
logLevelFlag := fs.String("log-level", getEnv("ZDDC_LOG_LEVEL", "info"),
|
|
"Log level: debug, info, warn, error.")
|
|
indexPathFlag := fs.String("index-path", getEnv("ZDDC_INDEX_PATH", ".archive"),
|
|
"URL segment that triggers the virtual archive index (default \".archive\").")
|
|
emailHeaderFlag := fs.String("email-header", getEnv("ZDDC_EMAIL_HEADER", "X-Auth-Request-Email"),
|
|
"HTTP header carrying the authenticated user's email.")
|
|
corsOriginFlag := fs.String("cors-origin", "",
|
|
"Comma-separated CORS allowlist. Empty (default) = CORS disabled. Set to your tool-host origin (e.g. https://tools.acme.com) only if browser-loaded pages from that origin call back into this server.")
|
|
insecureDirectFlag := fs.Bool("insecure-direct", os.Getenv("ZDDC_INSECURE_DIRECT") == "1",
|
|
"Allow plain HTTP on non-loopback addresses (only safe behind an authenticating proxy).")
|
|
insecureFlag := fs.Bool("insecure", os.Getenv("ZDDC_INSECURE") == "1",
|
|
"Allow startup with no root .zddc file (the tree is then publicly accessible). Default: refuse to start.")
|
|
noAuthFlag := fs.Bool("no-auth", os.Getenv("ZDDC_NO_AUTH") == "1",
|
|
"Skip ACL enforcement entirely. On master: anyone reads everything (dev / trusted-LAN / public-read deployments). On client: trust upstream's filtering. Distinct from --insecure (which gates startup-without-.zddc). Default: enforce ACLs.")
|
|
upstreamFlag := fs.String("upstream", os.Getenv("ZDDC_UPSTREAM"),
|
|
"Master URL (e.g. https://master.example.com). When set, this binary runs as a downstream proxy/cache/mirror against the master; --root becomes the cache directory. Empty (default) = run as master.")
|
|
modeFlag := fs.String("mode", getEnv("ZDDC_MODE", "cache"),
|
|
"Client mode: \"proxy\" (forward upstream live, no disk persistence), \"cache\" (default; persist responses on access), \"mirror\" (phase 3). Ignored when --upstream is empty.")
|
|
bearerFileFlag := fs.String("bearer-file", os.Getenv("ZDDC_BEARER_FILE"),
|
|
"Path to a 0600 file containing the master-issued token forwarded as Authorization: Bearer to upstream. See /.tokens on the master to issue one. Ignored when --upstream is empty.")
|
|
skipTLSVerifyFlag := fs.Bool("skip-tls-verify", os.Getenv("ZDDC_SKIP_TLS_VERIFY") == "1",
|
|
"Accept self-signed / untrusted TLS certs from the upstream. Distinct from --no-auth. Intended for dev or internal-CA scenarios only.")
|
|
mirrorSubtreeFlag := fs.String("mirror-subtree", os.Getenv("ZDDC_MIRROR_SUBTREE"),
|
|
"Comma-separated URL subtrees the access-triggered mirror walker keeps current (e.g. /Vendors/Acme,/Public). Empty + --mode=mirror = walk \"/\" (full mirror). Ignored when --mode != mirror.")
|
|
mirrorMinIntervalFlag := fs.Duration("mirror-min-interval", parseDurationOrDefault(os.Getenv("ZDDC_MIRROR_MIN_INTERVAL"), time.Hour),
|
|
"Minimum gap between walks of the same mirror subtree. Default 1h. Active mirrors revalidate as a side effect of use; idle subtrees generate zero upstream traffic until next access.")
|
|
opaURLFlag := fs.String("opa-url", getEnv("ZDDC_OPA_URL", "internal"),
|
|
"Policy decider endpoint: \"internal\" (built-in Go evaluator, default), \"http(s)://host:port\", or \"unix:///path/to/socket\".")
|
|
opaFailOpenFlag := fs.Bool("opa-fail-open", os.Getenv("ZDDC_OPA_FAIL_OPEN") == "1",
|
|
"External OPA only: on unreachable / non-2xx / malformed response, allow the request instead of denying. Default: fail closed.")
|
|
opaCacheTTLFlag := fs.Duration("opa-cache-ttl", parseDurationOrDefault(os.Getenv("ZDDC_OPA_CACHE_TTL"), time.Second),
|
|
"External OPA only: per-decision cache TTL. Amortizes round-trips on bursts of identical queries (e.g. .archive listing). Default 1s; set 0 to disable.")
|
|
appsPubKeyFlag := fs.String("apps-pubkey", os.Getenv("ZDDC_APPS_PUBKEY"),
|
|
"Path to the Ed25519 public key (PEM) used to verify signatures on URL-fetched apps: artifacts. Empty (default) = URL-fetched apps refused; only embedded + local-path apps work. Download zddc.varasys.io/pubkey.pem if you use the canonical channels.")
|
|
maxWriteBytesFlag := fs.Int64("max-write-bytes", parseInt64OrDefault(os.Getenv("ZDDC_MAX_WRITE_BYTES"), 256*1024*1024),
|
|
"Maximum PUT body size in bytes for the file API. Default 256 MiB. Larger requests are rejected with 413.")
|
|
cascadeModeFlag := fs.String("cascade-mode", getEnv("ZDDC_CASCADE_MODE", "delegated"),
|
|
"ACL cascade evaluation mode: \"delegated\" (default — subtree allow can override ancestor deny) or \"strict\" (ancestor explicit-deny is absolute; NIST AC-6).")
|
|
archiveRescanIntervalFlag := fs.Duration("archive-rescan-interval", parseDurationOrDefault(os.Getenv("ZDDC_ARCHIVE_RESCAN_INTERVAL"), 60*time.Second),
|
|
"Periodic full re-walk of the archive index. Required on SMB/CIFS-backed roots where inotify misses cross-client writes. Default 60s; set 0 to disable.")
|
|
accessLogFlag := fs.String("access-log", os.Getenv("ZDDC_ACCESS_LOG"),
|
|
"Tee structured access logs to this file (JSON, size-rotated). "+
|
|
"Default: <ZDDC_ROOT>/.zddc.d/logs/access-<hostname>.log. "+
|
|
"Set explicitly to empty (--access-log=) to disable.")
|
|
helpFlag := fs.Bool("help", false, "Print this help and exit.")
|
|
versionFlag := fs.Bool("version", false, "Print version info and exit.")
|
|
|
|
if args != nil {
|
|
if err := fs.Parse(args); err != nil {
|
|
if errors.Is(err, flag.ErrHelp) {
|
|
return Config{}, ErrHelpRequested
|
|
}
|
|
return Config{}, err
|
|
}
|
|
}
|
|
if *helpFlag {
|
|
return Config{}, ErrHelpRequested
|
|
}
|
|
if *versionFlag {
|
|
return Config{}, ErrVersionRequested
|
|
}
|
|
|
|
// CORS + AccessLog both have "unset → default; explicit-empty →
|
|
// disabled" semantics. The flag default is "" in both cases so we
|
|
// can't tell unset from explicit-empty via the flag alone —
|
|
// fs.Visit catches explicit flag use, and os.LookupEnv catches
|
|
// explicit env-var use.
|
|
corsFlagSet := false
|
|
accessLogFlagSet := false
|
|
if args != nil {
|
|
fs.Visit(func(f *flag.Flag) {
|
|
switch f.Name {
|
|
case "cors-origin":
|
|
corsFlagSet = true
|
|
case "access-log":
|
|
accessLogFlagSet = true
|
|
}
|
|
})
|
|
}
|
|
_, accessLogEnvSet := os.LookupEnv("ZDDC_ACCESS_LOG")
|
|
accessLogExplicit := accessLogFlagSet || accessLogEnvSet
|
|
|
|
cfg := Config{
|
|
Root: *rootFlag,
|
|
Addr: *addrFlag,
|
|
TLSCert: *tlsCertFlag,
|
|
TLSKey: *tlsKeyFlag,
|
|
LogLevel: *logLevelFlag,
|
|
IndexPath: *indexPathFlag,
|
|
EmailHeader: *emailHeaderFlag,
|
|
CORSOrigins: resolveCORS(corsFlagSet, *corsOriginFlag),
|
|
AccessLog: *accessLogFlag,
|
|
Insecure: *insecureFlag,
|
|
NoAuth: *noAuthFlag,
|
|
Upstream: *upstreamFlag,
|
|
Mode: *modeFlag,
|
|
BearerFile: *bearerFileFlag,
|
|
SkipTLSVerify: *skipTLSVerifyFlag,
|
|
MirrorSubtree: parseCSV(*mirrorSubtreeFlag),
|
|
MirrorMinInterval: *mirrorMinIntervalFlag,
|
|
OPAURL: *opaURLFlag,
|
|
OPAFailOpen: *opaFailOpenFlag,
|
|
OPACacheTTL: *opaCacheTTLFlag,
|
|
AppsPubKey: *appsPubKeyFlag,
|
|
MaxWriteBytes: *maxWriteBytesFlag,
|
|
CascadeMode: *cascadeModeFlag,
|
|
ArchiveRescanInterval: *archiveRescanIntervalFlag,
|
|
}
|
|
|
|
// Default Root to the current working directory.
|
|
if cfg.Root == "" {
|
|
cwd, err := os.Getwd()
|
|
if err != nil {
|
|
return Config{}, fmt.Errorf("--root not set and could not determine current directory: %w", err)
|
|
}
|
|
cfg.Root = cwd
|
|
}
|
|
|
|
info, err := os.Stat(cfg.Root)
|
|
if err != nil {
|
|
return Config{}, fmt.Errorf("--root %q is not accessible: %w", cfg.Root, err)
|
|
}
|
|
if !info.IsDir() {
|
|
return Config{}, fmt.Errorf("--root %q is not a directory", cfg.Root)
|
|
}
|
|
|
|
// Refuse to start when the served tree has no root .zddc file. With no
|
|
// .zddc anywhere in the chain, AllowedWithChain falls through to its
|
|
// "HasAnyFile=false → allow" default, so every directory is publicly
|
|
// accessible to anonymous callers. The vast majority of operators do not
|
|
// want that — and the few who do (a deliberately public archive) can pass
|
|
// --insecure to acknowledge it. See zddc/README.md § Access control.
|
|
//
|
|
// Skipped in client mode (cfg.Upstream != ""): the cache directory
|
|
// starts empty by design, so a missing .zddc is not a security
|
|
// concern — the cache layer doesn't evaluate ACLs locally
|
|
// (upstream filtering is the boundary; --no-auth on a client
|
|
// formalizes that). The directory will fill in as files are
|
|
// fetched, and any cached .zddc files come straight from upstream.
|
|
if !cfg.Insecure && cfg.Upstream == "" {
|
|
if _, err := os.Stat(filepath.Join(cfg.Root, ".zddc")); os.IsNotExist(err) {
|
|
return Config{}, fmt.Errorf(
|
|
"no %s/.zddc file found; the served tree would be publicly accessible to anonymous callers. "+
|
|
"Create a starter .zddc (at minimum: `admins: [you@yourcompany.com]`) "+
|
|
"or pass --insecure (or ZDDC_INSECURE=1) to acknowledge a deliberately-public deployment",
|
|
cfg.Root)
|
|
} else if err != nil {
|
|
return Config{}, fmt.Errorf("could not stat %s/.zddc: %w", cfg.Root, err)
|
|
}
|
|
}
|
|
|
|
// Audit-log default: if neither flag nor env was explicitly set,
|
|
// default to <Root>/.zddc.d/logs/access-<hostname>.log so the
|
|
// server captures an audit trail by default. Setting the flag/env
|
|
// to empty (--access-log=) is the explicit opt-out. Hostname is
|
|
// in the filename because operators typically run multiple zddc-
|
|
// server replicas against the same dataset (the .zddc.d directory
|
|
// is shared FS), and per-host filenames keep the JSON streams
|
|
// separable for downstream auditors.
|
|
if !accessLogExplicit {
|
|
host, herr := os.Hostname()
|
|
if herr != nil || host == "" {
|
|
host = "unknown"
|
|
}
|
|
cfg.AccessLog = filepath.Join(cfg.Root, ".zddc.d", "logs",
|
|
"access-"+host+".log")
|
|
}
|
|
|
|
// Determine TLS mode.
|
|
switch {
|
|
case cfg.TLSCert == "none":
|
|
cfg.TLSMode = "none"
|
|
case cfg.TLSCert == "" && cfg.TLSKey == "":
|
|
cfg.TLSMode = "selfsigned"
|
|
default:
|
|
cfg.TLSMode = "provided"
|
|
}
|
|
|
|
if cfg.TLSMode == "provided" && (cfg.TLSCert == "") != (cfg.TLSKey == "") {
|
|
return Config{}, errors.New("--tls-cert and --tls-key must both be set or both be empty")
|
|
}
|
|
|
|
switch cfg.CascadeMode {
|
|
case "", "delegated":
|
|
cfg.CascadeMode = "delegated"
|
|
case "strict":
|
|
// ok
|
|
default:
|
|
return Config{}, fmt.Errorf("--cascade-mode must be \"delegated\" or \"strict\", got %q", cfg.CascadeMode)
|
|
}
|
|
|
|
// Plain HTTP mode trusts the email header from any client. Only safe
|
|
// behind an authenticating reverse proxy. Refuse to start when binding
|
|
// plain HTTP to a non-loopback interface unless the operator has
|
|
// explicitly acknowledged the deployment shape.
|
|
//
|
|
// In client mode (Upstream set), the local instance never reads the
|
|
// email header to make decisions — auth is forwarded as a Bearer
|
|
// token to upstream and the local instance trusts upstream's
|
|
// filtering. So this check doesn't apply.
|
|
if cfg.Upstream == "" && cfg.TLSMode == "none" && !isLoopbackAddr(cfg.Addr) && !*insecureDirectFlag {
|
|
return Config{}, fmt.Errorf(
|
|
"--tls-cert=none binds plain HTTP to %q which trusts %s headers from any client; "+
|
|
"either use TLS (omit --tls-cert or supply a cert), bind to loopback (127.0.0.1: or [::1]:), "+
|
|
"or pass --insecure-direct to confirm an authenticating reverse proxy is in front",
|
|
cfg.Addr, cfg.EmailHeader)
|
|
}
|
|
|
|
// Client-mode validation. Only enforced when --upstream is set;
|
|
// the same flags are silently ignored in master mode.
|
|
if cfg.Upstream != "" {
|
|
switch cfg.Mode {
|
|
case "proxy", "cache", "mirror":
|
|
// ok
|
|
case "":
|
|
cfg.Mode = "cache"
|
|
default:
|
|
return Config{}, fmt.Errorf("--mode must be \"proxy\", \"cache\", or \"mirror\"; got %q", cfg.Mode)
|
|
}
|
|
if !strings.HasPrefix(cfg.Upstream, "http://") && !strings.HasPrefix(cfg.Upstream, "https://") {
|
|
return Config{}, fmt.Errorf("--upstream %q must start with http:// or https://", cfg.Upstream)
|
|
}
|
|
if strings.HasSuffix(cfg.Upstream, "/") {
|
|
cfg.Upstream = strings.TrimRight(cfg.Upstream, "/")
|
|
}
|
|
// Mirror mode: default subtree to "/" (full mirror) if the
|
|
// operator opted into mirror mode without specifying one.
|
|
if cfg.Mode == "mirror" && len(cfg.MirrorSubtree) == 0 {
|
|
cfg.MirrorSubtree = []string{"/"}
|
|
}
|
|
// Mirror subtrees are ignored in non-mirror modes — drop them
|
|
// rather than carry confusing dead config forward.
|
|
if cfg.Mode != "mirror" {
|
|
cfg.MirrorSubtree = nil
|
|
}
|
|
}
|
|
|
|
return cfg, nil
|
|
}
|
|
|
|
// Usage prints the flag list to w (stderr is the conventional caller).
|
|
// Returned format mirrors `flag.PrintDefaults` plus a one-line summary.
|
|
func Usage(w io.Writer) {
|
|
fmt.Fprintln(w, "Usage: zddc-server [flags]")
|
|
fmt.Fprintln(w, "")
|
|
fmt.Fprintln(w, "Each flag has an equivalent ZDDC_* environment variable; the flag wins on conflict.")
|
|
fmt.Fprintln(w, "ZDDC_ROOT defaults to the current working directory.")
|
|
fmt.Fprintln(w, "")
|
|
fmt.Fprintln(w, "Flags:")
|
|
fs := flag.NewFlagSet("zddc-server", flag.ContinueOnError)
|
|
fs.SetOutput(w)
|
|
// Re-register flags to populate Usage output (we discard the values).
|
|
fs.String("root", "", "Path to the served file tree. Default: ZDDC_ROOT or the current directory.")
|
|
fs.String("addr", ":8443", "Listen address (host:port). Default: ZDDC_ADDR or :8443.")
|
|
fs.String("tls-cert", "", "Path to a PEM TLS certificate. \"none\" disables TLS. Empty = self-signed.")
|
|
fs.String("tls-key", "", "Path to the matching PEM TLS private key.")
|
|
fs.String("log-level", "info", "Log level: debug, info, warn, error.")
|
|
fs.String("index-path", ".archive", "URL segment for the virtual archive index.")
|
|
fs.String("email-header", "X-Auth-Request-Email", "HTTP header carrying the authenticated user's email.")
|
|
fs.String("cors-origin", "", "Comma-separated CORS allowlist. Empty (default) = CORS disabled.")
|
|
fs.Bool("insecure-direct", false, "Allow plain HTTP on non-loopback addresses.")
|
|
fs.Bool("insecure", false, "Allow startup with no root .zddc file (publicly accessible). Default: refuse.")
|
|
fs.Bool("no-auth", false, "Skip ACL enforcement entirely. On master: anyone reads everything. On client: trust upstream's filtering. Distinct from --insecure.")
|
|
fs.String("upstream", "", "Master URL — when set, run as a downstream proxy/cache/mirror; --root becomes the cache directory. Empty (default) = master.")
|
|
fs.String("mode", "cache", "Client mode: proxy / cache / mirror. Ignored when --upstream is empty.")
|
|
fs.String("bearer-file", "", "Path to a 0600 file holding the master-issued bearer token forwarded to upstream. Ignored when --upstream is empty.")
|
|
fs.Bool("skip-tls-verify", false, "Accept self-signed / untrusted upstream TLS certs. Distinct from --no-auth. Dev / internal-CA scenarios only.")
|
|
fs.String("mirror-subtree", "", "Comma-separated URL subtrees the mirror walker keeps current. Empty + --mode=mirror = full mirror (\"/\"). Ignored when --mode != mirror.")
|
|
fs.Duration("mirror-min-interval", time.Hour, "Min gap between walks of the same mirror subtree. Default 1h.")
|
|
fs.String("opa-url", "internal", "Policy decider: \"internal\", \"http(s)://...\", or \"unix:///...\".")
|
|
fs.Bool("opa-fail-open", false, "External OPA: allow on transport error (default: deny / fail closed).")
|
|
fs.Duration("opa-cache-ttl", time.Second, "External OPA: per-decision cache TTL (default 1s; 0 disables).")
|
|
fs.String("apps-pubkey", "", "Path to PEM Ed25519 pubkey for verifying signed URL-fetched apps. Empty = URL apps refused.")
|
|
fs.String("access-log", "", "Tee structured access logs to this file (JSON, size-rotated). Default <ZDDC_ROOT>/.zddc.d/logs/access-<hostname>.log; --access-log= disables.")
|
|
fs.Duration("archive-rescan-interval", 60*time.Second, "Periodic full re-walk of the archive index (covers SMB inotify gap). Default 60s; 0 disables.")
|
|
fs.Bool("help", false, "Print this help and exit.")
|
|
fs.Bool("version", false, "Print version info and exit.")
|
|
fs.PrintDefaults()
|
|
}
|
|
|
|
// resolveCORS implements the precedence rules for the CORS allowlist:
|
|
// - flag explicitly set → use flag value (empty = disabled)
|
|
// - else env var explicitly set → use env value (empty = disabled)
|
|
// - else → default to nil (CORS disabled)
|
|
//
|
|
// Default-empty is intentional: the embedded-tools deployment path (the install
|
|
// default) serves tools and data from the same origin, so CORS is unneeded.
|
|
// Operators who deliberately load tools from a different origin (e.g. the
|
|
// CDN-bootstrap pattern from https://zddc.varasys.io, or self-hosted at
|
|
// https://tools.acme.com) opt in by setting the value explicitly. This avoids
|
|
// implicit cross-origin trust on third-party domains.
|
|
func resolveCORS(flagSet bool, flagValue string) []string {
|
|
if flagSet {
|
|
return parseCSV(flagValue)
|
|
}
|
|
if v, ok := os.LookupEnv("ZDDC_CORS_ORIGIN"); ok {
|
|
return parseCSV(v)
|
|
}
|
|
return nil
|
|
}
|
|
|
|
// parseCSV splits a comma-separated list and trims whitespace. Empty
|
|
// returns nil (which the middleware treats as "CORS disabled").
|
|
func parseCSV(s string) []string {
|
|
if s == "" {
|
|
return nil
|
|
}
|
|
parts := strings.Split(s, ",")
|
|
out := make([]string, 0, len(parts))
|
|
for _, p := range parts {
|
|
if t := strings.TrimSpace(p); t != "" {
|
|
out = append(out, t)
|
|
}
|
|
}
|
|
return out
|
|
}
|
|
|
|
// isLoopbackAddr reports whether addr binds only to a loopback interface.
|
|
// addr is in net.Listen form: "host:port", ":port", or "[ipv6]:port".
|
|
// ":port" means all interfaces, so it is NOT loopback.
|
|
func isLoopbackAddr(addr string) bool {
|
|
host, _, err := net.SplitHostPort(addr)
|
|
if err != nil {
|
|
return false
|
|
}
|
|
if host == "" {
|
|
return false
|
|
}
|
|
if host == "localhost" {
|
|
return true
|
|
}
|
|
ip := net.ParseIP(host)
|
|
if ip == nil {
|
|
return false
|
|
}
|
|
return ip.IsLoopback()
|
|
}
|
|
|
|
func getEnv(key, fallback string) string {
|
|
if v := os.Getenv(key); v != "" {
|
|
return v
|
|
}
|
|
return fallback
|
|
}
|
|
|
|
// parseDurationOrDefault parses a duration string ("1s", "500ms", "0", etc.).
|
|
// Returns def on empty input or parse error. Used for env-var defaults
|
|
// that need a sensible fallback rather than a hard error on typo.
|
|
func parseDurationOrDefault(s string, def time.Duration) time.Duration {
|
|
if s == "" {
|
|
return def
|
|
}
|
|
if d, err := time.ParseDuration(s); err == nil {
|
|
return d
|
|
}
|
|
return def
|
|
}
|
|
|
|
// parseInt64OrDefault parses a base-10 int64. Returns def on empty input
|
|
// or parse error.
|
|
func parseInt64OrDefault(s string, def int64) int64 {
|
|
if s == "" {
|
|
return def
|
|
}
|
|
var n int64
|
|
if _, err := fmt.Sscan(s, &n); err == nil {
|
|
return n
|
|
}
|
|
return def
|
|
}
|