Replaces the always-spawn-an-OCI-container model with a per-call
bubblewrap sandbox. Pandoc and chromium binaries are baked into the
zddc-server runtime image; each conversion runs them under bwrap's
Linux-namespace isolation. No daemon, no socket, no privileged outer
container, no OCI image pull at conversion time.
Why: the OCI engine paid ≈ 350 MB image pulls + 400 MB persistent
storage + ~300 ms per-conversion startup, plus required either an
on-host daemon socket (zddc-RCE → host-RCE in one hop) or nested
container privileges. bwrap gets the same sandbox properties
(--unshare-all, ro-bind /usr, tmpfs /tmp, clearenv, no-network) at
~5 ms per call and zero external dependencies. This is the same
primitive Flatpak uses for every app launch — battle-tested at scale
for "untrusted-input, short-lived, isolated."
Runner abstraction:
- `Runner.Run` signature: image string → ToolSpec{Image, Binary}.
Both fields populated by entry points; whichever engine is
installed reads the one it needs.
- `bwrapRunner` (new): assembles bwrap argv via `buildBwrapArgs`
helper (testable in isolation), spawns bwrap with the binary.
- `containerRunner` (renamed conceptually to "legacy fallback"):
unchanged behavior, still reachable for hosts that prefer OCI
containers per conversion.
Probe order in health.Probe: bwrap → podman → docker. First hit wins.
Engine kinds in Capabilities: "bwrap" | "podman" | "docker". The
no-engine error message now lists all three.
Config (cmd/zddc-server):
- new --convert-pandoc-binary / ZDDC_CONVERT_PANDOC_BINARY (default "pandoc")
- new --convert-chromium-binary / ZDDC_CONVERT_CHROMIUM_BINARY (default "chromium-browser")
- existing --convert-pandoc-image / --convert-chromium-image kept
for the OCI engine, doc updated to clarify they only apply there.
- --convert-engine helptext lists bwrap first.
Images:
- New `zddc/runtime.Containerfile` — alpine + bubblewrap + pandoc-cli +
chromium + font-noto. Documents build/publish workflow.
- helm/zddc-server-prod/values.yaml.example: runtimeImage default
switched to a placeholder for the new bundled runtime image; bare
alpine NO LONGER works for /.convert (clearly called out in the
comment).
- bitnest dev: /var/lib/zddc-dev-build/Containerfile mirrors the
production runtime image. Quadlet at /etc/containers/systemd/
zddc.container drops the podman-socket mount (no longer needed)
and sets ZDDC_CONVERT_ENGINE=bwrap explicitly to avoid silent
downgrades if a stray podman ends up on PATH.
Tests:
- convert_test.go: fakeRunner / recordingRunner now record ToolSpec.
- New TestToolSpecPopulation pins that both Image and Binary are
filled by every entry point.
- New TestBwrapArgs_SandboxFlagsPresent / MountTranslation /
RejectsBadMountSpec lock in the bwrap argv shape — a refactor that
drops a hardening flag or misroutes a mount fails this loud.
Docs:
- AGENTS.md § "Server-side document conversion" rewritten around
the bwrap-first model with podman/docker as legacy fallbacks.
- ARCHITECTURE.md convert reference updated.
- internal/convert package doc reflects the two-engine probe order.
Verified end-to-end on bitnest: probe reports
engine=bwrap pandoc_binary=pandoc chromium_binary=chromium-browser
on startup. All 15 Go test packages green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
549 lines
31 KiB
Go
549 lines
31 KiB
Go
package config
|
|
|
|
import (
|
|
"errors"
|
|
"flag"
|
|
"fmt"
|
|
"io"
|
|
"net"
|
|
"os"
|
|
"path/filepath"
|
|
"strings"
|
|
"time"
|
|
)
|
|
|
|
// Config holds all runtime configuration. Each field can be set via a
|
|
// command-line flag (--<name>) or environment variable (ZDDC_<NAME>);
|
|
// the flag takes precedence when both are present.
|
|
type Config struct {
|
|
Root string // --root / ZDDC_ROOT — served file tree (default: CWD)
|
|
Addr string // --addr / ZDDC_ADDR — bind address (default :8443)
|
|
TLSCert string // --tls-cert / ZDDC_TLS_CERT — PEM cert path; "none" = plain HTTP; empty = self-signed
|
|
TLSKey string // --tls-key / ZDDC_TLS_KEY — PEM key path
|
|
TLSMode string // computed: none/selfsigned/provided
|
|
LogLevel string // --log-level / ZDDC_LOG_LEVEL — debug/info/warn/error (default info)
|
|
IndexPath string // --index-path / ZDDC_INDEX_PATH — virtual archive prefix (default .archive)
|
|
EmailHeader string // --email-header / ZDDC_EMAIL_HEADER — auth header name (default X-Auth-Request-Email)
|
|
CORSOrigins []string // --cors-origin / ZDDC_CORS_ORIGIN — comma-separated allowlist; default empty (CORS disabled); explicit value enables
|
|
AccessLog string // --access-log / ZDDC_ACCESS_LOG — file path for tee'd JSON access log; empty = stderr only
|
|
Insecure bool // --insecure / ZDDC_INSECURE=1 — opt out of safety checks (currently: allow start without a root .zddc, leaving the tree publicly accessible)
|
|
NoAuth bool // --no-auth / ZDDC_NO_AUTH=1 — skip ACL enforcement entirely. This instance is NOT the security boundary; on master = "open" (anyone reads everything), on a client = "trust upstream's filtering, don't re-evaluate ACLs locally."
|
|
|
|
// Client-mode flags. When Upstream is non-empty, this binary runs
|
|
// as a downstream proxy/cache/mirror against the named master.
|
|
// Root then becomes the cache directory rather than the served
|
|
// data root. Master-mode flags (apps, archive, opa, etc.) are
|
|
// ignored in client mode — see cmd/zddc-server/main.go.
|
|
Upstream string // --upstream / ZDDC_UPSTREAM — master URL (https://master.example.com); empty = run as master
|
|
Mode string // --mode / ZDDC_MODE — "proxy" (no disk persistence), "cache" (default; persist on access), "mirror" (cache + access-triggered subtree warmer)
|
|
BearerFile string // --bearer-file / ZDDC_BEARER_FILE — path to a 0600 file containing the master-issued token to forward upstream
|
|
SkipTLSVerify bool // --skip-tls-verify / ZDDC_SKIP_TLS_VERIFY=1 — accept self-signed / untrusted upstream certs. Distinct from --no-auth; intended for dev/internal CA scenarios only.
|
|
MirrorSubtree []string // --mirror-subtree / ZDDC_MIRROR_SUBTREE — comma-separated subtree URL paths the access-triggered walker keeps current. Default `/` when --mode=mirror and unset; ignored otherwise.
|
|
MirrorMinInterval time.Duration // --mirror-min-interval / ZDDC_MIRROR_MIN_INTERVAL — minimum gap between walks of the same subtree. Idle subtrees stay quiet; bumping this reduces upstream load on busy mirrors. Default 1h.
|
|
OPAURL string // --opa-url / ZDDC_OPA_URL — policy decider endpoint: "internal" (default), "http(s)://..." (real OPA via HTTP), or "unix:///..." (OPA via Unix socket)
|
|
OPAFailOpen bool // --opa-fail-open / ZDDC_OPA_FAIL_OPEN=1 — when external OPA is unreachable, allow instead of deny (default: fail closed)
|
|
OPACacheTTL time.Duration // --opa-cache-ttl / ZDDC_OPA_CACHE_TTL — external mode only: per-decision cache TTL. Default 1s. Set 0s to disable.
|
|
AppsPubKey string // --apps-pubkey / ZDDC_APPS_PUBKEY — path to the Ed25519 public key (PEM) used to verify Ed25519 signatures on URL-fetched apps: artifacts. Empty = URL apps disabled (only embedded + local-path apps work). Operators using zddc.varasys.io's canonical channels download pubkey.pem from there.
|
|
MaxWriteBytes int64 // --max-write-bytes / ZDDC_MAX_WRITE_BYTES — upper bound on PUT body size. Default 256 MiB. Per-request limit; rejected with 413.
|
|
ArchiveRescanInterval time.Duration // --archive-rescan-interval / ZDDC_ARCHIVE_RESCAN_INTERVAL — periodic full re-walk of the archive index. Covers SMB/CIFS where inotify misses cross-client writes. Default 60s; 0 to disable.
|
|
|
|
// MD→{docx,html,pdf} conversion endpoint (see internal/convert).
|
|
// The server shells out to upstream pandoc + chromium container
|
|
// images via podman or docker, pulling each on first use via
|
|
// production default. The engine probe order is bwrap → podman →
|
|
// docker; the first one found on PATH wins. bwrap runs the
|
|
// pandoc + chromium binaries baked into the zddc-server image
|
|
// in a per-call Linux-namespace sandbox (no daemon, no socket,
|
|
// no OCI image pull). podman/docker are legacy fallbacks for
|
|
// hosts that already have a container engine and want OCI-image
|
|
// isolation per conversion.
|
|
ConvertPandocImage string // --convert-pandoc-image / ZDDC_CONVERT_PANDOC_IMAGE — image for MD→DOCX/HTML when the OCI engine is selected. Default docker.io/pandoc/latex:latest.
|
|
ConvertChromiumImage string // --convert-chromium-image / ZDDC_CONVERT_CHROMIUM_IMAGE — image for HTML→PDF when the OCI engine is selected. Default docker.io/zenika/alpine-chrome:latest.
|
|
ConvertPandocBinary string // --convert-pandoc-binary / ZDDC_CONVERT_PANDOC_BINARY — pandoc binary name (PATH-resolved) when the bwrap engine is selected. Default "pandoc".
|
|
ConvertChromiumBinary string // --convert-chromium-binary / ZDDC_CONVERT_CHROMIUM_BINARY — chromium binary name (PATH-resolved) when the bwrap engine is selected. Default "chromium-browser" (alpine); set to "chromium" on debian.
|
|
ConvertEngine string // --convert-engine / ZDDC_CONVERT_ENGINE — override sandbox binary (default: probe for bwrap, then podman, then docker).
|
|
ConvertPodmanSocket string // --convert-podman-socket / ZDDC_CONVERT_PODMAN_SOCKET — when non-empty, run podman in remote mode against this Unix socket (e.g. unix:///var/run/podman/podman.sock). Used with the Kubernetes sidecar pattern so zddc-server's own pod stays unprivileged.
|
|
ConvertScratchDir string // --convert-scratch-dir / ZDDC_CONVERT_SCRATCH_DIR — directory used for per-conversion scratch (template + HTML/PDF intermediates). Must be a path the remote podman can see at the same path. Empty = use $TMPDIR (local-mode default).
|
|
ConvertMemMiB int // --convert-mem-mib / ZDDC_CONVERT_MEM_MIB — per-container memory cap in MiB. Default 512.
|
|
ConvertCPUs string // --convert-cpus / ZDDC_CONVERT_CPUS — per-container CPU limit. Default "2".
|
|
ConvertPIDs int // --convert-pids / ZDDC_CONVERT_PIDS — per-container PID limit. Default 100.
|
|
ConvertTimeout time.Duration // --convert-timeout / ZDDC_CONVERT_TIMEOUT — per-conversion wall clock. Default 30s.
|
|
}
|
|
|
|
// ErrHelpRequested is returned by Load when --help is passed; the caller
|
|
// should print Usage() to stderr and exit 0.
|
|
var ErrHelpRequested = errors.New("help requested")
|
|
|
|
// ErrVersionRequested is returned by Load when --version is passed; the
|
|
// caller should print version info and exit 0.
|
|
var ErrVersionRequested = errors.New("version requested")
|
|
|
|
// Load reads configuration from CLI flags + environment variables.
|
|
//
|
|
// Precedence (highest → lowest): command-line flag, environment variable,
|
|
// hard-coded default. Special-cases:
|
|
// - --root / ZDDC_ROOT default to the current working directory if both
|
|
// are unset, so an operator can `cd /srv/zddc && zddc-server` with
|
|
// zero config.
|
|
// - --version and --help return distinguished sentinel errors; the caller
|
|
// handles printing and exit. Pass nil for args to skip flag parsing
|
|
// entirely (used by tests that set state via env vars only).
|
|
//
|
|
// Standard usage from main.go:
|
|
//
|
|
// cfg, err := config.Load(os.Args[1:])
|
|
// if errors.Is(err, config.ErrHelpRequested) { os.Exit(0) }
|
|
// if errors.Is(err, config.ErrVersionRequested) { /* print versions */ ; os.Exit(0) }
|
|
// if err != nil { ... }
|
|
func Load(args []string) (Config, error) {
|
|
fs := flag.NewFlagSet("zddc-server", flag.ContinueOnError)
|
|
// Discard flag's own error output; we wrap and return our own.
|
|
fs.SetOutput(io.Discard)
|
|
|
|
rootFlag := fs.String("root", os.Getenv("ZDDC_ROOT"),
|
|
"Path to the served file tree. Default: ZDDC_ROOT or the current directory.")
|
|
addrFlag := fs.String("addr", getEnv("ZDDC_ADDR", ":8443"),
|
|
"Listen address (host:port). Default: ZDDC_ADDR or :8443. **In client mode (--upstream set) the default downgrades to 127.0.0.1:8443** unless --addr/ZDDC_ADDR is set explicitly — the cache layer forwards a bearer to upstream without authenticating the local caller, so a non-loopback bind needs --insecure-direct to acknowledge an authenticating reverse proxy / network policy is in front.")
|
|
tlsCertFlag := fs.String("tls-cert", os.Getenv("ZDDC_TLS_CERT"),
|
|
"Path to a PEM TLS certificate. \"none\" disables TLS (plain HTTP). Empty means self-signed.")
|
|
tlsKeyFlag := fs.String("tls-key", os.Getenv("ZDDC_TLS_KEY"),
|
|
"Path to the matching PEM TLS private key.")
|
|
logLevelFlag := fs.String("log-level", getEnv("ZDDC_LOG_LEVEL", "info"),
|
|
"Log level: debug, info, warn, error.")
|
|
indexPathFlag := fs.String("index-path", getEnv("ZDDC_INDEX_PATH", ".archive"),
|
|
"URL segment that triggers the virtual archive index (default \".archive\").")
|
|
emailHeaderFlag := fs.String("email-header", getEnv("ZDDC_EMAIL_HEADER", "X-Auth-Request-Email"),
|
|
"HTTP header carrying the authenticated user's email.")
|
|
corsOriginFlag := fs.String("cors-origin", "",
|
|
"Comma-separated CORS allowlist. Empty (default) = CORS disabled. Set to your tool-host origin (e.g. https://tools.acme.com) only if browser-loaded pages from that origin call back into this server.")
|
|
insecureDirectFlag := fs.Bool("insecure-direct", os.Getenv("ZDDC_INSECURE_DIRECT") == "1",
|
|
"Allow plain HTTP on non-loopback addresses (only safe behind an authenticating proxy).")
|
|
insecureFlag := fs.Bool("insecure", os.Getenv("ZDDC_INSECURE") == "1",
|
|
"Allow startup with no root .zddc file (the tree is then publicly accessible). Default: refuse to start.")
|
|
noAuthFlag := fs.Bool("no-auth", os.Getenv("ZDDC_NO_AUTH") == "1",
|
|
"Skip ACL enforcement entirely. On master: anyone reads everything (dev / trusted-LAN / public-read deployments). On client: trust upstream's filtering. Distinct from --insecure (which gates startup-without-.zddc). Default: enforce ACLs.")
|
|
upstreamFlag := fs.String("upstream", os.Getenv("ZDDC_UPSTREAM"),
|
|
"Master URL (e.g. https://master.example.com). When set, this binary runs as a downstream proxy/cache/mirror against the master; --root becomes the cache directory. Empty (default) = run as master.")
|
|
modeFlag := fs.String("mode", getEnv("ZDDC_MODE", "cache"),
|
|
"Client mode: \"proxy\" (forward upstream live, no disk persistence), \"cache\" (default; persist responses on access), \"mirror\" (phase 3). Ignored when --upstream is empty.")
|
|
bearerFileFlag := fs.String("bearer-file", os.Getenv("ZDDC_BEARER_FILE"),
|
|
"Path to a 0600 file containing the master-issued token forwarded as Authorization: Bearer to upstream. See /.tokens on the master to issue one. Ignored when --upstream is empty.")
|
|
skipTLSVerifyFlag := fs.Bool("skip-tls-verify", os.Getenv("ZDDC_SKIP_TLS_VERIFY") == "1",
|
|
"Accept self-signed / untrusted TLS certs from the upstream. Distinct from --no-auth. Intended for dev or internal-CA scenarios only.")
|
|
mirrorSubtreeFlag := fs.String("mirror-subtree", os.Getenv("ZDDC_MIRROR_SUBTREE"),
|
|
"Comma-separated URL subtrees the access-triggered mirror walker keeps current (e.g. /Vendors/Acme,/Public). Empty + --mode=mirror = walk \"/\" (full mirror). Ignored when --mode != mirror.")
|
|
mirrorMinIntervalFlag := fs.Duration("mirror-min-interval", parseDurationOrDefault(os.Getenv("ZDDC_MIRROR_MIN_INTERVAL"), time.Hour),
|
|
"Minimum gap between walks of the same mirror subtree. Default 1h. Active mirrors revalidate as a side effect of use; idle subtrees generate zero upstream traffic until next access.")
|
|
opaURLFlag := fs.String("opa-url", getEnv("ZDDC_OPA_URL", "internal"),
|
|
"Policy decider endpoint: \"internal\" (built-in Go evaluator, default), \"http(s)://host:port\", or \"unix:///path/to/socket\".")
|
|
opaFailOpenFlag := fs.Bool("opa-fail-open", os.Getenv("ZDDC_OPA_FAIL_OPEN") == "1",
|
|
"External OPA only: on unreachable / non-2xx / malformed response, allow the request instead of denying. Default: fail closed.")
|
|
opaCacheTTLFlag := fs.Duration("opa-cache-ttl", parseDurationOrDefault(os.Getenv("ZDDC_OPA_CACHE_TTL"), time.Second),
|
|
"External OPA only: per-decision cache TTL. Amortizes round-trips on bursts of identical queries (e.g. .archive listing). Default 1s; set 0 to disable.")
|
|
appsPubKeyFlag := fs.String("apps-pubkey", os.Getenv("ZDDC_APPS_PUBKEY"),
|
|
"Path to the Ed25519 public key (PEM) used to verify signatures on URL-fetched apps: artifacts. Empty (default) = URL-fetched apps refused; only embedded + local-path apps work. Download zddc.varasys.io/pubkey.pem if you use the canonical channels.")
|
|
maxWriteBytesFlag := fs.Int64("max-write-bytes", parseInt64OrDefault(os.Getenv("ZDDC_MAX_WRITE_BYTES"), 256*1024*1024),
|
|
"Maximum PUT body size in bytes for the file API. Default 256 MiB. Larger requests are rejected with 413.")
|
|
archiveRescanIntervalFlag := fs.Duration("archive-rescan-interval", parseDurationOrDefault(os.Getenv("ZDDC_ARCHIVE_RESCAN_INTERVAL"), 60*time.Second),
|
|
"Periodic full re-walk of the archive index. Required on SMB/CIFS-backed roots where inotify misses cross-client writes. Default 60s; set 0 to disable.")
|
|
convertPandocImageFlag := fs.String("convert-pandoc-image", getEnv("ZDDC_CONVERT_PANDOC_IMAGE", "docker.io/pandoc/latex:latest"),
|
|
"Pandoc OCI image for MD→DOCX / MD→HTML, used only when the OCI engine (podman/docker) is selected. Pulled on first use via --pull=missing.")
|
|
convertChromiumImageFlag := fs.String("convert-chromium-image", getEnv("ZDDC_CONVERT_CHROMIUM_IMAGE", "docker.io/zenika/alpine-chrome:latest"),
|
|
"Chromium OCI image for HTML→PDF, used only when the OCI engine is selected. Pulled on first use via --pull=missing.")
|
|
convertPandocBinaryFlag := fs.String("convert-pandoc-binary", getEnv("ZDDC_CONVERT_PANDOC_BINARY", "pandoc"),
|
|
"Pandoc binary name (PATH-resolved) when the bwrap engine is selected. Default \"pandoc\".")
|
|
convertChromiumBinaryFlag := fs.String("convert-chromium-binary", getEnv("ZDDC_CONVERT_CHROMIUM_BINARY", "chromium-browser"),
|
|
"Chromium binary name (PATH-resolved) when the bwrap engine is selected. Default \"chromium-browser\" (alpine); set to \"chromium\" on debian/ubuntu.")
|
|
convertEngineFlag := fs.String("convert-engine", os.Getenv("ZDDC_CONVERT_ENGINE"),
|
|
"Conversion sandbox override (default: probe for bwrap, then podman, then docker).")
|
|
convertPodmanSocketFlag := fs.String("convert-podman-socket", os.Getenv("ZDDC_CONVERT_PODMAN_SOCKET"),
|
|
"Run podman in remote mode against this Unix socket URL (e.g. unix:///var/run/podman/podman.sock). When set, the engine binary is invoked as `podman --remote --url=<this> run …`; the actual container creation happens in whatever process owns the socket (typically a podman-system-service sidecar). Empty = local mode.")
|
|
convertScratchDirFlag := fs.String("convert-scratch-dir", os.Getenv("ZDDC_CONVERT_SCRATCH_DIR"),
|
|
"Scratch directory for per-conversion intermediates (template, HTML, PDF). In remote mode this MUST be a path that the podman-service side can see at the same path — typically a shared emptyDir mounted at the same mountPath in both containers. Empty = use $TMPDIR (local mode).")
|
|
convertMemMiBFlag := fs.Int("convert-mem-mib", parseIntOrDefault(os.Getenv("ZDDC_CONVERT_MEM_MIB"), 512),
|
|
"Per-conversion container memory limit in MiB. Default 512.")
|
|
convertCPUsFlag := fs.String("convert-cpus", getEnv("ZDDC_CONVERT_CPUS", "2"),
|
|
"Per-conversion container CPU limit (passed to --cpus). Default 2.")
|
|
convertPIDsFlag := fs.Int("convert-pids", parseIntOrDefault(os.Getenv("ZDDC_CONVERT_PIDS"), 100),
|
|
"Per-conversion container PID limit. Default 100.")
|
|
convertTimeoutFlag := fs.Duration("convert-timeout", parseDurationOrDefault(os.Getenv("ZDDC_CONVERT_TIMEOUT"), 30*time.Second),
|
|
"Per-conversion wall-clock timeout. Default 30s.")
|
|
accessLogFlag := fs.String("access-log", os.Getenv("ZDDC_ACCESS_LOG"),
|
|
"Tee structured access logs to this file (JSON, size-rotated). "+
|
|
"Default: <ZDDC_ROOT>/.zddc.d/logs/access-<hostname>.log. "+
|
|
"Set explicitly to empty (--access-log=) to disable.")
|
|
helpFlag := fs.Bool("help", false, "Print this help and exit.")
|
|
versionFlag := fs.Bool("version", false, "Print version info and exit.")
|
|
|
|
if args != nil {
|
|
if err := fs.Parse(args); err != nil {
|
|
if errors.Is(err, flag.ErrHelp) {
|
|
return Config{}, ErrHelpRequested
|
|
}
|
|
return Config{}, err
|
|
}
|
|
}
|
|
if *helpFlag {
|
|
return Config{}, ErrHelpRequested
|
|
}
|
|
if *versionFlag {
|
|
return Config{}, ErrVersionRequested
|
|
}
|
|
|
|
// CORS + AccessLog both have "unset → default; explicit-empty →
|
|
// disabled" semantics. The flag default is "" in both cases so we
|
|
// can't tell unset from explicit-empty via the flag alone —
|
|
// fs.Visit catches explicit flag use, and os.LookupEnv catches
|
|
// explicit env-var use.
|
|
corsFlagSet := false
|
|
accessLogFlagSet := false
|
|
addrFlagSet := false
|
|
if args != nil {
|
|
fs.Visit(func(f *flag.Flag) {
|
|
switch f.Name {
|
|
case "cors-origin":
|
|
corsFlagSet = true
|
|
case "access-log":
|
|
accessLogFlagSet = true
|
|
case "addr":
|
|
addrFlagSet = true
|
|
}
|
|
})
|
|
}
|
|
_, accessLogEnvSet := os.LookupEnv("ZDDC_ACCESS_LOG")
|
|
accessLogExplicit := accessLogFlagSet || accessLogEnvSet
|
|
_, addrEnvSet := os.LookupEnv("ZDDC_ADDR")
|
|
addrExplicit := addrFlagSet || addrEnvSet
|
|
|
|
cfg := Config{
|
|
Root: *rootFlag,
|
|
Addr: *addrFlag,
|
|
TLSCert: *tlsCertFlag,
|
|
TLSKey: *tlsKeyFlag,
|
|
LogLevel: *logLevelFlag,
|
|
IndexPath: *indexPathFlag,
|
|
EmailHeader: *emailHeaderFlag,
|
|
CORSOrigins: resolveCORS(corsFlagSet, *corsOriginFlag),
|
|
AccessLog: *accessLogFlag,
|
|
Insecure: *insecureFlag,
|
|
NoAuth: *noAuthFlag,
|
|
Upstream: *upstreamFlag,
|
|
Mode: *modeFlag,
|
|
BearerFile: *bearerFileFlag,
|
|
SkipTLSVerify: *skipTLSVerifyFlag,
|
|
MirrorSubtree: parseCSV(*mirrorSubtreeFlag),
|
|
MirrorMinInterval: *mirrorMinIntervalFlag,
|
|
OPAURL: *opaURLFlag,
|
|
OPAFailOpen: *opaFailOpenFlag,
|
|
OPACacheTTL: *opaCacheTTLFlag,
|
|
AppsPubKey: *appsPubKeyFlag,
|
|
MaxWriteBytes: *maxWriteBytesFlag,
|
|
ArchiveRescanInterval: *archiveRescanIntervalFlag,
|
|
ConvertPandocImage: *convertPandocImageFlag,
|
|
ConvertChromiumImage: *convertChromiumImageFlag,
|
|
ConvertPandocBinary: *convertPandocBinaryFlag,
|
|
ConvertChromiumBinary: *convertChromiumBinaryFlag,
|
|
ConvertEngine: *convertEngineFlag,
|
|
ConvertPodmanSocket: *convertPodmanSocketFlag,
|
|
ConvertScratchDir: *convertScratchDirFlag,
|
|
ConvertMemMiB: *convertMemMiBFlag,
|
|
ConvertCPUs: *convertCPUsFlag,
|
|
ConvertPIDs: *convertPIDsFlag,
|
|
ConvertTimeout: *convertTimeoutFlag,
|
|
}
|
|
|
|
// Default Root to the current working directory.
|
|
if cfg.Root == "" {
|
|
cwd, err := os.Getwd()
|
|
if err != nil {
|
|
return Config{}, fmt.Errorf("--root not set and could not determine current directory: %w", err)
|
|
}
|
|
cfg.Root = cwd
|
|
}
|
|
|
|
info, err := os.Stat(cfg.Root)
|
|
if err != nil {
|
|
return Config{}, fmt.Errorf("--root %q is not accessible: %w", cfg.Root, err)
|
|
}
|
|
if !info.IsDir() {
|
|
return Config{}, fmt.Errorf("--root %q is not a directory", cfg.Root)
|
|
}
|
|
|
|
// Refuse to start when the served tree has no root .zddc file. With no
|
|
// .zddc anywhere in the chain, AllowedWithChain falls through to its
|
|
// "HasAnyFile=false → allow" default, so every directory is publicly
|
|
// accessible to anonymous callers. The vast majority of operators do not
|
|
// want that — and the few who do (a deliberately public archive) can pass
|
|
// --insecure to acknowledge it. See zddc/README.md § Access control.
|
|
//
|
|
// Skipped in client mode (cfg.Upstream != ""): the cache directory
|
|
// starts empty by design, so a missing .zddc is not a security
|
|
// concern — the cache layer doesn't evaluate ACLs locally
|
|
// (upstream filtering is the boundary; --no-auth on a client
|
|
// formalizes that). The directory will fill in as files are
|
|
// fetched, and any cached .zddc files come straight from upstream.
|
|
if !cfg.Insecure && cfg.Upstream == "" {
|
|
if _, err := os.Stat(filepath.Join(cfg.Root, ".zddc")); os.IsNotExist(err) {
|
|
return Config{}, fmt.Errorf(
|
|
"no %s/.zddc file found; the served tree would be publicly accessible to anonymous callers. "+
|
|
"Create a starter .zddc (at minimum: `admins: [you@yourcompany.com]`) "+
|
|
"or pass --insecure (or ZDDC_INSECURE=1) to acknowledge a deliberately-public deployment",
|
|
cfg.Root)
|
|
} else if err != nil {
|
|
return Config{}, fmt.Errorf("could not stat %s/.zddc: %w", cfg.Root, err)
|
|
}
|
|
}
|
|
|
|
// Audit-log default: if neither flag nor env was explicitly set,
|
|
// default to <Root>/.zddc.d/logs/access-<hostname>.log so the
|
|
// server captures an audit trail by default. Setting the flag/env
|
|
// to empty (--access-log=) is the explicit opt-out. Hostname is
|
|
// in the filename because operators typically run multiple zddc-
|
|
// server replicas against the same dataset (the .zddc.d directory
|
|
// is shared FS), and per-host filenames keep the JSON streams
|
|
// separable for downstream auditors.
|
|
if !accessLogExplicit {
|
|
host, herr := os.Hostname()
|
|
if herr != nil || host == "" {
|
|
host = "unknown"
|
|
}
|
|
cfg.AccessLog = filepath.Join(cfg.Root, ".zddc.d", "logs",
|
|
"access-"+host+".log")
|
|
}
|
|
|
|
// Determine TLS mode.
|
|
switch {
|
|
case cfg.TLSCert == "none":
|
|
cfg.TLSMode = "none"
|
|
case cfg.TLSCert == "" && cfg.TLSKey == "":
|
|
cfg.TLSMode = "selfsigned"
|
|
default:
|
|
cfg.TLSMode = "provided"
|
|
}
|
|
|
|
if cfg.TLSMode == "provided" && (cfg.TLSCert == "") != (cfg.TLSKey == "") {
|
|
return Config{}, errors.New("--tls-cert and --tls-key must both be set or both be empty")
|
|
}
|
|
|
|
// Plain HTTP mode trusts the email header from any client. Only safe
|
|
// behind an authenticating reverse proxy. Refuse to start when binding
|
|
// plain HTTP to a non-loopback interface unless the operator has
|
|
// explicitly acknowledged the deployment shape.
|
|
//
|
|
// In client mode (Upstream set), the local instance never reads the
|
|
// email header to make decisions — auth is forwarded as a Bearer
|
|
// token to upstream and the local instance trusts upstream's
|
|
// filtering. So this check doesn't apply.
|
|
if cfg.Upstream == "" && cfg.TLSMode == "none" && !isLoopbackAddr(cfg.Addr) && !*insecureDirectFlag {
|
|
return Config{}, fmt.Errorf(
|
|
"--tls-cert=none binds plain HTTP to %q which trusts %s headers from any client; "+
|
|
"either use TLS (omit --tls-cert or supply a cert), bind to loopback (127.0.0.1: or [::1]:), "+
|
|
"or pass --insecure-direct to confirm an authenticating reverse proxy is in front",
|
|
cfg.Addr, cfg.EmailHeader)
|
|
}
|
|
|
|
// Client-mode validation. Only enforced when --upstream is set;
|
|
// the same flags are silently ignored in master mode.
|
|
if cfg.Upstream != "" {
|
|
switch cfg.Mode {
|
|
case "proxy", "cache", "mirror":
|
|
// ok
|
|
case "":
|
|
cfg.Mode = "cache"
|
|
default:
|
|
return Config{}, fmt.Errorf("--mode must be \"proxy\", \"cache\", or \"mirror\"; got %q", cfg.Mode)
|
|
}
|
|
if !strings.HasPrefix(cfg.Upstream, "http://") && !strings.HasPrefix(cfg.Upstream, "https://") {
|
|
return Config{}, fmt.Errorf("--upstream %q must start with http:// or https://", cfg.Upstream)
|
|
}
|
|
if strings.HasSuffix(cfg.Upstream, "/") {
|
|
cfg.Upstream = strings.TrimRight(cfg.Upstream, "/")
|
|
}
|
|
// Mirror mode: default subtree to "/" (full mirror) if the
|
|
// operator opted into mirror mode without specifying one.
|
|
if cfg.Mode == "mirror" && len(cfg.MirrorSubtree) == 0 {
|
|
cfg.MirrorSubtree = []string{"/"}
|
|
}
|
|
// Mirror subtrees are ignored in non-mirror modes — drop them
|
|
// rather than carry confusing dead config forward.
|
|
if cfg.Mode != "mirror" {
|
|
cfg.MirrorSubtree = nil
|
|
}
|
|
|
|
// Confused-deputy guard. The cache layer authenticates to
|
|
// upstream via the configured bearer but does NOT authenticate
|
|
// the local incoming caller, so a non-loopback bind exposes the
|
|
// cache as an open proxy that launders any caller's request
|
|
// with the bearer-holder's privileges. Two layers of defense:
|
|
//
|
|
// 1. When the operator didn't pass --addr / ZDDC_ADDR
|
|
// explicitly, downgrade the default to loopback. CLI
|
|
// users on a laptop get safe-by-default.
|
|
//
|
|
// 2. When the operator DID pick a non-loopback bind AND a
|
|
// bearer file is configured, refuse to start unless they
|
|
// also pass --insecure-direct (the existing flag for
|
|
// "I have an authenticating reverse proxy / network
|
|
// policy in front, the bare bind is intentional"). The
|
|
// helm cache chart sets ZDDC_INSECURE_DIRECT=1 + a
|
|
// Kubernetes-namespaced pod network for exactly this
|
|
// reason, so the chart path is unaffected.
|
|
if !addrExplicit {
|
|
cfg.Addr = "127.0.0.1:8443"
|
|
}
|
|
if cfg.BearerFile != "" && !isLoopbackAddr(cfg.Addr) && !*insecureDirectFlag {
|
|
return Config{}, fmt.Errorf(
|
|
"client mode (--upstream set) bound to %q is non-loopback AND a bearer is configured; "+
|
|
"the cache will forward that bearer to upstream on every incoming request without "+
|
|
"authenticating the caller, exposing an open-proxy confused-deputy to anyone reachable on this interface. "+
|
|
"Bind to loopback (127.0.0.1: or [::1]:) — the default in client mode unless you set --addr / ZDDC_ADDR explicitly — "+
|
|
"OR pass --insecure-direct (ZDDC_INSECURE_DIRECT=1) to acknowledge that an authenticating reverse proxy "+
|
|
"or network policy gates the bind interface",
|
|
cfg.Addr)
|
|
}
|
|
}
|
|
|
|
return cfg, nil
|
|
}
|
|
|
|
// Usage prints the flag list to w (stderr is the conventional caller).
|
|
// Returned format mirrors `flag.PrintDefaults` plus a one-line summary.
|
|
func Usage(w io.Writer) {
|
|
fmt.Fprintln(w, "Usage: zddc-server [flags]")
|
|
fmt.Fprintln(w, "")
|
|
fmt.Fprintln(w, "Each flag has an equivalent ZDDC_* environment variable; the flag wins on conflict.")
|
|
fmt.Fprintln(w, "ZDDC_ROOT defaults to the current working directory.")
|
|
fmt.Fprintln(w, "")
|
|
fmt.Fprintln(w, "Flags:")
|
|
fs := flag.NewFlagSet("zddc-server", flag.ContinueOnError)
|
|
fs.SetOutput(w)
|
|
// Re-register flags to populate Usage output (we discard the values).
|
|
fs.String("root", "", "Path to the served file tree. Default: ZDDC_ROOT or the current directory.")
|
|
fs.String("addr", ":8443", "Listen address (host:port). Default: ZDDC_ADDR or :8443.")
|
|
fs.String("tls-cert", "", "Path to a PEM TLS certificate. \"none\" disables TLS. Empty = self-signed.")
|
|
fs.String("tls-key", "", "Path to the matching PEM TLS private key.")
|
|
fs.String("log-level", "info", "Log level: debug, info, warn, error.")
|
|
fs.String("index-path", ".archive", "URL segment for the virtual archive index.")
|
|
fs.String("email-header", "X-Auth-Request-Email", "HTTP header carrying the authenticated user's email.")
|
|
fs.String("cors-origin", "", "Comma-separated CORS allowlist. Empty (default) = CORS disabled.")
|
|
fs.Bool("insecure-direct", false, "Allow plain HTTP on non-loopback addresses.")
|
|
fs.Bool("insecure", false, "Allow startup with no root .zddc file (publicly accessible). Default: refuse.")
|
|
fs.Bool("no-auth", false, "Skip ACL enforcement entirely. On master: anyone reads everything. On client: trust upstream's filtering. Distinct from --insecure.")
|
|
fs.String("upstream", "", "Master URL — when set, run as a downstream proxy/cache/mirror; --root becomes the cache directory. Empty (default) = master.")
|
|
fs.String("mode", "cache", "Client mode: proxy / cache / mirror. Ignored when --upstream is empty.")
|
|
fs.String("bearer-file", "", "Path to a 0600 file holding the master-issued bearer token forwarded to upstream. Ignored when --upstream is empty.")
|
|
fs.Bool("skip-tls-verify", false, "Accept self-signed / untrusted upstream TLS certs. Distinct from --no-auth. Dev / internal-CA scenarios only.")
|
|
fs.String("mirror-subtree", "", "Comma-separated URL subtrees the mirror walker keeps current. Empty + --mode=mirror = full mirror (\"/\"). Ignored when --mode != mirror.")
|
|
fs.Duration("mirror-min-interval", time.Hour, "Min gap between walks of the same mirror subtree. Default 1h.")
|
|
fs.String("opa-url", "internal", "Policy decider: \"internal\", \"http(s)://...\", or \"unix:///...\".")
|
|
fs.Bool("opa-fail-open", false, "External OPA: allow on transport error (default: deny / fail closed).")
|
|
fs.Duration("opa-cache-ttl", time.Second, "External OPA: per-decision cache TTL (default 1s; 0 disables).")
|
|
fs.String("apps-pubkey", "", "Path to PEM Ed25519 pubkey for verifying signed URL-fetched apps. Empty = URL apps refused.")
|
|
fs.String("access-log", "", "Tee structured access logs to this file (JSON, size-rotated). Default <ZDDC_ROOT>/.zddc.d/logs/access-<hostname>.log; --access-log= disables.")
|
|
fs.Duration("archive-rescan-interval", 60*time.Second, "Periodic full re-walk of the archive index (covers SMB inotify gap). Default 60s; 0 disables.")
|
|
fs.Bool("help", false, "Print this help and exit.")
|
|
fs.Bool("version", false, "Print version info and exit.")
|
|
fs.PrintDefaults()
|
|
}
|
|
|
|
// resolveCORS implements the precedence rules for the CORS allowlist:
|
|
// - flag explicitly set → use flag value (empty = disabled)
|
|
// - else env var explicitly set → use env value (empty = disabled)
|
|
// - else → default to nil (CORS disabled)
|
|
//
|
|
// Default-empty is intentional: the embedded-tools deployment path (the install
|
|
// default) serves tools and data from the same origin, so CORS is unneeded.
|
|
// Operators who deliberately load tools from a different origin (e.g. the
|
|
// CDN-bootstrap pattern from https://zddc.varasys.io, or self-hosted at
|
|
// https://tools.acme.com) opt in by setting the value explicitly. This avoids
|
|
// implicit cross-origin trust on third-party domains.
|
|
func resolveCORS(flagSet bool, flagValue string) []string {
|
|
if flagSet {
|
|
return parseCSV(flagValue)
|
|
}
|
|
if v, ok := os.LookupEnv("ZDDC_CORS_ORIGIN"); ok {
|
|
return parseCSV(v)
|
|
}
|
|
return nil
|
|
}
|
|
|
|
// parseCSV splits a comma-separated list and trims whitespace. Empty
|
|
// returns nil (which the middleware treats as "CORS disabled").
|
|
func parseCSV(s string) []string {
|
|
if s == "" {
|
|
return nil
|
|
}
|
|
parts := strings.Split(s, ",")
|
|
out := make([]string, 0, len(parts))
|
|
for _, p := range parts {
|
|
if t := strings.TrimSpace(p); t != "" {
|
|
out = append(out, t)
|
|
}
|
|
}
|
|
return out
|
|
}
|
|
|
|
// isLoopbackAddr reports whether addr binds only to a loopback interface.
|
|
// addr is in net.Listen form: "host:port", ":port", or "[ipv6]:port".
|
|
// ":port" means all interfaces, so it is NOT loopback.
|
|
func isLoopbackAddr(addr string) bool {
|
|
host, _, err := net.SplitHostPort(addr)
|
|
if err != nil {
|
|
return false
|
|
}
|
|
if host == "" {
|
|
return false
|
|
}
|
|
if host == "localhost" {
|
|
return true
|
|
}
|
|
ip := net.ParseIP(host)
|
|
if ip == nil {
|
|
return false
|
|
}
|
|
return ip.IsLoopback()
|
|
}
|
|
|
|
func getEnv(key, fallback string) string {
|
|
if v := os.Getenv(key); v != "" {
|
|
return v
|
|
}
|
|
return fallback
|
|
}
|
|
|
|
// parseDurationOrDefault parses a duration string ("1s", "500ms", "0", etc.).
|
|
// Returns def on empty input or parse error. Used for env-var defaults
|
|
// that need a sensible fallback rather than a hard error on typo.
|
|
func parseDurationOrDefault(s string, def time.Duration) time.Duration {
|
|
if s == "" {
|
|
return def
|
|
}
|
|
if d, err := time.ParseDuration(s); err == nil {
|
|
return d
|
|
}
|
|
return def
|
|
}
|
|
|
|
// parseInt64OrDefault parses a base-10 int64. Returns def on empty input
|
|
// or parse error.
|
|
func parseInt64OrDefault(s string, def int64) int64 {
|
|
if s == "" {
|
|
return def
|
|
}
|
|
var n int64
|
|
if _, err := fmt.Sscan(s, &n); err == nil {
|
|
return n
|
|
}
|
|
return def
|
|
}
|
|
|
|
func parseIntOrDefault(s string, def int) int {
|
|
if s == "" {
|
|
return def
|
|
}
|
|
var n int
|
|
if _, err := fmt.Sscan(s, &n); err == nil {
|
|
return n
|
|
}
|
|
return def
|
|
}
|