ZDDC/zddc
ZDDC c22bb19dab perf(server): gzip compression middleware on the entire mux
Add github.com/klauspost/compress/gzhttp wrapper around the request
handler. With MinSize(1024), responses ≥ 1 KB get gzip-encoded when
the client advertises Accept-Encoding: gzip; smaller bodies + 304
Not Modified pass through unchanged.

The wrapper auto-appends Vary: Accept-Encoding (compatible with the
existing Vary: Accept on directory.go's content-negotiated path).

Live-tested against zddc-server -root /tmp/empty:
  GET / w/ Accept-Encoding: gzip → 20.9 KB compressed (was 80.9 KB
                                   uncompressed). 74% reduction.
  Decompresses cleanly back to the original bytes.

Helps every code path that bypasses Caddy: devshell pods, local dev
binaries, tests, anywhere zddc-server is hit directly. Production
behind Caddy already had compression at the proxy layer; this just
makes the Go server self-sufficient.

Tests in cmd/zddc-server/main_test.go cover:
- large body + Accept-Encoding → compressed + Vary header
- small body → not compressed (under MinSize)
- no Accept-Encoding header → plain bytes
2026-05-03 23:31:18 -05:00
..
cmd/zddc-server perf(server): gzip compression middleware on the entire mux 2026-05-03 23:31:18 -05:00
internal perf(server): ETag + max-age=0 on embedded HTML responses 2026-05-03 23:28:18 -05:00
go.mod perf(server): gzip compression middleware on the entire mux 2026-05-03 23:31:18 -05:00
go.sum perf(server): gzip compression middleware on the entire mux 2026-05-03 23:31:18 -05:00
README.md refactor: separate website repo + deploy-host model 2026-05-02 09:14:40 -05:00
release.sh refactor: separate website repo + deploy-host model 2026-05-02 09:14:40 -05:00

zddc-server

A purpose-built HTTPS file server for ZDDC document archives. Designed to replace caddy file-server --browse with features specific to ZDDC workflows.

Features

  • High-performance static file serving — ETag, conditional GET, Cache-Control
  • Cascading .zddc ACL — email-based allow/deny lists evaluated bottom-up from requested directory to root
  • Caddy-compatible JSON listings — the Archive Browser works without modification
  • Virtual .archive index — resolve the earliest revision of any tracked document by URL
  • Filesystem watcher — archive index updates automatically when files change
  • Flexible TLS modes — self-signed, real certificates, or plain HTTP
  • Single static binary — CGO-free, no runtime dependencies; cross-compiled to Linux/macOS/Windows

Quick Start

zddc-server ships as a cross-compiled binary distributed via Codeberg release assets.

# Pick a tag from https://codeberg.org/VARASYS/ZDDC/releases (filter by zddc-server-v*)
curl -L -o zddc-server \
  https://codeberg.org/VARASYS/ZDDC/releases/download/zddc-server-vX.Y.Z/zddc-server-linux-amd64
chmod +x zddc-server

# Run against your archive root (HTTPS on :8443 with an in-memory self-signed cert)
ZDDC_ROOT=/srv/archive ./zddc-server

Or build from source (requires Go 1.24+):

git clone https://codeberg.org/VARASYS/ZDDC.git
cd ZDDC/zddc
go build -o zddc-server ./cmd/zddc-server
ZDDC_ROOT=/srv/archive ./zddc-server

For plain HTTP behind a reverse proxy, set ZDDC_TLS_CERT=none and ZDDC_INSECURE_DIRECT=1 — see "TLS" below.

There is no Containerfile / Dockerfile / compose file in this repo. Two ways to run zddc-server in Kubernetes / containers:

  • The example Helm charts under helm/ (zddc-server-prod/ for stable / zddc-server-dev/ for tracking main HEAD) compile zddc-server from source via init container — no image registry needed.
  • Roll your own image: copy the static binary into a FROM scratch or FROM alpine base in a few lines.

Environment Variables

Variable Default Description
ZDDC_ROOT (required) Absolute path to the served file tree
ZDDC_ADDR :8443 Bind address (host:port)
ZDDC_TLS_CERT (empty) Path to PEM certificate file. none = plain HTTP (no TLS); empty = generate self-signed
ZDDC_TLS_KEY (empty) Path to PEM private key file. Required when ZDDC_TLS_CERT is a file path; ignored otherwise
ZDDC_INSECURE_DIRECT (empty) Must be 1 when ZDDC_TLS_CERT=none and the bind address is non-loopback. Acknowledges that an authenticating reverse proxy is in front of zddc-server; without it, plain-HTTP non-loopback startup is refused
ZDDC_LOG_LEVEL info Log level: debug, info, warn, error
ZDDC_INDEX_PATH .archive URL path segment name for the virtual archive index
ZDDC_EMAIL_HEADER X-Auth-Request-Email HTTP request header containing the authenticated user's email (the oauth2-proxy / nginx auth-request convention)
ZDDC_CORS_ORIGIN https://zddc.varasys.io Comma-separated allowlist of origins permitted to make cross-origin requests. Empty value disables CORS entirely. Default lets ZDDC tools served from zddc.varasys.io (e.g. via the bootstrap pattern) call back into your deployed server.

ZDDC_TLS_CERT=none disables TLS entirely (plain HTTP). Both cert and key must be set together when using real certificates.

CORS

The default ZDDC_CORS_ORIGIN=https://zddc.varasys.io exists so the canonical ZDDC tool builds (hosted at zddc.varasys.io) can call back into your deployed zddc-server without extra configuration. If you self-host the tools on your own domain (e.g. tools.acme.com), set:

ZDDC_CORS_ORIGIN=https://tools.acme.com

Multiple origins are comma-separated. To disable CORS entirely (e.g. when all clients are same-origin), set ZDDC_CORS_ORIGIN= (empty value). The middleware echoes the matched origin back per-request and sets Access-Control-Allow-Credentials: true so the upstream-set X-Auth-Request-Email header crosses the boundary.

TLS

Plain HTTP (no TLS)

Set ZDDC_TLS_CERT=none to run without TLS. Recommended when an upstream reverse proxy (nginx, Caddy, Traefik) terminates external TLS and talks to zddc-server over plain HTTP on a private network. zddc-server requires ZDDC_INSECURE_DIRECT=1 for any non-loopback bind in this mode — an explicit acknowledgement that an authenticating proxy sits in front:

ZDDC_ROOT=/srv/archive \
ZDDC_TLS_CERT=none \
ZDDC_ADDR=:8080 \
ZDDC_INSECURE_DIRECT=1 \
  ./zddc-server

When ZDDC_TLS_CERT / ZDDC_TLS_KEY are empty (or when using real certificates), zddc-server generates an ECDSA P-256 self-signed certificate in memory at startup. The certificate changes on every restart — this is intentional and acceptable when an upstream reverse proxy terminates external TLS and uses this server only for encrypted in-datacenter transport.

To use a real certificate (e.g. from Let's Encrypt or an internal CA):

ZDDC_ROOT=/srv/archive \
ZDDC_TLS_CERT=/etc/ssl/zddc/server.crt \
ZDDC_TLS_KEY=/etc/ssl/zddc/server.key \
  ./zddc-server

Authentication

zddc-server does not perform authentication itself. It reads the user's email address from a request header (default: X-Auth-Request-Email) that must be set by an upstream reverse proxy (nginx, Caddy, Traefik, Azure Application Gateway, etc.) after authenticating the user.

If the header is absent, the user is treated as anonymous (empty email). A directory with no .zddc rules is publicly accessible; a directory with an allowlist requires a matching email.

.zddc Access Control Files

Place a .zddc YAML file in any directory to control access. Rules cascade from parent directories — child rules are appended to (not replaced by) parent rules.

# Example .zddc file
acl:
  allow:
    - "*@mycompany.com"       # all users at mycompany.com
    - "contractor@partner.com" # specific external user
  deny:
    - "intern@mycompany.com"  # override: block this specific user

ACL evaluation order

Rules are evaluated bottom-up: starting at the requested directory and walking toward the root. The first explicit match (allow or deny) at any level wins.

  1. Check deny patterns at the current level — if email matches → 403 Forbidden
  2. Check allow patterns at the current level — if email matches → allow
  3. No match at this level → walk up to parent directory and repeat
  4. If no .zddc files were found anywhere in the chain → allow (public, no rules)
  5. If .zddc files exist but email matched nothing → 403 Forbidden (not on any list)

This model supports three user tiers in a single tree:

Level Rule Effect
Root allow: ["*@company.com"] All company users see everything
Project dir allow: ["team@company.com"] Restricts to the project team
Vendor subdir allow: ["vendor@ext.com"] Grants a third-party access to their folder only

A vendor navigating to their subdirectory is allowed by the deepest matching rule, even if a higher-level rule would deny them.

Glob patterns

* matches any sequence of characters within one side of the @ boundary:

Pattern Matches
*@mycompany.com Any user at mycompany.com
alice@* alice at any domain
* Any non-empty email
alice@example.com Exact match only

Directory visibility

Directories for which the user lacks access are omitted from JSON listings entirely — they are neither listed nor queryable. A direct request to a denied path returns 403.

Reserved hidden segments

Two prefixes are filtered from listings under ZDDC_ROOT:

  • .-prefixed (e.g. /.devshell/, /Project-A/.internal/notes.md) — excluded from listings and 404 on direct HTTP access. The recognized virtual prefixes (.archive, .admin) are explicitly permitted through. This lets operators store side-state (caches, dev-shell home dirs, snapshot staging) on the same volume that's served, without exposing it.
  • _-prefixed (e.g. /_template/) — excluded from listings only. Direct URL access still works. Use this for operator-managed scaffolding the user shouldn't browse to but might link to (e.g. a _template/ directory of stub-HTML examples to copy into project subdirs).

Admin Debug Page

zddc-server exposes a built-in debug page at /.admin/ for operators who can push code/images but cannot kubectl exec into the running container. It surfaces:

  • /.admin/whoami — every header on the current request, the configured email header name, the value observed at that name, and the resolved email. This is the first thing to look at when access logs show email=anonymous — it tells you exactly which (if any) header the upstream proxy is sending.
  • /.admin/config — the resolved Config (env vars). Equivalent to kubectl exec -- env | grep ^ZDDC_ for diagnosing chart / deployment overrides.
  • /.admin/logs — recent log entries (last 500) from an in-memory ring buffer. Optional ?level=info|warn|error|debug and ?since=<RFC3339> query params. At ZDDC_LOG_LEVEL=debug every request also logs its full header map under msg=request headers — useful for diagnosing proxy / SSO header passthrough (e.g. confirming which header carries the email). Note: that dump includes auth tokens and cookies; only enable debug in trusted environments.
  • /.admin/ — HTML dashboard that fetches the three JSON endpoints client-side.

Authorization

Authorization is via an admins: list in the root .zddc file (<ZDDC_ROOT>/.zddc). Patterns use the same glob syntax as acl.allow / acl.deny:

admins:
  - alice@mycompany.com
  - "*@admin.mycompany.com"
acl:
  allow:
    - "*@mycompany.com"

Only the root-level admins entry is honored — subdirectory .zddc files' admins keys are ignored. Otherwise anyone with subtree write access could elevate themselves.

If the root .zddc has no admins list (or no .zddc exists), every admin endpoint returns 404 to every caller. Non-admin requests also receive 404 (not 403) so the existence of the admin page is invisible to unauthorized callers.

Forward-auth target for upstream proxies

zddc-server also exposes GET /.auth/admin — a machine-only endpoint that returns 200 if the caller's resolved email is in the root .zddc admins: list, 403 otherwise. No body, no redirect, no UI; it is a pure authorization decision intended to be polled by an upstream proxy's forward-auth directive (Caddy forward_auth, nginx auth_request, Traefik ForwardAuth, etc.).

The intended use case is gating adjacent services on the same pod / host that don't have their own ACL. Concretely: the dev-shell deployment runs both zddc-server and code-server behind one Caddy listener; Caddy uses forward_auth to ask /.auth/admin whether the caller is allowed to reach /devshell/* (the IDE) before forwarding. zddc-server's own routes (/, /<project>/, /.archive/, etc.) keep their existing .zddc-cascade ACL and don't go through this endpoint.

# example: protect /devshell/* with forward_auth on /.auth/admin
handle_path /devshell/* {
    forward_auth 127.0.0.1:9090 {
        uri /.auth/admin
        copy_headers X-Auth-Request-Email
    }
    reverse_proxy 127.0.0.1:8443  # code-server
}

The check is cheap (one map lookup against the cached PolicyChain); calling it on every request is fine. Edits to /srv/.zddc propagate within the fsnotify watcher's debounce window (~2 s) — no service restart needed.

Caveats

  • Logs are in-memory and lost on restart. The buffer holds the most recent 500 records; for long-term audit, parse the stderr stream the way you already do.
  • The page reads only configuration and request state — it does not modify anything.
  • An interactive terminal is not yet available; that's planned as a follow-up behind a separate ZDDC_ADMIN_TERM=1 env-var gate so it stays opt-in.

Apps: virtual tool HTMLs

zddc-server virtually serves the five tool HTMLs (archive, transmittal, classifier, mdedit, landing) at the appropriate paths. The current-stable build of each tool is baked into the binary at compile time via //go:embed; that's the default. No fetch happens out of the box.

Where each tool is served

App Available at
archive every directory (multi-project, project, archive, vendor)
classifier any Incoming, Working, or Staging directory and its subtree
mdedit any Working directory and its subtree
transmittal any Staging directory and its subtree
landing only at the deployment root (the project picker)

Outside these locations, the corresponding <app>.html URL returns 404.

Override and version-pin

For any path, the resolution order is:

  1. Real file at the path — operator drops archive.html (or any other) into a directory; the static handler serves it. Beats everything below.
  2. Closer-to-leaf .zddc apps: entry — walks .zddc files leaf→root for an apps.<app> entry. The first match wins. Spec is one of:
    • stable / beta / alpha (canonical upstream channel)
    • v0.0.4 / v0.0 / v0 (canonical upstream version pin)
    • https://... (full URL to a custom mirror)
    • ./local.html / /abs/path.html (local file)
  3. Embedded — the build-time HTML compiled into the binary.

URL sources are fetched once on first request and cached forever in <ZDDC_ROOT>/_app/<host>/<path>. There is no background refresh and no hash verification — to pull a new build, delete the cache file. Concurrent misses for the same URL share one outbound fetch (singleflight). Direct URL access to /_app/... is blocked at dispatch; cached HTMLs are served only via the apps resolver.

If a configured URL fetch fails (network down, 5xx), the server falls back to the embedded copy and emits a one-time WARN log per source. The X-ZDDC-Source response header always reports what was served: fetch:URL, cache:URL, path:/abs, or embedded:<app>@<build>.

Example

# <ZDDC_ROOT>/Project-A/.zddc
apps:
  classifier: alpha                                                # track alpha for this project
  archive: https://my-mirror.internal/zddc/archive_v0.0.4.html     # custom mirror, pinned
  mdedit: ./our-mdedit.html                                        # local fork

Env vars

Variable Default Purpose
ZDDC_BUILD_VERSION dev String stamped into X-ZDDC-Source: embedded:<app>@<v>

The landing page fetches GET / (with Accept: application/json) to retrieve the list of top-level project directories the requesting user has access to. It renders checkboxes for each project and opens archive.html?projects=Proj-A,Proj-B when the user clicks "Open Archive".

Presets (named project selections) are stored in the browser's localStorage — no server-side state required.

Shared URLs: the ?projects= parameter is preserved in the archive browser URL so users can email direct links to a pre-filtered view. If the recipient does not have access to a project listed in the URL, a warning banner is shown.

Access Logging

Every HTTP request is logged as a structured slog entry at INFO level:

Field Description
ts Request arrival timestamp (RFC3339)
email User email from the configured header, or anonymous
method HTTP method
path URL path
status HTTP response status code
bytes Response body bytes written
duration_ms Request duration in milliseconds

Log output goes to stderr. Use ZDDC_LOG_LEVEL=warn to suppress access logs if needed, or pipe stderr to a log aggregator.

Virtual Archive Index (.archive)

Any URL path segment named .archive (configurable via ZDDC_INDEX_PATH) is intercepted by the server and treated as a virtual document index.

The index is built at startup by scanning all transmittal folders under ZDDC_ROOT. It maps each (project, trackingNumber, revision, modifier) tuple to the file from the chronologically earliest transmittal folder within that project that contains it.

Project scoping

The .archive index is scoped to the project — i.e. the first slash-separated segment of the request's .archive context path. The same tracking number issued under two different projects does NOT collide; each project's .archive/ surfaces only that project's documents.

A request to /.archive/... at the very root has no project segment to scope by and returns 404 Not Found. Stable references must always be project-rooted (e.g. /ProjectA/.archive/TRK-001.html).

Within one project, two different files claiming to be the same (tracking, rev) are an authoring mistake. The chronological winner still wins, but a WARN log is emitted with both paths so the conflict can be diagnosed and corrected.

URL patterns

URL Resolves to
GET /Project/.archive/TRK-001.html Latest base revision of TRK-001 within Project
GET /Project/.archive/TRK-001_A.html Base revision A of TRK-001 within Project
GET /Project/.archive/TRK-001_A+C1.html Modifier C1 of revision A of TRK-001 within Project
GET /Project/.archive/ JSON listing of Project's resolvable entries
GET /Project/sub/sub/.archive/TRK-001.html Same as the top-level Project listing — depth within a project doesn't change scope
GET /.archive/... 404 — root has no project segment

All successful responses are 302 Found redirects to the actual file URL. ACL is enforced on both the .archive context directory and the resolved target file.

Why "earliest" transmittal?

Within one project, any file claiming to be TRK-001_A (IFC) should be identical across transmittals (same content, same SHA-256). If the same tracking number and revision appears in multiple transmittals, the first one received chronologically is treated as the authoritative copy. A later arrival with a different file path is an error condition; the server logs a WARN with both paths but does not change the winner.

Index refresh

The index refreshes automatically via an fsnotify filesystem watcher. Changes are debounced by 2 seconds before the relevant transmittal folder is re-indexed.

Note for Azure Files: Azure SMB mounts do not support inotify/fsnotify reliably. The watcher will log a warning and the index will only be updated by restarting the server.

ZDDC Filename Convention

The server parses filenames following the ZDDC convention:

trackingNumber_revision (status) - title.extension
Part Format Example
trackingNumber No spaces or underscores 123456-EL-SPC-2623
revision ~?[A-Z0-9]+(\+[CBNQ][0-9]+)? A, ~B, C+C1
status One of the valid status codes IFC, REC, ---
title Free text Electrical Specification

Valid status codes: IFA IFB IFC IFD IFI IFP IFR IFU REC RSA RSB RSC RSD RSI ---

Transmittal folder format: YYYY-MM-DD_trackingNumber (STATUS) - title

Integration with Archive Browser

The Archive Browser (archive.html) can connect to zddc-server in HTTP mode. The server returns JSON directory listings in exactly the same format as Caddy's file-server --browse — no changes to archive/js/source.js are needed.

To use: install archive.html at ZDDC_ROOT/archive.html (or any subdirectory) — either the actual built tool downloaded by the self-contained install snippet, or one of the six-line stubs from the project-subdir / track-upstream snippets that fetches it. Then open it via the zddc-server URL; the app will auto-connect and scan the directory tree.

Distribution

Each stable release is a Codeberg git tag (zddc-server-vX.Y.Z) with four pre-built binaries attached as release assets:

File Platform
zddc-server-linux-amd64 Linux (x86-64)
zddc-server-darwin-amd64 macOS (Intel)
zddc-server-darwin-arm64 macOS (Apple Silicon)
zddc-server-windows-amd64.exe Windows (x86-64)

All binaries are statically linked (CGO disabled), built with -trimpath -ldflags="-s -w -X main.version=<ver>". No runtime dependencies.

Download URLs from Codeberg directly:

https://codeberg.org/VARASYS/ZDDC/releases/download/zddc-server-vX.Y.Z/zddc-server-linux-amd64

Browse all releases at https://codeberg.org/VARASYS/ZDDC/releases.

There is no alpha/beta channel for binary distribution. Active dev/soak happens via the helm/zddc-server-dev/ chart, which builds zddc-server from source on every pod restart against any commit you point it at. There is no container image; if you want your own, copy the static binary into a FROM scratch or FROM alpine base in a few lines, or use one of the helm charts which compile from source via init container.

Env-var contract (for chart consumers)

Downstream Helm charts and Compose files should set these explicitly:

Variable Typical value (behind ingress + SSO) Purpose
ZDDC_ROOT /srv Path of the served archive (volume mount)
ZDDC_TLS_CERT none TLS terminated upstream
ZDDC_INSECURE_DIRECT 1 Acknowledge plain HTTP behind a trusted proxy
ZDDC_ADDR :8080 Match service / probe port
ZDDC_EMAIL_HEADER X-Auth-Request-Email Header your auth proxy sets
ZDDC_CORS_ORIGIN https://your-host Origins permitted to call back into the server

See "Environment Variables" above for the full list.

Building from source

Requires Go 1.24+.

# Single binary for the host platform
(cd zddc && go build -o zddc-server ./cmd/zddc-server)

# All four release platforms (cross-compiled, statically linked)
./build   # at the repo root — silently skips if Go isn't on PATH
              # → outputs to zddc/dist/zddc-server-{linux,darwin,windows}-*

To run unit tests:

(cd zddc && go test ./...)

Release tagging

zddc-server has no separate release script. The repo's top-level ./build alpha|beta|release [version] is the canonical path: it cross-compiles the four binaries inside the containerized Go toolchain, copies them into dist/release-output/ with the lockstep symlink chain (one set per platform), regenerates the per-version + per-channel stub pages, refreshes the index, and (on stable cuts) tags zddc-server-v<X.Y.Z> alongside the five HTML-tool tags.

./build release            # lockstep stable, coordinated next version
./build release 1.2.0      # lockstep stable, explicit version
./build alpha              # lockstep alpha cut
./build beta               # lockstep beta cut
./deploy --releases        # publish dist/release-output/ to /srv/zddc/releases/

The script tags every tool but does NOT push — finish with git push origin main && git push origin --tags (and run ./deploy to put the artifacts on the live site).

Prerequisites:

  • Go 1.24+ available inside the build container (downloaded automatically — docker.io/golang:1.24-alpine).
  • podman (preferred) or docker on PATH.

Single-developer / solo-release flow by design — no CI babysitting, no separate dashboard to debug. The script fails loudly and visibly on the developer's terminal if anything goes wrong.

Versioning

Clean semver, lockstep across all six tools (5 HTML + zddc-server). Stable cuts get <tool>-vX.Y.Z tags for every tool, all six sharing the same X.Y.Z. There are no alpha/beta tags — channel URLs are stable URLs by design (counters defeat that). Active dev runs via helm/zddc-server-dev/, which builds from source on each rollout.

The two existing zddc-server-v0.0.8-alpha.1 and zddc-server-v0.0.8-alpha.2 tags from a previous experiment stay as historical artifacts; no new alpha/beta tags are created going forward.


Notes:

  • The .archive virtual path resolves ZDDC tracking numbers to their earliest-received revision
  • ACL is enforced via bottom-up .zddc file evaluation