Compare commits

...

5 commits

Author SHA1 Message Date
9a3e4d8fa7 chore(embedded): cut v0.0.16-beta with loading-efficiency wins
All checks were successful
Notify chart dev on beta cut / notify-chart-dev (push) Successful in 6s
Bake into the dev binary:
  - ETag + max-age=0 on embedded HTML (304s on repeat loads)
  - gzip compression middleware (~75% wire-size reduction)
  - vendored jszip + docx-preview in archive/transmittal/classifier
  - tee'd file-based access log via --access-log
2026-05-04 06:29:23 -05:00
8df0defbd2 feat(server): tee access log to a rotated file for on-disk audit trail
Add --access-log <path> (env ZDDC_ACCESS_LOG). When set, every access-
log record is written as a JSON line to the configured file in
addition to the existing slog.Default() stderr output. Empty (default)
keeps the prior behavior — stderr only.

Rotation via gopkg.in/natefinch/lumberjack.v2:
  100 MB per file, 10 backups, 90-day max age, gzip rotated files.

Operator usage (e.g. behind a Caddy/quadlet stack):
  zddc-server --access-log /srv/.zddc.d/logs/access.log ...

Architecture:
  AccessLogMiddleware now takes an optional *slog.Logger. main.go wires
  it via setupAccessAuditLog() which builds a slog.JSONHandler over a
  lumberjack rotator. Stderr emission stays via slog.Default(); the
  audit logger gets the same fields in line-delimited JSON, the format
  every standard log shipper (Vector, Loki, fluentbit, journalbeat)
  parses natively.

Tests cover the audit logger receiving the same email/path/status
fields as the stderr stream.
2026-05-03 23:38:14 -05:00
eaecaaee29 perf(tools): vendor jszip + docx-preview for archive/transmittal/classifier
Same pattern as the browse fix. archive, transmittal, classifier
previously CDN-loaded jszip + docx-preview on first preview of a
.zip / .docx file via shared/preview-lib.js's loadLibrary helper.
That meant each first-preview blocked on a CDN round-trip + parse,
and broke entirely under restrictive networks or CSPs.

Vendor both libs under shared/vendor/ and concat them at the top of
each tool's build, ahead of init.js. window.JSZip + window.docx are
now defined immediately on page load. Drop the redundant loadLibrary
calls (and classifier's stray <script src="cdn..."> tag in the
template, plus archive's bespoke loadJSZip helper in export.js).

xlsx (SheetJS) intentionally stays CDN-loaded — at ~900 KB it's too
large to inline, and only fires on .xlsx preview which is a rarer
path.

Bundle size impact (uncompressed):
  archive:     304 KB → 476 KB  (+172 KB)
  transmittal: 449 KB → 621 KB  (+172 KB)
  classifier:  252 KB → 424 KB  (+172 KB)

With the gzip middleware (~75% reduction on HTML) and ETag-cached
revalidation now in place, the wire-size delta is ~40 KB per tool
on the first load and 0 on every subsequent load until redeploy.
2026-05-03 23:34:28 -05:00
c22bb19dab perf(server): gzip compression middleware on the entire mux
Add github.com/klauspost/compress/gzhttp wrapper around the request
handler. With MinSize(1024), responses ≥ 1 KB get gzip-encoded when
the client advertises Accept-Encoding: gzip; smaller bodies + 304
Not Modified pass through unchanged.

The wrapper auto-appends Vary: Accept-Encoding (compatible with the
existing Vary: Accept on directory.go's content-negotiated path).

Live-tested against zddc-server -root /tmp/empty:
  GET / w/ Accept-Encoding: gzip → 20.9 KB compressed (was 80.9 KB
                                   uncompressed). 74% reduction.
  Decompresses cleanly back to the original bytes.

Helps every code path that bypasses Caddy: devshell pods, local dev
binaries, tests, anywhere zddc-server is hit directly. Production
behind Caddy already had compression at the proxy layer; this just
makes the Go server self-sufficient.

Tests in cmd/zddc-server/main_test.go cover:
- large body + Accept-Encoding → compressed + Vary header
- small body → not compressed (under MinSize)
- no Accept-Encoding header → plain bytes
2026-05-03 23:31:18 -05:00
e021f14609 perf(server): ETag + max-age=0 on embedded HTML responses
The apps subsystem previously sent Cache-Control: public, max-age=300|3600,
must-revalidate but no ETag. With must-revalidate and no validator, the
browser cannot return 304 — it has to refetch the full body once max-age
expires. For mdedit that's 920 KB on every reload after an hour.

Add a content-addressed ETag (sha256 hex prefix, 32 chars) to:
- apps/handler.go's serveBody + serveEmbedded (both paths now emit ETag
  + handle If-None-Match short-circuit to 304)
- handler/directory.go's embedded:browse fallback (mirror behavior so
  the bare-directory landing serves the same way)

Drop max-age to 0 with must-revalidate: every page load revalidates,
but a matching ETag returns 304 with empty body. Steady-state cost of
a reload drops from N KB to a few hundred bytes. When the binary is
redeployed, the ETag changes (content hash) and the next request
returns 200 with the new bytes.

Tests in apps/handler_test.go cover both paths:
- TestServer_Embedded_ConditionalGET: full GET, matching INM, stale INM
- TestEmbeddedETag_Stable: same bytes → same ETag, different → different

Live smoke (curl against zddc-server -root /tmp/empty):
  GET /            → 200, ETag set, body = 80919 bytes (landing.html)
  GET / + INM:tag  → 304 Not Modified, empty body
2026-05-03 23:28:18 -05:00
29 changed files with 517 additions and 104 deletions

View file

@ -27,8 +27,14 @@ concat_files \
"css/print.css" \
> "$css_temp"
# JavaScript files to concatenate in order
# JavaScript files to concatenate in order. Vendored libraries first
# (jszip, docx-preview) so window.JSZip + window.docx are defined before
# any tool code runs — replaces the previous CDN loadLibrary() calls in
# table.js + export.js. xlsx is intentionally still CDN-loaded on demand
# (~900 KB; too large to inline).
concat_files \
"../shared/vendor/jszip.min.js" \
"../shared/vendor/docx-preview.min.js" \
"../shared/zddc.js" \
"../shared/hash.js" \
"../shared/theme.js" \

View file

@ -53,12 +53,14 @@
return;
}
// Check if JSZip is loaded
// JSZip is vendored (concat'd by build.sh), so window.JSZip is
// already defined. Defensive check in case a future refactor
// reorders things.
if (typeof JSZip === 'undefined') {
// Dynamically load JSZip
await loadJSZip();
alert('JSZip library not bundled — rebuild archive with shared/vendor/jszip.min.js');
return;
}
const zip = new JSZip();
const selectedFiles = [];
@ -123,17 +125,6 @@
}
}
// Load JSZip library dynamically
function loadJSZip() {
return new Promise((resolve, reject) => {
const script = document.createElement('script');
script.src = 'https://cdnjs.cloudflare.com/ajax/libs/jszip/3.10.1/jszip.min.js';
script.onload = resolve;
script.onerror = reject;
document.head.appendChild(script);
});
}
// Show progress indicator
function showProgress(message, current, total) {
let progressDiv = document.getElementById('progressIndicator');
@ -260,7 +251,6 @@
rowsToCSV,
exportCSV,
downloadSelected,
loadJSZip,
showProgress,
hideProgress,
downloadFile,

View file

@ -609,13 +609,13 @@
if (!container) return;
try {
await loadLibrary('https://cdnjs.cloudflare.com/ajax/libs/jszip/3.10.1/jszip.min.js');
await loadLibrary('https://cdn.jsdelivr.net/npm/docx-preview@latest/dist/docx-preview.min.js');
// jszip + docx-preview are vendored (concatenated by build.sh
// ahead of every tool module), so window.JSZip and window.docx
// are already defined here.
const arrayBuffer = await (file.handle
? file.handle.getFile().then(f => f.arrayBuffer())
: fetch(file.url).then(r => r.arrayBuffer()));
container.innerHTML = '';
await window.docx.renderAsync(arrayBuffer, container);
} catch (err) {

View file

@ -25,8 +25,14 @@ concat_files \
"css/spreadsheet.css" \
> "$css_temp"
# JavaScript files to concatenate in order
# JavaScript files to concatenate in order. Vendored libraries first
# (jszip, docx-preview) so window.JSZip + window.docx are defined before
# any tool code runs. Replaces the previous <script src="cdn..."> tag in
# template.html plus the loadLibrary CDN calls in preview.js. xlsx stays
# CDN-loaded on demand (~900 KB; too large to inline).
concat_files \
"../shared/vendor/jszip.min.js" \
"../shared/vendor/docx-preview.min.js" \
"../shared/zddc.js" \
"../shared/hash.js" \
"../shared/theme.js" \

View file

@ -383,12 +383,10 @@
if (!container) return;
try {
await loadLibrary('https://cdn.jsdelivr.net/npm/jszip@3/dist/jszip.min.js');
await loadLibrary('https://cdn.jsdelivr.net/npm/docx-preview@latest/dist/docx-preview.min.js');
// jszip + docx-preview vendored by build.sh — already in scope.
const blob = await getFileBlob(file);
const arrayBuffer = await blob.arrayBuffer();
container.innerHTML = '';
await window.docx.renderAsync(arrayBuffer, container);
} catch (err) {

View file

@ -5,7 +5,6 @@
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>ZDDC Classifier</title>
<link rel="icon" type="image/svg+xml" href="{{FAVICON}}">
<script src="https://cdnjs.cloudflare.com/ajax/libs/jszip/3.10.1/jszip.min.js"></script>
<style>
{{CSS_PLACEHOLDER}}
</style>

View file

@ -1792,7 +1792,7 @@ body.help-open .app-header {
</svg>
<div class="header-title-group">
<span class="app-header__title">ZDDC Markdown</span>
<span class="build-timestamp"><span style="color:red;font-weight:bold">v0.0.16-beta · 2026-05-04 · 62ce6e9</span></span>
<span class="build-timestamp"><span style="color:red;font-weight:bold">v0.0.16-beta · 2026-05-04 · 8df0def</span></span>
</div>
<button id="addDirectoryBtn" class="btn btn-primary" title="Add a local directory">Add Local Directory</button>
<button id="refreshHeaderBtn" class="btn btn-secondary hidden" title="Refresh directory" aria-label="Refresh" style="font-size:1.1rem;"></button>

8
shared/vendor/docx-preview.min.js vendored Normal file

File diff suppressed because one or more lines are too long

View file

@ -36,8 +36,13 @@ concat_files \
"css/print.css" \
> "$css_temp"
# JavaScript files to concatenate in order
# JavaScript files to concatenate in order. Vendored libraries first
# (jszip, docx-preview) so window.JSZip + window.docx are defined before
# any tool code runs — replaces the previous CDN loadLibrary() calls
# scattered through files-preview.js. xlsx stays CDN-loaded on demand.
concat_files \
"../shared/vendor/jszip.min.js" \
"../shared/vendor/docx-preview.min.js" \
"../shared/zddc.js" \
"../shared/hash.js" \
"../shared/theme.js" \

View file

@ -200,8 +200,7 @@
return;
}
try {
await loadLibrary('https://cdn.jsdelivr.net/npm/jszip@3/dist/jszip.min.js');
await loadLibrary('https://cdn.jsdelivr.net/npm/docx-preview@latest/dist/docx-preview.min.js');
// jszip + docx-preview vendored by build.sh — already in scope.
var arrayBuffer = await getFileArrayBuffer(file);
container.innerHTML = '';
await window.docx.renderAsync(arrayBuffer, container);
@ -474,7 +473,7 @@
}
try {
updatePreviewStatus('Loading ZIP...');
await loadLibrary('https://cdn.jsdelivr.net/npm/jszip@3/dist/jszip.min.js');
// JSZip vendored by build.sh — already in scope.
var arrayBuffer = await zipFile.arrayBuffer();
var zip = await JSZip.loadAsync(arrayBuffer);
var sourceEntries = [];

View file

@ -20,6 +20,9 @@ import (
"codeberg.org/VARASYS/ZDDC/zddc/internal/handler"
"codeberg.org/VARASYS/ZDDC/zddc/internal/tlsutil"
"codeberg.org/VARASYS/ZDDC/zddc/internal/zddc"
"github.com/klauspost/compress/gzhttp"
"gopkg.in/natefinch/lumberjack.v2"
)
// version is the binary's own version, injected at build time via
@ -106,13 +109,20 @@ func main() {
// the context the outer ACL middleware set.
// CORSMiddleware — Origin / preflight handling.
// dispatch — the actual request handler.
mux.Handle("/", handler.ACLMiddleware(cfg, handler.AccessLogMiddleware(handler.CORSMiddleware(cfg, http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
auditLogger := setupAccessAuditLog(cfg.AccessLog)
mux.Handle("/", handler.ACLMiddleware(cfg, handler.AccessLogMiddleware(auditLogger, handler.CORSMiddleware(cfg, http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
dispatch(cfg, idx, logRing, appsServer, w, r)
})))))
gzWrapper, err := newGzipWrapper()
if err != nil {
slog.Error("gzhttp wrapper init", "err", err)
os.Exit(1)
}
srv := &http.Server{
Addr: cfg.Addr,
Handler: mux,
Handler: gzWrapper(mux),
TLSConfig: tlsCfg,
}
@ -146,6 +156,58 @@ func main() {
slog.Info("stopped")
}
// setupAccessAuditLog constructs a slog.Logger writing JSON lines to a
// size-rotated file at the operator-configured path. Returns nil if no
// path is configured — AccessLogMiddleware then logs only to stderr
// (existing behavior).
//
// Rotation is via lumberjack: 100 MB per file, 10 backups, 90-day max
// age, gzip compression on rotated files. Tuning is fixed (not exposed
// as flags) — these defaults match what an audit-trail use case needs;
// operators wanting stricter retention can wire up logrotate against
// the rotated files themselves.
//
// File-permission posture: lumberjack creates new logs with mode 0600
// (running user only). For multi-user audit access, the operator should
// use group-readable parent directory permissions and either chmod the
// log out-of-band or run a forwarder that has its own read access.
// Parent directory must already exist — this function does NOT mkdir,
// since we'd need to assume too much about umask/owner.
func setupAccessAuditLog(path string) *slog.Logger {
if path == "" {
return nil
}
rotator := &lumberjack.Logger{
Filename: path,
MaxSize: 100, // megabytes per file before rotation
MaxBackups: 10,
MaxAge: 90, // days
Compress: true,
}
// JSON handler — line-delimited JSON is the format every standard
// log shipper (Vector, Loki promtail, fluentbit, journalbeat) parses
// natively, and stays grep-friendly for ad-hoc inspection.
h := slog.NewJSONHandler(rotator, &slog.HandlerOptions{Level: slog.LevelInfo})
slog.Info("access log file enabled",
"path", path, "max_size_mb", 100, "max_backups", 10, "max_age_days", 90)
return slog.New(h)
}
// newGzipWrapper builds the gzip middleware applied to the entire mux.
// MinSize(1024) skips compressing tiny responses where the framing
// overhead exceeds the savings (304 Not Modified, error pages, small
// JSON listings under ~1 KB). The wrapper honors Accept-Encoding (passes
// through unchanged when the client doesn't advertise gzip), appends
// Vary: Accept-Encoding automatically, and passes through 304s untouched.
// Yields ~75% size reduction on the larger embedded HTML responses
// (mdedit: 920 KB → ~250 KB on the wire).
//
// Extracted so tests can construct an equivalent wrapper without going
// through the full main() server boot.
func newGzipWrapper() (func(http.Handler) http.HandlerFunc, error) {
return gzhttp.NewWrapper(gzhttp.MinSize(1024))
}
// setupApps creates the cache + fetcher + server. No seeding, no refresh,
// no admin UI — the server fetches once on first request, caches forever
// in <ZDDC_ROOT>/_app/, and falls back to the embedded HTML on any failure.

View file

@ -210,3 +210,81 @@ func mustWrite(t *testing.T, path, body string) {
t.Fatalf("write %s: %v", path, err)
}
}
// TestGzhttpWrapper_CompressesLargeResponses asserts the gzhttp wrapper
// behavior we wire in main(): responses above MinSize get gzip-encoded
// when the client advertises Accept-Encoding: gzip; small responses
// pass through uncompressed; HEAD requests still set Vary correctly.
//
// We construct the wrapper the same way main() does (1024 byte minsize)
// and exercise it against a tiny test handler — full end-to-end is
// covered by the live curl smoke test in CI / dev verification.
func TestGzhttpWrapper_CompressesLargeResponses(t *testing.T) {
// Re-create the wrapper config from main.go so this test stays in
// sync with the real wiring.
wrapper, err := newGzipWrapper()
if err != nil {
t.Fatalf("newGzipWrapper: %v", err)
}
largeBody := strings.Repeat("ZDDC ", 4000) // ~20 KB, well over MinSize
smallBody := "ok"
handler := wrapper(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "text/html; charset=utf-8")
if r.URL.Path == "/large" {
_, _ = w.Write([]byte(largeBody))
} else {
_, _ = w.Write([]byte(smallBody))
}
}))
srv := httptest.NewServer(handler)
defer srv.Close()
t.Run("large body with Accept-Encoding gzip → compressed", func(t *testing.T) {
req, _ := http.NewRequest(http.MethodGet, srv.URL+"/large", nil)
req.Header.Set("Accept-Encoding", "gzip")
// Disable transparent decompression so we can read the raw bytes
// and confirm the wire format.
client := &http.Client{Transport: &http.Transport{DisableCompression: true}}
resp, err := client.Do(req)
if err != nil {
t.Fatal(err)
}
defer resp.Body.Close()
if got := resp.Header.Get("Content-Encoding"); got != "gzip" {
t.Errorf("Content-Encoding = %q, want gzip", got)
}
if got := resp.Header.Get("Vary"); !strings.Contains(strings.ToLower(got), "accept-encoding") {
t.Errorf("Vary = %q, want to contain Accept-Encoding", got)
}
})
t.Run("small body → not compressed", func(t *testing.T) {
req, _ := http.NewRequest(http.MethodGet, srv.URL+"/small", nil)
req.Header.Set("Accept-Encoding", "gzip")
client := &http.Client{Transport: &http.Transport{DisableCompression: true}}
resp, err := client.Do(req)
if err != nil {
t.Fatal(err)
}
defer resp.Body.Close()
if got := resp.Header.Get("Content-Encoding"); got == "gzip" {
t.Errorf("Content-Encoding = gzip; small response should not be compressed")
}
})
t.Run("no Accept-Encoding → not compressed", func(t *testing.T) {
req, _ := http.NewRequest(http.MethodGet, srv.URL+"/large", nil)
client := &http.Client{Transport: &http.Transport{DisableCompression: true}}
resp, err := client.Do(req)
if err != nil {
t.Fatal(err)
}
defer resp.Body.Close()
if got := resp.Header.Get("Content-Encoding"); got != "" {
t.Errorf("Content-Encoding = %q; client without Accept-Encoding should get plain", got)
}
})
}

View file

@ -4,6 +4,8 @@ go 1.24
require (
github.com/fsnotify/fsnotify v1.9.0
github.com/klauspost/compress v1.18.6
gopkg.in/natefinch/lumberjack.v2 v2.2.1
gopkg.in/yaml.v3 v3.0.1
)

View file

@ -1,8 +1,12 @@
github.com/fsnotify/fsnotify v1.9.0 h1:2Ml+OJNzbYCTzsxtv8vKSFD9PbJjmhYF14k/jKC7S9k=
github.com/fsnotify/fsnotify v1.9.0/go.mod h1:8jBTzvmWwFyi3Pb8djgCCO5IBqzKJ/Jwo8TRcHyHii0=
github.com/klauspost/compress v1.18.6 h1:2jupLlAwFm95+YDR+NwD2MEfFO9d4z4Prjl1XXDjuao=
github.com/klauspost/compress v1.18.6/go.mod h1:cwPg85FWrGar70rWktvGQj8/hthj3wpl0PGDogxkrSQ=
golang.org/x/sys v0.26.0 h1:KHjCJyddX0LoSTb3J+vWpupP9p0oznkqVk/IfjymZbo=
golang.org/x/sys v0.26.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/natefinch/lumberjack.v2 v2.2.1 h1:bBRl1b0OH9s/DuPhuXpNl+VtCaJXFZ5/uEFST95x9zc=
gopkg.in/natefinch/lumberjack.v2 v2.2.1/go.mod h1:YD8tP3GAjkrDg1eZH7EGmyESg/lsYskCTPBJVb9jqSc=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=

View file

@ -1,6 +1,11 @@
package apps
import _ "embed"
import (
"crypto/sha256"
"encoding/hex"
_ "embed"
"sync"
)
// Embedded fallback: the five tool HTMLs from the time the binary was
// built. Used as a last-resort served-bytes when (cache miss) AND
@ -56,3 +61,29 @@ func EmbeddedBytes(app string) []byte {
}
return b
}
// EmbeddedETag returns a strong ETag (sha256-hex prefix, 32 chars) for the
// app's embedded bytes. Computed lazily on first call per-app and memoized
// — the embedded slot is fixed for the binary's lifetime, so the ETag
// changes only when the binary is redeployed. Empty slot returns "".
//
// Used by apps.Server.serveEmbedded to issue conditional-GET-friendly
// responses: with this ETag + Cache-Control: max-age=0, must-revalidate,
// every page load revalidates and gets a 304 unless the binary has been
// updated. Saves re-transmitting 50920 KB tool HTMLs on every reload.
func EmbeddedETag(app string) string {
if v, ok := etagCacheByApp.Load(app); ok {
return v.(string)
}
body := EmbeddedBytes(app)
if body == nil {
return ""
}
sum := sha256.Sum256(body)
etag := hex.EncodeToString(sum[:])[:32]
etagCacheByApp.Store(app, etag)
return etag
}
// etagCacheByApp memoizes EmbeddedETag results keyed by app name.
var etagCacheByApp sync.Map

File diff suppressed because one or more lines are too long

View file

@ -896,7 +896,7 @@ body {
</svg>
<div class="header-title-group">
<span class="app-header__title">ZDDC Browse</span>
<span class="build-timestamp"><span style="color:red;font-weight:bold">v0.0.16-beta · 2026-05-04 · 62ce6e9</span></span>
<span class="build-timestamp"><span style="color:red;font-weight:bold">v0.0.16-beta · 2026-05-04 · 8df0def</span></span>
</div>
<button id="addDirectoryBtn" class="btn btn-primary">Add Local Directory</button>
<button id="refreshHeaderBtn" class="btn btn-secondary hidden" title="Refresh listing" aria-label="Refresh listing" style="font-size:1.1rem;"></button>

File diff suppressed because one or more lines are too long

View file

@ -885,7 +885,7 @@ body {
</svg>
<div class="header-title-group">
<span class="app-header__title">ZDDC</span>
<span class="build-timestamp"><span style="color:red;font-weight:bold">v0.0.16-beta · 2026-05-04 · 62ce6e9</span></span>
<span class="build-timestamp"><span style="color:red;font-weight:bold">v0.0.16-beta · 2026-05-04 · 8df0def</span></span>
</div>
</div>
<div class="header-right">

View file

@ -1792,7 +1792,7 @@ body.help-open .app-header {
</svg>
<div class="header-title-group">
<span class="app-header__title">ZDDC Markdown</span>
<span class="build-timestamp"><span style="color:red;font-weight:bold">v0.0.16-beta · 2026-05-04 · 62ce6e9</span></span>
<span class="build-timestamp"><span style="color:red;font-weight:bold">v0.0.16-beta · 2026-05-04 · 8df0def</span></span>
</div>
<button id="addDirectoryBtn" class="btn btn-primary" title="Add a local directory">Add Local Directory</button>
<button id="refreshHeaderBtn" class="btn btn-secondary hidden" title="Refresh directory" aria-label="Refresh" style="font-size:1.1rem;"></button>

File diff suppressed because one or more lines are too long

View file

@ -1,8 +1,8 @@
# Generated by build.sh — do not edit. One <app>=<build label> per line.
archive=v0.0.16-beta · 2026-05-04 · 62ce6e9
transmittal=v0.0.16-beta · 2026-05-04 · 62ce6e9
classifier=v0.0.16-beta · 2026-05-04 · 62ce6e9
mdedit=v0.0.16-beta · 2026-05-04 · 62ce6e9
landing=v0.0.16-beta · 2026-05-04 · 62ce6e9
form=v0.0.16-beta · 2026-05-04 · 62ce6e9
browse=v0.0.16-beta · 2026-05-04 · 62ce6e9
archive=v0.0.16-beta · 2026-05-04 · 8df0def
transmittal=v0.0.16-beta · 2026-05-04 · 8df0def
classifier=v0.0.16-beta · 2026-05-04 · 8df0def
mdedit=v0.0.16-beta · 2026-05-04 · 8df0def
landing=v0.0.16-beta · 2026-05-04 · 8df0def
form=v0.0.16-beta · 2026-05-04 · 8df0def
browse=v0.0.16-beta · 2026-05-04 · 8df0def

View file

@ -1,13 +1,13 @@
package apps
import (
"bytes"
"crypto/sha256"
"encoding/hex"
"errors"
"net/http"
"os"
"path/filepath"
"strings"
"time"
"codeberg.org/VARASYS/ZDDC/zddc/internal/zddc"
)
@ -92,14 +92,14 @@ func (s *Server) Serve(w http.ResponseWriter, r *http.Request, app string, chain
// Malformed `.zddc` spec — operator's fault. Log and serve embedded.
s.Fetcher.Logger.Warn("apps.Resolve failed; serving embedded",
"app", app, "request_dir", requestDir, "err", err)
s.serveEmbedded(w, app, err)
s.serveEmbedded(w, r, app, err)
return
}
if !hasOverride {
// No `.zddc apps:` entry anywhere up the chain and no `?v=` either →
// embedded is the authoritative default.
s.serveEmbedded(w, app, nil)
s.serveEmbedded(w, r, app, nil)
return
}
@ -138,7 +138,7 @@ func (s *Server) Serve(w http.ResponseWriter, r *http.Request, app string, chain
s.Fetcher.Logger.Warn("path source unreadable; serving embedded",
"app", app, "path", src.Path, "err", err)
}
s.serveEmbedded(w, app, err)
s.serveEmbedded(w, r, app, err)
return
}
s.serveBody(w, r, body, "path:"+src.Path)
@ -149,7 +149,7 @@ func (s *Server) Serve(w http.ResponseWriter, r *http.Request, app string, chain
body, err := s.Fetcher.Fetch(r.Context(), src.URL)
if err != nil {
s.Fetcher.LogEmbeddedFallback(app, src.URL, err)
s.serveEmbedded(w, app, err)
s.serveEmbedded(w, r, app, err)
return
}
sourceTag := "fetch:" + src.URL
@ -161,14 +161,39 @@ func (s *Server) Serve(w http.ResponseWriter, r *http.Request, app string, chain
s.serveBody(w, r, body, sourceTag)
}
func (s *Server) serveBody(w http.ResponseWriter, r *http.Request, body []byte, sourceHeader string) {
w.Header().Set("Content-Type", "text/html; charset=utf-8")
// writeWithETag writes body with a strong ETag derived from `etag`, the
// cache-friendly headers, and short-circuits to 304 Not Modified when the
// client's `If-None-Match` matches. `max-age=0, must-revalidate` means the
// browser revalidates on every load — and the matching ETag returns 304
// with empty body, so the steady-state cost of a reload is ~200 bytes
// instead of the full HTML payload (50920 KB depending on the tool).
func writeWithETag(w http.ResponseWriter, r *http.Request, body []byte, etag, contentType, sourceHeader string) {
quotedTag := `"` + etag + `"`
w.Header().Set("ETag", quotedTag)
w.Header().Set("Cache-Control", "public, max-age=0, must-revalidate")
w.Header().Set("Content-Type", contentType)
w.Header().Set("X-ZDDC-Source", sourceHeader)
w.Header().Set("Cache-Control", "public, max-age=3600, must-revalidate")
http.ServeContent(w, r, "", time.Time{}, bytes.NewReader(body))
if match := r.Header.Get("If-None-Match"); match != "" && match == quotedTag {
w.WriteHeader(http.StatusNotModified)
return
}
_, _ = w.Write(body)
}
func (s *Server) serveEmbedded(w http.ResponseWriter, app string, _ error) {
// bodyETag computes a stable 32-hex-char ETag for an arbitrary body. Used
// for the URL/path-sourced response path (the bytes vary per cache-fetch
// or per file read, so memoizing per-app would be wrong).
func bodyETag(body []byte) string {
sum := sha256.Sum256(body)
return hex.EncodeToString(sum[:])[:32]
}
func (s *Server) serveBody(w http.ResponseWriter, r *http.Request, body []byte, sourceHeader string) {
writeWithETag(w, r, body, bodyETag(body), "text/html; charset=utf-8", sourceHeader)
}
func (s *Server) serveEmbedded(w http.ResponseWriter, r *http.Request, app string, _ error) {
body := EmbeddedBytes(app)
if len(body) == 0 {
w.Header().Set("Retry-After", "60")
@ -179,8 +204,7 @@ func (s *Server) serveEmbedded(w http.ResponseWriter, app string, _ error) {
http.StatusServiceUnavailable)
return
}
w.Header().Set("Content-Type", "text/html; charset=utf-8")
w.Header().Set("X-ZDDC-Source", "embedded:"+app+"@"+s.BuildVer)
w.Header().Set("Cache-Control", "public, max-age=300, must-revalidate")
_, _ = w.Write(body)
writeWithETag(w, r, body, EmbeddedETag(app),
"text/html; charset=utf-8",
"embedded:"+app+"@"+s.BuildVer)
}

View file

@ -290,3 +290,88 @@ func TestServer_VParam_FullURLForm(t *testing.T) {
t.Errorf("body=%q", rec.Body.String())
}
}
// TestServer_Embedded_ConditionalGET verifies the ETag/If-None-Match dance
// for the embedded fallback path: a fresh GET returns 200 with an ETag,
// and a follow-up with a matching If-None-Match returns 304 + empty body.
// This is the cache-friendliness fix that lets a browser revalidate
// against zddc-server's embedded HTML without re-transferring the bytes.
func TestServer_Embedded_ConditionalGET(t *testing.T) {
srv, _, root := newTestServer(t, []byte("upstream"))
saved := embeddedArchive
embeddedArchive = []byte("EMBEDDED archive bytes for ETag test")
defer func() {
embeddedArchive = saved
etagCacheByApp.Delete("archive") // reset memoization for sibling tests
}()
etagCacheByApp.Delete("archive") // ensure clean state for THIS test
chain := zddc.PolicyChain{Levels: []zddc.ZddcFile{{}}}
// First request: full body + ETag header.
rec1 := httptest.NewRecorder()
srv.Serve(rec1, httptest.NewRequest(http.MethodGet, "/archive.html", nil), "archive", chain, root)
if rec1.Code != http.StatusOK {
t.Fatalf("first GET: status=%d body=%s", rec1.Code, rec1.Body.String())
}
etag := rec1.Header().Get("ETag")
if etag == "" {
t.Fatalf("first GET: missing ETag header")
}
if cc := rec1.Header().Get("Cache-Control"); !strings.Contains(cc, "max-age=0") || !strings.Contains(cc, "must-revalidate") {
t.Errorf("first GET: Cache-Control=%q (want max-age=0 + must-revalidate)", cc)
}
if !strings.Contains(rec1.Body.String(), "EMBEDDED archive bytes") {
t.Errorf("first GET: body=%q", rec1.Body.String())
}
// Second request with matching If-None-Match: 304, empty body.
rec2 := httptest.NewRecorder()
req2 := httptest.NewRequest(http.MethodGet, "/archive.html", nil)
req2.Header.Set("If-None-Match", etag)
srv.Serve(rec2, req2, "archive", chain, root)
if rec2.Code != http.StatusNotModified {
t.Fatalf("If-None-Match match: status=%d (want 304)", rec2.Code)
}
if rec2.Body.Len() != 0 {
t.Errorf("304 response should have empty body; got %d bytes", rec2.Body.Len())
}
// Third request with stale If-None-Match: 200, full body.
rec3 := httptest.NewRecorder()
req3 := httptest.NewRequest(http.MethodGet, "/archive.html", nil)
req3.Header.Set("If-None-Match", `"deadbeef"`)
srv.Serve(rec3, req3, "archive", chain, root)
if rec3.Code != http.StatusOK {
t.Errorf("stale If-None-Match: status=%d (want 200)", rec3.Code)
}
if rec3.Body.Len() == 0 {
t.Errorf("stale If-None-Match: empty body; want full")
}
}
// TestEmbeddedETag_Stable asserts EmbeddedETag is deterministic and
// content-addressed: same bytes → same ETag, different bytes → different.
func TestEmbeddedETag_Stable(t *testing.T) {
saved := embeddedArchive
defer func() {
embeddedArchive = saved
etagCacheByApp.Delete("archive")
}()
embeddedArchive = []byte("alpha")
etagCacheByApp.Delete("archive")
a1 := EmbeddedETag("archive")
a2 := EmbeddedETag("archive")
if a1 == "" || a1 != a2 {
t.Errorf("EmbeddedETag should be stable for same bytes; got %q vs %q", a1, a2)
}
embeddedArchive = []byte("beta")
etagCacheByApp.Delete("archive")
b := EmbeddedETag("archive")
if b == a1 {
t.Errorf("EmbeddedETag should differ for different bytes; both %q", b)
}
}

View file

@ -23,6 +23,7 @@ type Config struct {
IndexPath string // --index-path / ZDDC_INDEX_PATH — virtual archive prefix (default .archive)
EmailHeader string // --email-header / ZDDC_EMAIL_HEADER — auth header name (default X-Auth-Request-Email)
CORSOrigins []string // --cors-origin / ZDDC_CORS_ORIGIN — comma-separated allowlist; default https://zddc.varasys.io; empty disables
AccessLog string // --access-log / ZDDC_ACCESS_LOG — file path for tee'd JSON access log; empty = stderr only
}
// ErrHelpRequested is returned by Load when --help is passed; the caller
@ -73,6 +74,8 @@ func Load(args []string) (Config, error) {
"Comma-separated CORS allowlist. Empty = CORS disabled. Default: ZDDC_CORS_ORIGIN or https://zddc.varasys.io.")
insecureDirectFlag := fs.Bool("insecure-direct", os.Getenv("ZDDC_INSECURE_DIRECT") == "1",
"Allow plain HTTP on non-loopback addresses (only safe behind an authenticating proxy).")
accessLogFlag := fs.String("access-log", os.Getenv("ZDDC_ACCESS_LOG"),
"Tee structured access logs to this file (JSON, size-rotated). Empty = stderr only.")
helpFlag := fs.Bool("help", false, "Print this help and exit.")
versionFlag := fs.Bool("version", false, "Print version info and exit.")
@ -113,6 +116,7 @@ func Load(args []string) (Config, error) {
IndexPath: *indexPathFlag,
EmailHeader: *emailHeaderFlag,
CORSOrigins: resolveCORS(corsFlagSet, *corsOriginFlag),
AccessLog: *accessLogFlag,
}
// Default Root to the current working directory.
@ -182,6 +186,7 @@ func Usage(w io.Writer) {
fs.String("email-header", "X-Auth-Request-Email", "HTTP header carrying the authenticated user's email.")
fs.String("cors-origin", "", "Comma-separated CORS allowlist. Empty = CORS disabled.")
fs.Bool("insecure-direct", false, "Allow plain HTTP on non-loopback addresses.")
fs.String("access-log", "", "Tee structured access logs to this file (JSON, size-rotated). Empty = stderr only.")
fs.Bool("help", false, "Print this help and exit.")
fs.Bool("version", false, "Print version info and exit.")
fs.PrintDefaults()

View file

@ -118,12 +118,19 @@ func ServeDirectory(cfg config.Config, w http.ResponseWriter, r *http.Request) {
}
return
}
// ETag + max-age=0 + must-revalidate: every request re-validates and
// gets a 304 unless the binary has been redeployed (the ETag is a
// content hash, computed once at startup and memoized in apps.embed).
// Saves re-transmitting ~230 KB of browse.html on every page load
// while still picking up redeploys immediately.
etag := `"` + apps.EmbeddedETag("browse") + `"`
w.Header().Set("ETag", etag)
w.Header().Set("Cache-Control", "public, max-age=0, must-revalidate")
w.Header().Set("Content-Type", "text/html; charset=utf-8")
w.Header().Set("X-ZDDC-Source", "embedded:browse")
// no-cache here too — browse.html has session-tied content (the
// directory listing it loads via fetch), and we want browser to
// always re-validate so deployed-binary updates appear immediately
// rather than after a 5-minute cache window.
w.Header().Set("Cache-Control", "no-cache")
if match := r.Header.Get("If-None-Match"); match != "" && match == etag {
w.WriteHeader(http.StatusNotModified)
return
}
_, _ = w.Write(body)
}

View file

@ -741,7 +741,7 @@ body.help-open .app-header {
</svg>
<div class="header-title-group">
<span class="app-header__title" id="form-title">ZDDC Form</span>
<span class="build-timestamp"><span style="color:red;font-weight:bold">v0.0.16-beta · 2026-05-04 · 62ce6e9</span></span>
<span class="build-timestamp"><span style="color:red;font-weight:bold">v0.0.16-beta · 2026-05-04 · 8df0def</span></span>
</div>
</div>
<div class="header-right">

View file

@ -67,8 +67,19 @@ func (rw *responseWriter) Write(b []byte) (int, error) {
return n, err
}
// AccessLogMiddleware logs a structured line per HTTP request after the response is written.
func AccessLogMiddleware(next http.Handler) http.Handler {
// AccessLogMiddleware logs a structured line per HTTP request after the
// response is written.
//
// Always emits to slog.Default() (stderr) so server-lifecycle logs and
// access logs share an output stream by default.
//
// If `auditLogger` is non-nil, the same structured fields are also written
// to it. The intended caller wires up auditLogger with a JSON handler
// pointing at a rotating file (see cmd/zddc-server's setupAccessAuditLog),
// so an operator gets a persisted audit trail on disk in addition to the
// stderr stream — useful when stderr is not journald-captured (e.g.
// container logging where the orchestrator drops stderr after restarts).
func AccessLogMiddleware(auditLogger *slog.Logger, next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
// Capture request start time
start := time.Now()
@ -88,8 +99,7 @@ func AccessLogMiddleware(next http.Handler) http.Handler {
email = "anonymous"
}
// Log access
slog.Info("access",
args := []any{
"ts", start.Format(time.RFC3339),
"email", email,
"method", r.Method,
@ -97,6 +107,15 @@ func AccessLogMiddleware(next http.Handler) http.Handler {
"status", wrapped.status,
"bytes", wrapped.bytes,
"duration_ms", durationMs,
)
}
// Stderr stream (existing behavior).
slog.Info("access", args...)
// Audit file (when configured). Same fields, separate handler so
// the file can be JSON-formatted regardless of stderr's handler.
if auditLogger != nil {
auditLogger.Info("access", args...)
}
})
}

View file

@ -32,7 +32,7 @@ func TestAccessLogReadsEmailFromACLContext(t *testing.T) {
// Correct order: ACL is outer, AccessLog is inner. AccessLog reads
// email from the context ACL populated.
chain := ACLMiddleware(cfg, AccessLogMiddleware(noop))
chain := ACLMiddleware(cfg, AccessLogMiddleware(nil, noop))
req := httptest.NewRequest(http.MethodGet, "/foo", nil)
req.Header.Set("X-Auth-Request-Email", "alice@example.com")
@ -60,7 +60,7 @@ func TestAccessLogAnonymousWhenNoEmail(t *testing.T) {
w.WriteHeader(http.StatusOK)
})
chain := ACLMiddleware(cfg, AccessLogMiddleware(noop))
chain := ACLMiddleware(cfg, AccessLogMiddleware(nil, noop))
req := httptest.NewRequest(http.MethodGet, "/foo", nil)
// Note: no X-Auth-Request-Email header set.
@ -90,7 +90,7 @@ func TestAccessLogOuterDoesNotSeeInnerContext(t *testing.T) {
})
// Inverted order — the ORIGINAL buggy chain.
chain := AccessLogMiddleware(ACLMiddleware(cfg, noop))
chain := AccessLogMiddleware(nil, ACLMiddleware(cfg, noop))
req := httptest.NewRequest(http.MethodGet, "/foo", nil)
req.Header.Set("X-Auth-Request-Email", "alice@example.com")
@ -104,3 +104,35 @@ func TestAccessLogOuterDoesNotSeeInnerContext(t *testing.T) {
t.Errorf("expected the inverted (buggy) chain to fall back to email=anonymous, got: %s", got)
}
}
// TestAccessLogMiddleware_AuditLoggerReceivesSameFields verifies the
// optional audit-logger argument: when non-nil, it gets a parallel copy
// of every access record. Used by main.go to tee access logs to a
// rotating file in addition to stderr.
func TestAccessLogMiddleware_AuditLoggerReceivesSameFields(t *testing.T) {
var auditBuf bytes.Buffer
auditLogger := slog.New(slog.NewJSONHandler(&auditBuf, &slog.HandlerOptions{Level: slog.LevelInfo}))
noop := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(http.StatusTeapot)
_, _ = w.Write([]byte("hi"))
})
cfg := config.Config{EmailHeader: "X-Auth-Request-Email"}
chain := ACLMiddleware(cfg, AccessLogMiddleware(auditLogger, noop))
req := httptest.NewRequest(http.MethodGet, "/some/path", nil)
req.Header.Set("X-Auth-Request-Email", "bob@example.com")
chain.ServeHTTP(httptest.NewRecorder(), req)
out := auditBuf.String()
if !strings.Contains(out, `"email":"bob@example.com"`) {
t.Errorf("audit log missing email field; got: %s", out)
}
if !strings.Contains(out, `"path":"/some/path"`) {
t.Errorf("audit log missing path; got: %s", out)
}
if !strings.Contains(out, `"status":418`) {
t.Errorf("audit log missing status code; got: %s", out)
}
}