ZDDC/ARCHITECTURE.md

# ZDDC Architecture

This document is the single authoritative reference for how ZDDC tools are designed and built. It covers the shared single-file HTML application pattern, the build system, tool-specific architectural decisions, and contribution guidelines.

---

## Why Single-File HTML Applications

Every ZDDC tool compiles to a single self-contained `.html` file — no servers, no installers, no subscriptions.

| Principle | Rationale |
|-----------|-----------|
| **Reliability** | Opens in any modern Chromium-based browser without network access or external services |
| **Portability** | Can be emailed, archived, or deployed to air-gapped environments with no tooling |
| **Auditability** | Source, embedded data, and output travel together, satisfying ZDDC traceability requirements |
| **Longevity** | Static assets remain functional long after build environments have changed |
| **Simplicity** | A single `.html` file eliminates deployment steps and brittle dependency chains |

---

## Repository Structure

Every HTML tool follows the same directory layout:

```
tool/
  README.md        # Feature scope, UI design, domain rules, help content
  css/             # Logically separated stylesheets (one responsibility per file)
  js/              # Vanilla ES modules (one responsibility per file)
  template.html    # Shell markup with {{PLACEHOLDER}} markers for development
  build.sh         # Inlines css/ and js/ into dist/tool.html
  dist/
    tool.html      # Generated output — never edit this manually
```

Website files (what `zddc.varasys.io` serves) — committed in this repo as static assets, but the actual built tool HTML and zddc-server binaries live on Codeberg as release assets:

```
website/
  index.html                    # hand-edited intro page (root URL)
  releases/
    index.html                  # versions index, regenerated by build.sh from the Codeberg API
    manifest.json               # <tool>-<channel> → tag map, regenerated by build.sh; the level-2 stub fetches this
  bootstrap/
    level1/<tool>.html          # same-origin stubs for project subdirectories
    track-{alpha,beta,stable}/  # per-channel level-2 stubs (5 tools each)
```

The per-version `<tool>_v<X.Y.Z>.html` files and zddc-server binaries are **not** committed — they live on Codeberg as release assets attached to git tags. Caddy at `zddc.varasys.io` reverse-proxies `/releases/<tag>/<asset>` to the corresponding Codeberg URL, so consumers (operators' bootstrap stubs, `zddc-use`, the chart Dockerfiles) only ever talk to `zddc.varasys.io`.

There is no `website/dev/`. To preview a build locally, open `dist/tool.html` directly via the dev server. To publish on `zddc.varasys.io`, cut a release.

Vendor dependencies (bundled third-party libraries) live in `tool/vendor/` if present. The build script is responsible for inlining them into the output.

---

## Documentation ownership

Each topic has exactly one authoritative home; everything else links to it.

| Topic | Single home | Linked from |
|---|---|---|
| What ZDDC is + tool channel links + dual-mode (local/server) overview + install snippets | `website/index.html` (hand-edited intro for `zddc.varasys.io/`) | repo `README.md`, `bootstrap/README.md` |
| File-naming convention spec (status codes, modifiers, folder format) | `website/reference.html` | repo `README.md`, in-tool help text |
| Versions + channel builds index of every tool | `website/releases/index.html` (regenerated by `build.sh`) | website intro nav, "Browse all versions" link |
| Customer-deployment install (copy-paste shell snippets, level-1/2 stubs, `?v=`, audit) | `bootstrap/README.md` | website intro, `zddc/README.md` |
| zddc-server operations: env vars, ACL syntax, `.archive` URLs, container vs binary | `zddc/README.md` | `AGENTS.md`, `bootstrap/README.md`, website intro |
| Build / release / channel commands | `AGENTS.md` | repo `README.md` ("see AGENTS.md") |
| Architecture & internal patterns | `ARCHITECTURE.md` (this file) | `AGENTS.md` |
| Per-tool internal design quirks | `<tool>/README.md` | (linked from website intro tool cards) |

`website/index.html` is **hand-edited static content** (analogous to `reference.html`), not the landing-tool output. The landing tool ships only as a Codeberg release asset (`landing-v<X.Y.Z>` tag → `landing_v<X.Y.Z>.html` asset) — the self-contained install snippet curls the current-stable asset through Caddy at `zddc.varasys.io/releases/<tag>/landing_v<ver>.html` and saves it as `<deployment-root>/index.html` for customer sites where the project picker UI is actually useful (it queries `zddc-server` for the project list). The public website at `zddc.varasys.io/` has nothing to pick, so its root URL is the introduction page.

When updating documentation, prefer linking over duplicating. If you find yourself rewriting the file-naming convention in a tool's README, link to `reference.html` instead.

---

## Build System

### How It Works

Each tool's `build.sh`:

1. Reads CSS files in declaration order, concatenates them
2. Reads JS files in declaration order, concatenates them
3. Processes `template.html` with `awk`, replacing `{{PLACEHOLDER}}` markers with the concatenated content and stripping CDN `<script>`/`<link>` tags
4. Writes the result to `dist/tool.html`
5. If `--release <channel-or-version>` was passed, calls `promote_release` to tag the commit and upload the dist HTML as a Codeberg release asset (via `shared/publish-codeberg-release.sh`).

The top-level `build.sh` at the repository root calls all five tool build scripts in sequence, regenerates `website/bootstrap/` (level-1 stubs and per-channel level-2 stubs), and then queries the Codeberg API once to rewrite `website/releases/index.html` and `manifest.json` so the website's versions index reflects current Codeberg state.

### Channels

Three release channels. Each `--release` invocation tags the commit and uploads the resulting HTML to Codeberg as a release asset; nothing is written under `website/releases/` other than the regenerated `index.html` / `manifest.json`.

- **Stable** — versioned, immutable. `--release [version]` tags `<tool>-v<version>` in git and uploads `<tool>_v<version>.html` to the new Codeberg release. Skips automatically when there is no source change since the last stable tag.
- **Beta** — `--release beta` tags `<tool>-v<next-patch>-beta.N` (auto-incrementing counter) and uploads `<tool>_v<next-patch>-beta.N.html`. The Codeberg release is marked `prerelease: true`. On-page label: `vX.Y.Z-beta · <date> · <sha>`.
- **Alpha** — `--release alpha` tags `<tool>-v<next-patch>-alpha.N` and uploads, analogous to beta.

A plain `sh tool/build.sh` (no `--release`) is a dev build: it produces `dist/<tool>.html` only, with the on-page label `vX.Y.Z-alpha · <full-ts> · <sha>[-dirty]`. No tag, no Codeberg upload.

Stable releases do not automatically advance the alpha/beta channels. Use `./freshen-channel <tool> <channel>` (worktree-based, no manual `git checkout`) to cut a fresh `-alpha.N` / `-beta.N` from the current stable tag — channel discipline rule 4 says do this after every stable release.

The on-page `{{BUILD_LABEL}}` is rendered red+bold for dev/alpha/beta builds (`is_red=1`) and black for stable releases. The label format is:

| Build              | Label                                                  |
|--------------------|--------------------------------------------------------|
| dev (no `--release`) | `v0.0.6-alpha · 2026-04-27 14:00:00 · abc1234[-dirty]` |
| `--release alpha`  | `v0.0.6-alpha · 2026-04-27 · abc1234`                  |
| `--release beta`   | `v0.0.6-beta · 2026-04-27 · abc1234`                   |
| `--release [ver]`  | `v0.0.5`                                               |

`X.Y.Z` for non-stable labels is the **next-stable target** — patch+1 from the latest clean `<tool>-vX.Y.Z` tag. Dev builds use the full timestamp + `-dirty` marker so iterative work is distinguishable from a formal `--release alpha` cut (which stamps date-only and is committed-clean by definition).

### Two-level bootstrap

Customer deployments under `zddc-server` use a two-level bootstrap pattern that keeps tool installation decoupled from publishing. See `bootstrap/README.md` for the full story; in short:

- **Level 1**: per-project stub at `<project>/<tool>.html` that fetches `../<tool>.html` (always same-origin). One file per project per tool, never edited after install.
- **Level 2** (optional): site admin replaces `<deployment-root>/<tool>.html` with a stub that fetches `https://zddc.varasys.io/releases/manifest.json` to resolve `<tool>-<channel>` → tag, then fetches `https://zddc.varasys.io/releases/<tag>/<tool>_v<version>.html` (Caddy reverse-proxies to the Codeberg release-asset URL). Switches the whole site to a channel. Without it, `<deployment-root>/<tool>.html` is just the actual built tool HTML (self-contained install).

`document.write()` chains across both levels; origin stays at the deployment domain throughout. CORS only matters at level 2 (cross-origin to `zddc.varasys.io`); level 1 is same-origin.

The stubs are generated from `bootstrap/level{1,2}.html.tmpl` by the root `build.sh` and published as standalone files under `website/bootstrap/level1/` and `website/bootstrap/track-<channel>/`. The home page's "Install on your server" section prints copy-paste shell snippets that `curl` these into the operator's deployment directory.

### Build Script Requirements

Every `build.sh` must:

- Begin with `#!/bin/sh` and `set -eu` (POSIX sh, not bash)
- Source `shared/build-lib.sh` first (provides `ensure_exists`, `concat_files`, `build_timestamp`, `compute_build_label`, `promote_release`)
- Fail immediately on missing source files (`ensure_exists` pattern)
- Clean up temp files on exit (use `trap cleanup EXIT`)
- Accept `--release [<version>|alpha|beta]` — explicit version or channel name; otherwise produce a dev build

### HTML Embedding Safety

When inlining JavaScript into a `<script>` block, the HTML parser scans for the exact string `</script>` to terminate the block — backslash escaping (`<\/script>`) does **not** prevent termination. Any JS source file or vendor library that contains `</tag>` sequences inside string literals or template literals will break the inline `<script>` block.

The rule is:

> **All `</` sequences in inlined JavaScript must be escaped as `<\/` using `sed`.**

Both the app JS concatenation step and any vendor JS bundling step must run through:

```bash
sed 's#</#<\\/#g' "$input_js" > "$safe_js"
```

Then use `</script>` (not `<\/script>`) to close the `<script>` block, since the content no longer contains any `</` sequences that the parser could misread.

This is already enforced for mdedit's vendor bundling. It is the contributor's responsibility to ensure new tools follow this pattern.

### Vendor Dependencies

Some tools bundle third-party libraries. These live in `tool/vendor/` and are committed to the repository. The build script inlines them into `dist/tool.html`.

**Current vendor files:**

| Tool | Library | File | Notes |
|------|---------|------|-------|
| mdedit | Toast UI Editor v3.2.2 | `vendor/toastui-editor-all.min.js` | Markdown editor with live preview |
| mdedit | Toast UI Editor CSS | `vendor/toastui-editor.min.css` | Editor stylesheet |
| transmittal | jszip, docx-preview, xlsx | CDN at runtime | Optional preview features; tool works without them |

**Runtime CDN loading exception**: The transmittal tool loads jszip, docx-preview, and xlsx from CDN at runtime via `loadLibrary()` forDOCX/XLSX preview functionality. These are **optional enhancements**—core transmittal functionality (JSON payload communication) works without them. This exception is documented here because:

1. The core transmittal features (creating, signing, verifying SHA-256 digests) do not depend on these libraries
2. Preview functionality gracefully degrades if libraries fail to load
3. Bundling would significantly increase file size for rarely-used features

**Rule**: Runtime CDN loading is allowed only when:
- Features are strictly optional (graceful degradation)
- Core functionality works without the external library
- Library is clearly documented as non-essential

`template.html` for tools with vendor deps loads those deps from CDN for convenient local development. The build script replaces CDN tags with the bundled vendor files in the output.

### Development vs Production

| Context | Tailwind / Vendor | How to run |
|---------|-------------------|-----------|
| Development | CDN (live, from `template.html`) | Open `template.html` directly in Chromium |
| Production | Bundled / Static CSS | Run `bash tool/build.sh`, open `dist/tool.html` |

For mdedit specifically: `template.html` loads Toast UI from CDN and uses Tailwind Play CDN. The build replaces Toast UI with the bundled vendor file and replaces the Tailwind CDN script with the static `css/tailwind-utils.css` subset.

---

## JavaScript Architecture

### Vanilla JS Only

All tools use plain JavaScript — no TypeScript, no frameworks, no bundlers. Dependencies are managed manually via vendor files.

### Module Pattern

Each JS file wraps its code in an IIFE or module-scope block and registers its API on `window.app.modules`:

```javascript
// js/mymodule.js
(function() {
    function doSomething() { ... }

    window.app.modules.mymodule = { doSomething };
})();
```

Two top-level globals:

- `window.app` — per-tool app state, modules, and debug surface (every tool)
- `window.zddc` — shared filename/folder/revision parsers from `shared/zddc.js` (every tool)

No other globals. Never expose implementation internals beyond what's needed for testing.

### Module Load Order

JS files are concatenated in the order declared in `build.sh`. Each file can assume earlier files' modules are available on `window.app`. Circular dependencies are not permitted — modules must be layered.

Typical ordering:

```
app.js          ← Declares window.app and top-level state
utils.js        ← Stateless helpers (no dependencies)
store.js        ← State management (depends on app.js)
[domain].js     ← Feature modules (depend on store/utils)
main.js         ← Initialization (depends on all modules)
```

### State Management

Tools manage state in one of two patterns:

**1. Direct state on `window.app`** (archive, classifier, mdedit)

```javascript
window.app = { files: [], selectedFolders: new Set(), modules: {}, ... };
```

State is read directly; mutations trigger explicit re-render calls. Classifier additionally layers a small pub-sub on top via `store.js` (`store.on('files', render)`).

**2. Proxy-based reactive state** (transmittal)

```javascript
const state = createReactiveState({ mode: 'edit', published: false });
state.subscribe((prop, newVal) => { /* auto-update UI */ });
state.mode = 'view'; // Proxy notifies all subscribers automatically
```

Use reactive state when the same property drives multiple independent UI elements. Use direct state when the data flow is simple and unidirectional.

---

## Tool-Specific Architecture

### Archive Browser

**Pattern:** Direct mutation of `window.app.{directories, files, filteredFiles, selectedFiles, ...}`, helper modules namespaced under `window.app.modules.{events, table, urlState, source, ...}`. Supports two source modes (`window.app.sourceMode`): `'local'` (File System Access API) and `'http'` (zddc-server JSON browse).

**Two-level directory structure required:**

```
root-directory/
  transmittal-folder/         ← "grouping folder" — must be a subdirectory
    123456-EL-SPC-0001_A (IFC) - Spec.pdf
    ...
```

Files at the root level are ignored. The grouping folder list and transmittal folder list are populated from the first two levels of the selected directory. Files are only counted in `filteredFiles` after ZDDC filename parsing succeeds.

**Key DOM IDs:** `#addDirectoryBtn`, `#noDirectoryMessage`, `.main-container`, `#filesTableBody`, `#fileCount`, `#selectedCount`, `#selectAllGroupingCheckbox`.

---

### Document Classifier

**Pattern:** Event-driven store (`store.js`) with `notify()` / `on()` pub-sub, spreadsheet rendering on `'files'` events.

**File object shape** (as produced by `scanner.js`):

```javascript
{
    trackingNumber: '123456-EL-SPC-2623',
    title: 'Specification',
    revision: 'A',
    status: 'IFC',
    extension: 'pdf',                 // no leading dot
    originalFilename: '...',          // filename without extension
    name: '...',                      // full filename with extension
    path: 'folder/filename.pdf',
    size: 45000,
    isDirectory: false,
    manualFilename: null              // set if user overrides computed name
}
```

**`computeNewFilename(file)`** (in `utils.js`) returns `file.originalFilename + '.' + file.extension` if any required field is missing.

**Main app panel** (`#mainApp`) stays hidden (class `hidden`) until a real directory is opened via `showDirectoryPicker`. State can be injected via `store.setFolderTree()` + `store.setSelectedFolders()` for testing without triggering the picker.

---

### Markdown Editor (mdedit)

**Pattern:** Global functions (`window.updateToc`), editor instances managed per file-path in a `Map`, File System Access API for direct file read/write.

**Dependencies:** Toast UI Editor v3.2.2 (bundled), Tailwind utility subset (static CSS).

**Toast UI availability check:**

```javascript
if (typeof toastui === 'undefined') {
    // Graceful degradation — show error message
}
const editor = new toastui.Editor({ el: container, ... });
```

**Key DOM IDs:** `#app`, `#select-directory`, `#welcome-screen`, `#file-tree`, `#content-container`.

**File tree:** Populated after `showDirectoryPicker()` resolves. File items are rendered as DOM children of `#file-tree`. Clicking a file opens it in the editor panel.

---

### Transmittal Creator

**Pattern:** Proxy-based reactive state, two-phase hydration, ECDSA digital signatures, SHA-256 file integrity.

**Two-phase hydration:**

1. **`populateStatic()`** — called before publishing. Fills all form fields and the file table into the HTML so the output is readable without JavaScript (progressive enhancement for SharePoint, email clients, etc.).
2. **`hydrate()`** — called on page load of a published transmittal. Hides the "Not Validated" static warning, runs signature verification, and enables interactive features.

**Progressive enhancement matrix:**

| Feature | No JavaScript | With JavaScript |
|---------|--------------|-----------------|
| Content display | ✅ Full | ✅ Full |
| File table | ✅ Shown | ✅ Shown |
| Digest / signatures | ✅ Listed | ✅ Listed + cryptographically verified |
| Validation status | ⚠️ "Not Validated" badge | ✅ "Verified" / ❌ "Invalid" |
| Editing | ❌ Disabled | ✅ Enabled (if draft) |
| Column filtering | ❌ No | ✅ Yes |

**Data store:** A `<script id="transmittal-data" type="application/json">` element embedded in the published HTML holds the full transmittal payload. On load, `data.js` reads and parses it; all UI state derives from this JSON.

**Reactive state:**

```javascript
// app.state is a Proxy — assignments auto-notify subscribers
app.state.mode = 'view';  // Triggers UI updates automatically
```

Subscribe for cross-cutting concerns:
```javascript
app.state.subscribe((property, newValue) => {
    if (property === 'mode') updateModeToggleLabel(newValue);
});
```

**Security model:** ECDSA P-256 signing of the SHA-256 digest. Signatures are stored in the JSON payload. Any number of signers can co-sign. Verification runs client-side in the browser's Web Crypto API — no server required.

**Key module globals:** `window.transmittalApp` exposes `app.data`, `app.state`, and `app.modules` for debugging and testing.

---

## CSS Architecture

All tools use vanilla CSS. No frameworks at build time (mdedit's Tailwind utilities are pre-generated static CSS).

**Common conventions:**

- CSS variables for theme colors and spacing in `base.css`
- Component-scoped class names (no global utilities except where Tailwind provides them)
- `.hidden` class uses `display: none !important` for JavaScript show/hide
- Print styles in a separate `print.css`

**mdedit Tailwind subset:**

`css/tailwind-utils.css` contains only the ~80 Tailwind v3 utility classes actually used in `template.html`. If a new utility class is needed in the template, add it here. Classes follow Tailwind v3 naming and values exactly.

---

## Testing

Tests use Playwright with Chromium only (File System Access API requires it).

### Running Tests

```bash
npm test                    # all tools
npx playwright test archive # single tool
npx playwright test --debug # debug mode
```

### Test Structure

Each tool has a spec file in `tests/`:

```
tests/
  archive.spec.js      ← 2 tests: load + directory scan
  classifier.spec.js   ← 2 tests: load + store injection
  mdedit.spec.js       ← 2 tests: load + file tree render
  transmittal.spec.js  ← 2 tests: paste round-trip + filesystem round-trip
  fixtures/
    mock-fs-api.js     ← Reusable File System Access API mock
    transmittal-data.js
    zddc-filenames.js
```

### Mock File System API

`MOCK_FS_INIT_SCRIPT` (from `tests/fixtures/mock-fs-api.js`) overrides `showDirectoryPicker`, `showOpenFilePicker`, and `showSaveFilePicker`. Inject it via `page.addInitScript` before navigating.

```javascript
// Flat directory
window.__setMockDirectory('name', [{ name: 'file.pdf', content: '...', size: 100 }]);

// Nested directory tree
window.__setMockDirectoryTree('name', {
    'subfolder': { 'file.pdf': 'content' },
    'root-file.md': 'content',
});
```

### Writing Tests

Follow the pattern in `tests/transmittal.spec.js`:

- Use ESM `import` syntax
- Inject `MOCK_FS_INIT_SCRIPT` in `test.beforeEach` for any test that navigates to a tool page
- Use `waitUntil: 'domcontentloaded'` or `'load'` (not `'networkidle'` — the bundled scripts may keep the network active)
- Prefer `page.waitForFunction` over `page.waitForSelector` for app-state readiness
- Assert through the store/module API for tests that don't need visible DOM

---

## Code Standards

| Rule | Rationale |
|------|-----------|
| No `</script>` or any `</tag>` in JS string literals | Breaks inline HTML embedding — escape with `'<' + '/tag>'` or use `<\/` in `sed` at build time |
| No external dependencies at runtime | Self-contained output requirement |
| No TypeScript, no bundlers | Keeps the build system auditable and simple |
| Only `window.app` and `window.zddc` are global | Keeps the global namespace clean; expose only what's needed for debugging |
| Defensive input validation | File System API handles and user-pasted data are untrusted |
| Update README.md when features ship | Documentation parity is a delivery requirement, not optional |

---

## Git Workflow

**Branching:** short-lived feature branches (`feature/<name>`, `bugfix/<name>`, `hotfix/<name>`), squash-merged to `main` and immediately deleted. Quick fixes (typos, one-liners) go direct to `main`.

**Commit messages:** Conventional Commits — `<type>(<scope>): <description>`. Types: `feat`, `fix`, `docs`, `style`, `refactor`, `perf`, `test`, `chore`. See `AGENTS.md` for the full table and examples.

**Releases:** Tag the commit after confirming `dist/` is current. Format: `{project}-v{version}` (e.g. `archive-v1.0.0`). Semantic versioning applies. There is no CI/CD — the built `.html` file is already committed to the repo.

```bash
bash tool/build.sh                  # rebuild dist/
git add -f tool/dist/tool.html      # stage if needed
git commit -m "chore(tool): rebuild for vX.Y.Z"
git tag tool-vX.Y.Z
git push origin main --tags

git tag -l "archive-v*"             # list releases
git push origin :refs/tags/tag-name # delete a remote tag
```

---

## Adding a New Tool

1. Create `tool/` with the standard directory layout
2. Write `template.html` with `{{CSS_PLACEHOLDER}}` and `{{JS_PLACEHOLDER}}` markers
3. Write `tool/build.sh` following the pattern of an existing tool
4. Add `bash "$SCRIPT_DIR/tool/build.sh"` to the root `build.sh`
5. Add a test project entry to `playwright.config.js`
6. Create a stub `tests/tool.spec.js`
7. Force-add the dist output: `git add -f tool/dist/tool.html`

If the tool requires vendor dependencies, download them to `tool/vendor/`, add them to `.gitignore` exclusions if appropriate, and update `build.sh` to inline them (with the `</` escaping step).