feat(browse): "Download (zip)" — pull the current directory's subtree as a zip
A "⤓ Download (zip)" button in the browse toolbar (shown once a
directory is loaded) downloads the directory you're currently
viewing — and everything under it you're allowed to see — as a single
.zip. Navigate into a subfolder first to grab just that subtree.
- Server mode: an <a download> at "<currentPath>?zip=1" — zddc-server
streams the ACL-filtered zip (see the previous commit), nothing held
in the browser.
- Offline (file://) mode: new browse/js/download.js walks the picked
folder with the FS-Access API in two passes — metadata first (so it
can confirm() before loading >~2000 files / ~500 MB into memory),
then bytes — bundles with the already-vendored JSZip, and triggers a
blob download. Hidden entries (".":/"_"-prefixed) are skipped, the
zip's top level is "<folderName>/…" so it unpacks tidily, and the
status bar shows progress.
Wired in browse/js/events.js (button click + show/hide alongside the
refresh button); concatenated into browse/build.sh; ARCHITECTURE.md +
AGENTS.md note the ?zip=1 endpoint and the button.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
parent
81e065e5b0
commit
141fef88fb
7 changed files with 219 additions and 0 deletions
|
|
@ -493,6 +493,8 @@ ZDDC_ROOT=/path/to/your/archive ZDDC_TLS_CERT=none ZDDC_ADDR=:8080 \
|
|||
|
||||
**`.zip` files are navigable directories.** `GET …/Foo.zip/` → JSON listing of the zip's members (or browse HTML); `GET …/Foo.zip/sub/doc.pdf` → that one member, extracted and streamed (Range/ETag via `http.ServeContent`); `GET …/Foo.zip` (no slash) → the raw `.zip` download, unchanged. Write methods to a path inside a `.zip` → 405 (read-only). ACL = the chain of the directory *containing* the zip (a zip has no `.zddc`, like `.archive`). Code: `internal/zipfs` (member listing/extraction with a zip-slip guard) + `handler.ServeZip`, routed by `splitZipPath` in `dispatch` *before* the file-API branch (gated by a cheap `.zip/` substring check so ordinary requests don't pay an `os.Stat`-per-segment walk). Client-side, `shared/zip-source.js` (`ZipDirectoryHandle`/`ZipFileHandle` over JSZip) gives the archive and browse tools the same navigation offline. Archive treats a `.zip` whose name minus `.zip` parses as a transmittal-folder name as that transmittal folder (`isTransmittalFolderZip` in `archive/js/parser.js`); browse expands any `.zip`. Nested zips: the server serves one level (`…/Foo.zip/inner.zip` is the inner zip's bytes; `…/Foo.zip/inner.zip/` isn't a listing) — clients that need deeper nesting fetch the inner zip whole and recurse with JSZip.
|
||||
|
||||
**`GET /dir/?zip=1` — subtree download.** Streams an `application/zip` of every readable file under `/dir/` (recursively), `Content-Disposition: attachment; filename="<dir>.zip"`, `X-ZDDC-Source: subtree-zip`. ACL-filtered per file's containing-dir `.zddc` chain (per-dir decision cache, same as `serveArchiveListing`); skips `.`/`_`-prefixed entries (`.zddc`, `_template`, `_app`); adds a `.zip` *file* it meets as opaque bytes (does not recurse). Streamed, so an empty/fully-denied subtree is a valid empty zip, not a 403. The query check is in `dispatch`'s `info.IsDir()` branch right after the directory ACL gate (so it works on `/dir` and `/dir/`); code: `handler.ServeSubtreeZip`. The browse tool's toolbar "Download (zip)" button uses it in server mode; offline it bundles the picked folder with JSZip (`confirm()` above ~2000 files / ~500 MB).
|
||||
|
||||
### Client mode (proxy / cache / mirror)
|
||||
|
||||
When `--upstream <url>` is set, the binary runs as a **downstream client** of another zddc-server instead of a master. `cmd/zddc-server/main.go` short-circuits to `runClient(cfg)`, which builds a `*cache.Cache` from `zddc/internal/cache/` and uses it as the entire request handler — no archive index, no apps server, no watcher, no OPA decider, no ACL middleware, no token store.
|
||||
|
|
|
|||
|
|
@ -713,6 +713,8 @@ The schema keys that drive built-in behavior:
|
|||
|
||||
**Zip-backed directories.** A `.zip` file is also a navigable directory: `GET …/Foo.zip/` returns a JSON listing of the zip's members (or the browse SPA for an HTML request) and `GET …/Foo.zip/sub/doc.pdf` extracts and streams that one member — so a client navigating a zipped transmittal folder never downloads the whole archive. `GET …/Foo.zip` (no trailing slash) is unchanged: the raw `.zip` download. Read-only: `PUT`/`DELETE`/`POST` to a path inside a `.zip` is rejected (405). ACL is the chain of the directory *containing* the zip — a zip carries no `.zddc` of its own, the same model as the `.archive` virtual surface. Implemented by `internal/zipfs` + `handler.ServeZip`, routed via `splitZipPath` in the dispatcher (before the file-API branch). Offline tools (archive's scanner, browse's tree) get the same capability client-side via `shared/zip-source.js` — a `ZipDirectoryHandle`/`ZipFileHandle` pair over JSZip that mimics the File-System-Access surface. The archive tool treats a `.zip` whose name minus `.zip` parses as a transmittal-folder name as that transmittal folder; the browse tool expands *any* `.zip`.
|
||||
|
||||
**Subtree download.** `GET /some/dir/?zip=1` (the query form works on both `/dir` and `/dir/`) streams an `application/zip` of every readable file under that directory, recursively — `Content-Disposition: attachment; filename="<dir>.zip"`. It's `handler.ServeSubtreeZip`: a `filepath.WalkDir` that ACL-gates each file by the `.zddc` chain of its containing directory (the same per-directory decision cache `serveArchiveListing` uses), skips hidden entries (`.`/`_`-prefixed: `.zddc`, `_template`, `_app`), and adds any `.zip` *file* it meets as opaque bytes (it does **not** recurse into it — that's the navigable-surface above, a different feature). The response is streamed straight onto the `ResponseWriter` (`zip.Store` for already-compressed extensions, `zip.Deflate` otherwise), so a fully-ACL-denied or empty subtree yields a valid empty zip rather than a 403 (a stream can't change status after the headers go out). The browse tool's toolbar **Download (zip)** button hits this for the directory in view in server mode; offline (file://) it walks the picked folder itself with JSZip (with a `confirm()` above ~2000 files / ~500 MB, since the whole tree is buffered in browser memory).
|
||||
|
||||
**WORM** (write-once-read-many). A `worm: [principal...]` list on a `.zddc` marks that path (and descendants) immutable: `w`/`d`/`a` are stripped for everyone non-admin; `c` survives only for the listed principals (who get read + write-once-create); `r` for outsiders is whatever the normal ACL granted (the worm list doesn't itself confer read). Admins (root / subtree) bypass entirely — the escape hatch for mis-filed documents. `defaults.zddc.yaml` puts `worm: [document_controller]` on `archive/<party>/{received,issued}`, so the canonical immutable-archive convention is unchanged; the difference is an operator can mark any path WORM, or rename `received`/`issued`, without a code change.
|
||||
|
||||
**Standard roles.** `defaults.zddc.yaml` references two roles (both shipped empty — a fresh deployment grants nothing until an operator populates them): `document_controller` (read/write across a project, `rwc` at `archive/`, subtree-admin of `working/` and `staging/`, the WORM-create principal in `received/issued`, `rwcd` at `incoming/` for the QC-and-transfer workflow) and `project_team` (read-only across the project; their own `working/<email>/` home and anything they create under `incoming/` get a creator-owned auto-own `.zddc` that wins via deepest-match, so "read-only except what I own" falls out of the cascade with no special rule).
|
||||
|
|
|
|||
|
|
@ -56,6 +56,7 @@ concat_files \
|
|||
"js/preview-markdown.js" \
|
||||
"js/grid.js" \
|
||||
"js/upload.js" \
|
||||
"js/download.js" \
|
||||
"js/events.js" \
|
||||
"js/app.js" \
|
||||
> "$js_raw"
|
||||
|
|
|
|||
141
browse/js/download.js
Normal file
141
browse/js/download.js
Normal file
|
|
@ -0,0 +1,141 @@
|
|||
// download.js — "Download (zip)" for the currently-viewed directory.
|
||||
//
|
||||
// Server mode: just point an <a download> at "<currentPath>?zip=1" —
|
||||
// zddc-server streams an ACL-filtered .zip of the subtree, so nothing
|
||||
// is held in the browser.
|
||||
//
|
||||
// FS-API (offline) mode: there's no server, so we walk the picked
|
||||
// folder ourselves, bundle every file with JSZip, and download the
|
||||
// blob. A two-pass walk (metadata first, then bytes) lets us warn
|
||||
// before loading a very large tree into memory.
|
||||
(function () {
|
||||
'use strict';
|
||||
|
||||
var state = window.app.state;
|
||||
|
||||
// Soft thresholds for the offline bundle: above either, confirm()
|
||||
// before loading everything into memory.
|
||||
var WARN_FILE_COUNT = 2000;
|
||||
var WARN_TOTAL_BYTES = 500 * 1024 * 1024;
|
||||
|
||||
function events() { return window.app.modules.events; }
|
||||
|
||||
function isHiddenName(name) {
|
||||
return name.length === 0 || name[0] === '.' || name[0] === '_';
|
||||
}
|
||||
|
||||
function fmtMB(bytes) { return (bytes / (1024 * 1024)).toFixed(1) + ' MB'; }
|
||||
|
||||
// Trigger a browser download of a Blob (revokes the object URL after).
|
||||
function downloadBlob(filename, blob) {
|
||||
var a = document.createElement('a');
|
||||
a.href = URL.createObjectURL(blob);
|
||||
a.download = filename;
|
||||
document.body.appendChild(a);
|
||||
a.click();
|
||||
setTimeout(function () {
|
||||
URL.revokeObjectURL(a.href);
|
||||
a.remove();
|
||||
}, 0);
|
||||
}
|
||||
|
||||
// Trigger a download from a same-origin server URL via Content-Disposition.
|
||||
function downloadUrl(filename, url) {
|
||||
var a = document.createElement('a');
|
||||
a.href = url;
|
||||
a.download = filename; // hint; the server's Content-Disposition wins
|
||||
document.body.appendChild(a);
|
||||
a.click();
|
||||
setTimeout(function () { a.remove(); }, 0);
|
||||
}
|
||||
|
||||
// Recursively collect every (non-hidden) file under dirHandle into
|
||||
// `out` as { relPath, handle, size }, accumulating into `tally`.
|
||||
// relPrefix is the slash-terminated path within the picked root
|
||||
// ("" at the root).
|
||||
async function collectFiles(dirHandle, relPrefix, out, tally) {
|
||||
for await (var pair of dirHandle.entries()) {
|
||||
var name = pair[0];
|
||||
var handle = pair[1];
|
||||
if (isHiddenName(name)) continue;
|
||||
if (handle.kind === 'directory') {
|
||||
await collectFiles(handle, relPrefix + name + '/', out, tally);
|
||||
} else {
|
||||
var size = 0;
|
||||
try {
|
||||
var f = await handle.getFile();
|
||||
size = f.size || 0;
|
||||
} catch (_e) { /* permission lost — count it as 0 */ }
|
||||
out.push({ relPath: relPrefix + name, handle: handle, size: size });
|
||||
tally.count++;
|
||||
tally.bytes += size;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
async function downloadFsSubtree(rootHandle) {
|
||||
var ev = events();
|
||||
ev.statusInfo('Scanning ' + rootHandle.name + '…');
|
||||
var files = [];
|
||||
var tally = { count: 0, bytes: 0 };
|
||||
await collectFiles(rootHandle, '', files, tally);
|
||||
if (files.length === 0) {
|
||||
ev.statusInfo(rootHandle.name + ' is empty — nothing to download.');
|
||||
return;
|
||||
}
|
||||
if (tally.count > WARN_FILE_COUNT || tally.bytes > WARN_TOTAL_BYTES) {
|
||||
var ok = window.confirm(
|
||||
'This folder has ' + tally.count + ' files (~' + fmtMB(tally.bytes) + ').\n\n'
|
||||
+ 'Building the zip loads them all into memory — it may be slow or crash the tab.\n\n'
|
||||
+ 'Continue?');
|
||||
if (!ok) { ev.statusClear(); return; }
|
||||
}
|
||||
var zip = new window.JSZip();
|
||||
for (var i = 0; i < files.length; i++) {
|
||||
ev.statusInfo('Zipping ' + rootHandle.name + '… (' + (i + 1) + '/' + files.length + ')');
|
||||
var f = await files[i].handle.getFile();
|
||||
var buf = await f.arrayBuffer();
|
||||
zip.file(rootHandle.name + '/' + files[i].relPath, buf);
|
||||
}
|
||||
ev.statusInfo('Generating ' + rootHandle.name + '.zip…');
|
||||
var blob = await zip.generateAsync({ type: 'blob' });
|
||||
downloadBlob(rootHandle.name + '.zip', blob);
|
||||
ev.statusInfo('Downloaded ' + rootHandle.name + '.zip (' + files.length + ' files)');
|
||||
}
|
||||
|
||||
function downloadServerSubtree() {
|
||||
var dir = (state.currentPath || '/').replace(/\/$/, '');
|
||||
var name = (dir.split('/').filter(Boolean).pop()) || 'download';
|
||||
events().statusInfo('Preparing ' + name + '.zip…');
|
||||
downloadUrl(name + '.zip', dir + '/?zip=1');
|
||||
// The browser owns the download from here; clear the hint shortly.
|
||||
setTimeout(function () { events().statusClear(); }, 2500);
|
||||
}
|
||||
|
||||
var busy = false;
|
||||
|
||||
async function downloadCurrentSubtree() {
|
||||
if (busy) return;
|
||||
var btn = document.getElementById('downloadZipBtn');
|
||||
busy = true;
|
||||
if (btn) btn.disabled = true;
|
||||
try {
|
||||
if (state.source === 'server') {
|
||||
downloadServerSubtree();
|
||||
} else if (state.source === 'fs' && state.rootHandle) {
|
||||
await downloadFsSubtree(state.rootHandle);
|
||||
} else {
|
||||
events().statusError('Nothing to download — open a directory first.');
|
||||
}
|
||||
} catch (e) {
|
||||
events().statusError('Download failed: ' + (e && e.message ? e.message : e));
|
||||
} finally {
|
||||
busy = false;
|
||||
if (btn) btn.disabled = false;
|
||||
}
|
||||
}
|
||||
|
||||
window.app.modules.download = {
|
||||
downloadCurrentSubtree: downloadCurrentSubtree
|
||||
};
|
||||
})();
|
||||
|
|
@ -69,6 +69,7 @@
|
|||
function applySourceUI() {
|
||||
var add = document.getElementById('addDirectoryBtn');
|
||||
var refresh = document.getElementById('refreshHeaderBtn');
|
||||
var dlZip = document.getElementById('downloadZipBtn');
|
||||
if (add) {
|
||||
if (state.source === 'server') {
|
||||
add.classList.remove('btn-primary');
|
||||
|
|
@ -85,6 +86,15 @@
|
|||
refresh.classList.add('hidden');
|
||||
}
|
||||
}
|
||||
// "Download (zip)" is meaningful once a directory is loaded
|
||||
// (server or local); it zips the directory currently in view.
|
||||
if (dlZip) {
|
||||
if (state.source) {
|
||||
dlZip.classList.remove('hidden');
|
||||
} else {
|
||||
dlZip.classList.add('hidden');
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
async function refreshListing() {
|
||||
|
|
@ -122,6 +132,12 @@
|
|||
var refresh = document.getElementById('refreshHeaderBtn');
|
||||
if (refresh) refresh.addEventListener('click', refreshListing);
|
||||
|
||||
var dlZip = document.getElementById('downloadZipBtn');
|
||||
if (dlZip) dlZip.addEventListener('click', function () {
|
||||
var d = window.app.modules.download;
|
||||
if (d) d.downloadCurrentSubtree();
|
||||
});
|
||||
|
||||
// Sort dropdown — change → tree re-renders with the new sort.
|
||||
// Format of option value: "<key>:<asc|desc>". Defaults match
|
||||
// state.sort initial values (name:asc).
|
||||
|
|
|
|||
|
|
@ -55,6 +55,9 @@
|
|||
<div class="browse-toolbar">
|
||||
<nav class="breadcrumbs" id="breadcrumbs" aria-label="Path"></nav>
|
||||
<span class="toolbar__count" id="entryCount"></span>
|
||||
<button id="downloadZipBtn" class="btn btn-sm btn-secondary hidden"
|
||||
title="Download this folder (and everything under it you can access) as a .zip"
|
||||
aria-label="Download this folder as a zip">⤓ Download (zip)</button>
|
||||
<label class="sort-control" for="sortBy" title="Sort tree entries">
|
||||
<span class="sort-control__label">Sort:</span>
|
||||
<select id="sortBy" class="sort-control__select" aria-label="Sort tree entries">
|
||||
|
|
@ -132,6 +135,12 @@
|
|||
<dt>ZIP files</dt>
|
||||
<dd>Behave as folders — click to inspect contents inline. JSZip is
|
||||
bundled, so this works offline.</dd>
|
||||
<dt>⤓ Download (zip)</dt>
|
||||
<dd>Downloads the directory you're currently viewing — and everything
|
||||
under it that you're allowed to see — as a single <code>.zip</code>.
|
||||
Navigate into a subfolder first to download just that subtree. Online,
|
||||
the server streams it; locally, the browser bundles the picked folder
|
||||
(a confirmation appears if it's very large).</dd>
|
||||
<dt>Refresh</dt>
|
||||
<dd>Re-fetches the current directory listing — works for both
|
||||
local (re-enumerates the FS handle) and online (re-fetches the JSON).</dd>
|
||||
|
|
|
|||
|
|
@ -1,6 +1,7 @@
|
|||
import { test, expect } from '@playwright/test';
|
||||
import { MOCK_FS_INIT_SCRIPT } from './fixtures/mock-fs-api.js';
|
||||
import * as path from 'path';
|
||||
import * as fs from 'fs/promises';
|
||||
|
||||
const HTML_PATH = path.resolve('browse/dist/browse.html');
|
||||
|
||||
|
|
@ -147,4 +148,51 @@ test.describe('Browse', () => {
|
|||
await expect(page.locator('#previewTitle')).toHaveText(/note\.txt/);
|
||||
await expect(page.locator('#previewBody')).toContainText('a note inside the zip');
|
||||
});
|
||||
|
||||
test('Download (zip) bundles the current folder offline', async ({ page }) => {
|
||||
await page.goto(`file://${HTML_PATH}`, { waitUntil: 'load' });
|
||||
await page.waitForSelector('#addDirectoryBtn', { timeout: 15000 });
|
||||
|
||||
await page.evaluate(() => {
|
||||
window.__setMockDirectoryTree('mock-folder', {
|
||||
'a.txt': 'AAA',
|
||||
'sub': { 'b.txt': 'BBB', 'deep': { 'c.txt': 'CCC' } },
|
||||
'.zddc': 'acl: { permissions: { "*": r } }', // hidden — must not be in the zip
|
||||
'_template': { 'scaffold.txt': 'x' }, // hidden dir — must not be in the zip
|
||||
});
|
||||
});
|
||||
await page.locator('#addDirectoryBtn').click();
|
||||
await page.waitForSelector('#browseRoot:not(.hidden)', { timeout: 10000 });
|
||||
|
||||
// The Download (zip) button appears once a directory is loaded.
|
||||
const dlBtn = page.locator('#downloadZipBtn');
|
||||
await expect(dlBtn).toBeVisible();
|
||||
|
||||
const [download] = await Promise.all([
|
||||
page.waitForEvent('download'),
|
||||
dlBtn.click(),
|
||||
]);
|
||||
expect(download.suggestedFilename()).toBe('mock-folder.zip');
|
||||
|
||||
const file = await download.path();
|
||||
const buf = await fs.readFile(file);
|
||||
// Valid zip: starts with the local-file-header magic, non-trivial.
|
||||
expect(buf.length).toBeGreaterThan(50);
|
||||
expect(buf.subarray(0, 4)).toEqual(Buffer.from([0x50, 0x4b, 0x03, 0x04]));
|
||||
|
||||
// Introspect the entries with the bundled JSZip (in-page).
|
||||
const b64 = buf.toString('base64');
|
||||
const entries = await page.evaluate(async (b64) => {
|
||||
const bin = atob(b64);
|
||||
const bytes = new Uint8Array(bin.length);
|
||||
for (let i = 0; i < bin.length; i++) bytes[i] = bin.charCodeAt(i);
|
||||
const z = await window.JSZip.loadAsync(bytes);
|
||||
return Object.keys(z.files).filter((n) => !z.files[n].dir).sort();
|
||||
}, b64);
|
||||
expect(entries).toEqual([
|
||||
'mock-folder/a.txt',
|
||||
'mock-folder/sub/b.txt',
|
||||
'mock-folder/sub/deep/c.txt',
|
||||
]);
|
||||
});
|
||||
});
|
||||
|
|
|
|||
Loading…
Reference in a new issue