shelf

A small HTTP file shelf: browse and download one directory tree in a browser, behind an authenticating reverse proxy, and hand a single file to someone who has no account via an unguessable expiring link. Single static binary, Go standard library only, no database, no accounts of its own.

Status: M3 + v1.1 — browse, download, share links, upload + mkdir, and on-the-fly format conversion through external filters. The frozen scope lives in SPEC.md; read that before changing behaviour — including §10, the v1.1 amendment.

Why it exists

Serving a directory is easy; serving one without accidentally serving the rest of the filesystem is the whole job. shelf puts every filesystem access behind a single os.Root (Go 1.24), so .., absolute paths and symlinks that leave the root are refused by the kernel-facing layer rather than by string handling of our own.

Install

Each v* tag publishes a static linux/amd64 binary and its checksum as release assets (.forgejo/workflows/release.yml, built in the pinned Go image, CGO_ENABLED=0):

tag=v0.1.0
base=https://git.varasys.io/varasys/shelf/releases/download/$tag
curl -fLO $base/shelf-$tag-linux-amd64
curl -fLO $base/shelf-$tag-linux-amd64.sha256
sha256sum -c shelf-$tag-linux-amd64.sha256
install -m 0755 shelf-$tag-linux-amd64 /usr/local/bin/shelf

Verify the checksum before installing — an automated deployer must (an Ansible get_url without a checksum: silently skips an existing file, so a version bump installs nothing while reporting success).

Build

Requires Go 1.24 or newer (os.Root).

CGO_ENABLED=0 go build -o shelf ./cmd/shelf

Run the checks the way CI does:

gofmt -l .          # must print nothing
go vet ./...
go test ./...

Run

install -m 0600 /dev/null /etc/shelf/share.key
head -c 32 /dev/urandom > /etc/shelf/share.key

./shelf \
  --root /srv/shelf \
  --gui-listen 127.0.0.1:8080 \
  --share-listen 127.0.0.1:8081 \
  --trusted-proxies 10.20.30.10/32,10.20.10.8/32 \
  --hmac-key-file /etc/shelf/share.key \
  --share-url-base https://sync.example.com \
  --share-max-ttl 720h \
  --max-upload 268435456 \
  --filter "md:html=/usr/bin/pandoc -f markdown -t html"

Leave --hmac-key-file and --share-url-base off and shelf browses and downloads with sharing switched off: no control in the listing, and /s/… answers 404 like anything else.

Flags

Every flag has a SHELF_* environment fallback; the flag wins when both are given.

Flag	Env	Default	Meaning
--root	SHELF_ROOT	(required)	Directory to serve. All filesystem access is confined to it.
--gui-listen	SHELF_GUI_LISTEN	127.0.0.1:8080	Listen address of the private GUI plane: host:port, or unix:/absolute/path.sock. Ignored for a plane systemd socket-activates.
--share-listen	SHELF_SHARE_LISTEN	127.0.0.1:8081	Listen address of the public share plane.
--trusted-proxies	SHELF_TRUSTED_PROXIES	(empty)	Comma-separated CIDRs (a bare IP is accepted and means a single host) allowed to reach the GUI plane over TCP. Empty means nobody — the GUI plane rejects every request.
--trusted-peer-uids	SHELF_TRUSTED_PEER_UIDS	(empty)	Comma-separated uids or usernames allowed to reach the GUI plane over a unix socket, where there is no address to check. Empty means nobody. Names are resolved once at startup, so an unknown one is a refusal to start.
--max-upload	SHELF_MAX_UPLOAD	0	Maximum size in bytes of one uploaded file — and the switch for the whole write surface: zero disables POST /api/upload and POST /api/mkdir, and the listing then offers no controls for either.
--hmac-key-file	SHELF_HMAC_KEY_FILE	(empty)	Key file for share tokens. If set, the file must exist, be a regular file, be mode 0600 and be non-empty, or the process refuses to start. Its raw bytes are the key — 32 bytes from /dev/urandom is the recommendation. Empty means no link can be minted or served.
--share-url-base	SHELF_SHARE_URL_BASE	(empty)	Public origin links are built from, e.g. https://sync.example.com. Must be an absolute https URL with a host and no credentials, query or fragment; a path prefix is allowed. Empty disables share creation — shelf cannot guess the name the world reaches it by. Requires --hmac-key-file.
--share-max-ttl	SHELF_SHARE_MAX_TTL	720h (30 days)	Longest lifetime a link may be given. A longer request is clamped to it, and the response says so.
--filter	SHELF_FILTER	(none)	Declare one conversion: "<src-ext>:<dst-format>=<argv...>". Repeatable (the env variable holds one spec per line; any --filter flag replaces the whole env list). Destination formats: html, txt, pdf, json.
--filter-timeout	SHELF_FILTER_TIMEOUT	30s	Hard limit on one conversion. The filter is killed past it and the request answers 502.
--filter-max-output	SHELF_FILTER_MAX_OUTPUT	67108864 (64 MiB)	Largest converted document to hold and serve. Exceeding it fails the request; output is never truncated.

Endpoints

GUI plane (--gui-listen)

• GET / and GET /any/sub/dir — server-rendered HTML listing: name, size, modification time, directories first, link to the parent.
• GET /any/file — download. Content-Disposition: inline by default, attachment with ?dl=1.
• GET /any/file?format=<dst> — the file converted by a declared filter (below).
• POST /api/share — mint a share link (below).
• POST /api/upload?dir=<path>[&overwrite=1] — upload one file (below).
• POST /api/mkdir?dir=<path> with name — create a directory (below).
• GET /healthz — 200 ok.

All four API endpoints are routed on method and path, so a real file called api/upload in the root is still listed and downloadable with GET.

Share plane (--share-listen)

• GET /s/<token>/<filename> — the shared file, Content-Disposition: attachment, Range supported.
• GET /healthz — 200 ok.
• Everything else, and every failure — 404 with an identical body.

Share links

A link is <share-url-base>/s/<token>/<filename>:

https://sync.example.com/s/<payload>.<signature>/nested.txt
                            |         |          └ the real basename: cosmetic, but checked
                            |         └ base64url(HMAC-SHA256(key, payload))
                            └ base64url(path NUL decimal-unix-expiry)

e.g. .../s/c3ViL25lc3RlZC50eHQAMTgwMDAwMDAwMA.<43 chars of MAC>/nested.txt
           ^ that payload decodes to  sub/nested.txt\0 1800000000

The token carries the file path and the expiry, signed with the key; nothing is stored when a link is created, so there is no share database to back up or leak. The trailing file name is cosmetic — the bytes come from the signed path — but it must be the real basename, so a link describes itself and cannot be retitled by whoever passes it on.

Revocation is rotation. Replace --hmac-key-file and restart: every outstanding link dies at once. There is no per-link revocation, because that would be state.

Minting from the listing is a plain form (share column → pick a lifetime → link). From a script:

curl -sS -X POST https://shelf.internal.example/api/share \
  -H 'Content-Type: application/json' \
  -d '{"path":"sub/nested.txt","ttl":"24h"}'
{"url":"https://sync.example.com/s/c3Vi….RUZv…/nested.txt","path":"/sub/nested.txt",
 "expires":"2026-08-15T09:41:07Z","ttl":"24h0m0s","clamped":false}

ttl is a Go duration and is required — shelf does not guess how long a file should be exposed. A value above --share-max-ttl is clamped, and the reply says so ("clamped":true plus a message). A form post gets the same thing as an HTML page; send Accept: application/json or a JSON body to get JSON. The request goes through the GUI plane's two gates like everything else, and the user who minted the link is logged with the path and expiry — that log line is the audit trail. The token is never logged.

Uploads and directories (M3)

shelf may create. It may not destroy: there is no delete and no rename, ever, in v1 (SPEC.md §7). Both mutations are off until --max-upload is set to a positive size.

curl -sS -X POST -H 'Accept: application/json' \
  -F file=@report.pdf \
  'https://shelf.internal.example/api/upload?dir=/reports'
{"path":"/reports/report.pdf","name":"report.pdf","size":18422,"replaced":false}

curl -sS -X POST -H 'Content-Type: application/json' \
  -d '{"name":"2026"}' 'https://shelf.internal.example/api/mkdir?dir=/reports'
{"path":"/reports/2026","name":"2026"}

• The file streams into a temporary .shelf-upload-<random>.partial in the destination directory, and is renamed into place only after a complete read, an fsync and a close. So a reader never sees a partial file under the real name, a failed upload leaves nothing behind, and two writers racing for one name leave one whole file rather than a blend. (Rename is atomic only within a filesystem, which is why the temporary is not in /tmp.)
• An existing name is refused with 409; repeat with ?overwrite=1 to replace it.
• Over --max-upload is refused with 413, and nothing is written.
• The name must be a plain name — no / or \, no .., no control characters, at most 255 bytes — for both endpoints. A dotfile is a fine name; . and .. are not names. shelf refuses a path rather than quietly shortening it to its last element, so it never stores a file under a name the client did not send.
• ?dir= goes through the same containment as a browsed path, so a traversal in it is refused exactly like one in a URL.
• A browser form gets 303 back to the listing; send Accept: application/json (or a JSON body, for mkdir) for the object above.

Format conversion (v1.1)

A file can be served through an external command you declared, chosen by its extension and the requested format:

--filter "md:html=/usr/bin/pandoc -f markdown -t html"
--filter "md:pdf=/usr/local/bin/md2pdf"
--filter "csv:json=/opt/tools/csv2json --strict"

Then GET /notes/mydoc.md?format=html runs pandoc with the file on stdin and serves its stdout as text/html, named mydoc.html.

• GUI plane only. ?format= is honoured behind the auth gates and nowhere else: the share plane never converts — it ignores the parameter and serves the file's bytes. Converters are large programs with long CVE histories, and no public link may reach an exec. This is the rule the whole feature is built around (SPEC.md §10).
• An undeclared mapping is a 404, the same answer an unknown path gets. shelf does not publish which conversions it was configured with.
• The plugin API is stdin → stdout. Nothing is loaded into the process, nothing is discovered, and there is no plugin interface: adding or removing a converter is a flag and a restart. No shell is involved — the argv is split on whitespace and executed directly, so there is no quoting, globbing or $(...). If you need any of that, write a two-line wrapper script and declare that. argv[0] must be an absolute path.
• A conversion that fails is a 502 with the reason (and the filter's first line of stderr): a non-zero exit, a timeout, a missing binary, or output over the cap. The output is captured in full before a byte is sent, so a converter that crashes half-way gives you a clean error rather than a truncated document that looks complete.
• The Content-Type comes from the destination format, through a fixed map (html, txt, pdf, json), never from what the child produced. Converted html is also served with Content-Security-Policy: sandbox.
• The child is caged: --filter-timeout, --filter-max-output, a fixed minimal environment, and a working directory in the system temp dir.
• A missing binary is discovered at request time, not at startup. There is no --filter-check on purpose: a binary can be installed, removed or replaced while shelf runs, so a startup check would prove something about the past and invite trusting it.
• Not included, deliberately: no cache of converted output, no chained conversions, no autodetection (the source is the file's extension, literally).

Deployment model

browser ──▶ Caddy (TLS) ──▶ Authelia forward-auth ──▶ shelf GUI plane :8080
                                                       (Remote-User: alice)

internet ──▶ Caddy ──(allowlisted /s/ path only)──▶ shelf share plane :8081

• The GUI plane is private. It is reached only through the reverse proxy, which authenticates the user (Authelia forward-auth) and sets Remote-User. shelf believes that header only when the TCP peer address is inside --trusted-proxies; the peer address always comes from the connection, never from X-Forwarded-For. Bind the GUI plane to an interface only the proxy can reach, and list the proxy's address.
• The share plane is public, allowlisted at the origin to the /s/ path only. It has no identity gate — the signed token is the authorization. --share-url-base must be the origin that reaches this plane, not the GUI one.
• Health probes. /healthz on the GUI plane is deliberately kept inside the trusted-proxy check: it is exempt from the Remote-User requirement (a probe has no user) but not from the address check. So the probe host's address must be in --trusted-proxies — e.g. the control node that runs the check — otherwise the probe gets a 403 and looks like an outage. The share plane's /healthz is ungated.

shelf writes structured logs to stderr (log/slog, text handler) and exits cleanly on SIGINT/SIGTERM, so it runs as a plain systemd unit with no wrapper.

Layout

cmd/shelf/         wiring: flags -> os.Root -> two listeners -> signal handling
internal/config/   flag + env parsing and validation
internal/httpx/    peer-address resolution and the trusted-proxy gate
internal/fsx/      containment: the ..-refusal and open-then-stat, shared by both planes
internal/token/    share tokens: canonical payload + HMAC-SHA256, no filesystem knowledge
internal/gui/      the private plane: listing, download, link minting, /healthz
internal/share/    the public plane: one file by signed token

Notes

• The module path is varasys.io/shelf. It is a binary, not a library, so the path is a stable name rather than a fetchable location; point it at the repository host if the module is ever meant to be go get-able.
• Entries in a listing are what the directory contains, so a symlink that points outside the root is listed — following it returns 404. Containment is enforced at open time, not by hiding names.
• A share link is resolved through the same os.Root and the same ..-refusal as a browsed path. The signature proves shelf minted the payload; it does not make the payload safe, and a key that ever leaks must not become a traversal primitive.
• POST /api/share is routed on method and path, so a real file called api/share in the root is still listed and downloadable with GET. The same holds for /api/upload and /api/mkdir.
• The commit step of an upload is a renameat(2) against the destination directory's own descriptor, because os.Root gained Rename only in Go 1.25 and this program is pinned to 1.24. On a non-Linux build shelf refuses to write rather than falling back to a path-based rename that would give up the containment property (SPEC.md §7.1).

Socket activation and multi-tenant deployment (v1.2)

shelf serves one root. Two people who must not see each other's files therefore get two shelves — separate processes, separate --root, separate uids — rather than one shelf that knows about users. The separation is then the kernel's file permissions, not a routing decision inside this program, and a bug here cannot cross it: the process serving one tenant cannot read the other's files at all.

To address N processes individually without giving each one a port, shelf accepts systemd socket activation:

# shelf-gui@.socket
[Socket]
ListenStream=/run/shelf/%i/gui.sock
FileDescriptorName=gui          # REQUIRED: descriptors are adopted by name, never by position
SocketUser=root
SocketGroup=caddy
SocketMode=0660

# shelf@.service
[Service]
User=shelf-%i
ExecStart=/usr/local/bin/shelf --root /srv/tenants/%i --trusted-peer-uids caddy
Sockets=shelf-gui@%i.socket

and the reverse proxy picks the socket from the identity it authenticated:

map {http.request.header.Remote-User} {tenant} {
    casey    casey
    richard  richard
    default  ""              # unknown user routes nowhere
}
reverse_proxy unix//run/shelf/{tenant}/gui.sock

Notes that matter:

• FileDescriptorName= is required. A descriptor named anything else — including the socket unit name systemd falls back to — is a startup error, not an ignored socket. An unadopted socket that systemd still reports as active looks exactly like a working deployment.
• The uid is the gate, not the socket's mode. Over a unix socket shelf checks the peer's uid from SO_PEERCRED against --trusted-peer-uids; --trusted-proxies does not apply and is reported as ignored. Set SocketMode=0660 anyway — defence in depth — but the program does not rely on it.
• Both planes may be activated (FileDescriptorName=share for the other), or either one may fall back to its flag. A plane that systemd did not pass binds its flag as usual.
• None of this is required. With no LISTEN_FDS in the environment shelf binds --gui-listen/--share-listen exactly as before, which is how a self-hoster runs one instance from a command line.