metronome/docs/rust-port.md

Rust port — staged plan

This is the plan for the native-Rust direction first discussed alongside the A/B-bootloader idea. What changed since then: the track format is now formally specced (docs/track-format.md) with a golden-vector conformance suite (tests/run.mjs). That suite is the thing that makes a port safe — any Rust engine has a precise, executable definition of "correct" to validate against, the same one engine.js and app.py already pass.

The core idea

Port from the inside out, lowest-risk first. The pure logic (track codec, then the scheduler) is host-testable with zero hardware and is gated by the existing golden vectors. Only once that's proven do we touch drivers, A/B, and the actual firmware. We do not rip out CircuitPython until the Rust engine passes the vectors and the drivers are proven on hardware.

Architecture: one firmware core, modular drivers per form factor

Trying many form factors (Kit, Explorer, Grid/Scroll Pack, …) is how we discover the line between core and driver. In Rust that line is enforced by the type system instead of copied by hand — today each CircuitPython form factor is its own ~1,500-line app.py clone; the Rust build is one core crate plus a thin per-board binary.

pm-core — the core (no_std, zero hardware):

• the track-format codec (rust/track-format, Stage 1) and the scheduler/clock (Stage 2, already no_std and building for RP2350),
• playback-flow (rep/end/continue, segment seams), app state, set-list model,
• the USB-MIDI / live-sync / firmware-update protocol logic (the SysEx opcode handling, which is form-factor-independent). It is host-testable and gated by the golden vectors — the same suite engine.js and app.py pass. This is "core."

Driver traits — what the core is generic over (the swappable part): define small project traits — Display (or render straight to an embedded-graphics DrawTarget), Inputs (yields button / touch events), Clicker (audio out), Indicator (RGB) — and write each concrete driver against embedded-hal bus traits (I2c, SpiBus, OutputPin, DelayNs). The core's UI code then doesn't care whether the target is a 17×7 mono matrix or a 320×480 colour TFT.

Per-board binary crates — pm-kit, pm-explorer, pm-grid: a thin main.rs BSP that instantiates the right concrete drivers and hands them to the generic core:

• Grid (Scroll Pack): IS31FL3731 over I²C (a DrawTarget for a 17×7 mono frame) + 4 GPIO buttons.
• Kit: ST7796 320×480 over SPI — driven by a custom St7796 struct (direct port of pico/main.py), not mipidsi, which fought the panel's geometry/CS (see rust-st7796-cs-gotcha); UI still renders through an embedded-graphics framebuffer. GT911 touch over I²C; WS2812 via ws2812-pio; I²S to the PCM5102A via PIO.
• Explorer: ST7789V 320×240 over an 8-bit parallel (8080) bus via mipidsi's ParallelInterface — a different driver story from the Kit's SPI (see the matrix below).

Display driver matrix (researched 2026-06-03)

Displays are not the gate — every controller has a real Rust path; the buses differ:

Form factor	Controller	Bus	Rust driver	Status
Kit (pm-kit)	ST7796 320×480	SPI	custom St7796 (port of pico/main.py) + embedded-graphics framebuffer; mipidsi dropped	✅ on hardware — but tearing (see below)
Explorer (pm-explorer)	ST7789V 320×240	8-bit parallel 8080	mipidsi ParallelInterface (start here; be ready to port directly if geometry fights, as on the Kit)	path proven upstream; not yet built
Grid (pm-grid)	IS31FL3731 17×7 mono	I²C	vendored bulk-framebuffer driver (port of pico-scroll/app.py's Matrix); is31fl3731 crate not used	✅ built + compiles (LED-first milestone) — see below
Kit touch	GT911	I²C	gt911 or gt9x crate	✅ mature (blocking + async, 5-point)

ST7796 (Kit) — only partially working: tearing. Pixels are correct and the panel boots, but the image tears badly. The cause is structural, not a bug: mipidsi has no TE-pin / vsync / partial-update / double-buffer support (confirmed against the upstream repo github.com/almindor/mipidsi — it offers only optional draw batching), and this ST7796 module doesn't break out the TE (tearing-effect) line to sync writes against scan-out. So writes race the panel's refresh. Mitigations available to us, none from the crate: (a) redraw only changed full-width row-bands to shrink the tear window — already done; (b) DMA each band as one tight burst; (c) sync to a TE GPIO only if a module that exposes that pin is sourced. Treat tearing as an open hardware/firmware item, not "display done." See rust-st7796-cs-gotcha.

Explorer parallel bus — correctness and performance are decoupled. mipidsi's ParallelInterface drives the data pins through an OutputBus trait. The shipped Generic8BitBus is plain GPIO bit-bang (embedded-hal OutputPins) — works immediately, just CPU-bound. For speed, implement OutputBus over RP2350 PIO (the PIO supports 8080/6800 bus timing; the C/TFT_eSPI world hits ~4 ms for a full 320×480 clear this way) — a drop-in swap that leaves mipidsi + embedded-graphics + pm-ui untouched. Worst case is "slow but functional," never "impossible," so the bit-bang fallback de-risks the whole Explorer bring-up.

The honest caveat (what the Grid prototype is teaching us): a 17×7 mono grid and a 320×480 touch TFT are too different for one pixel-identical UI. So the clean split is core engine + protocol + state = fully shared; the view = per-display-class. The Grid is the most extreme, minimal display in the lineup, which makes it the best forcing-function for finding exactly where that boundary falls before we commit drivers to Rust. The CircuitPython pico-scroll/ build exists to nail that UI down on real hardware first.

Stages

Stage 0 — toolchain in a container

Add a Rust toolchain image (mirroring hardware/eda/): a Containerfile with rustup, the thumbv8m.main-none-eabihf target (RP2350 is Cortex-M33), flip-link, probe-rs, elf2uf2. Driven by a run.sh like the EDA one. Never on the host.

Stage 1 — track-format crate ✅ DONE (rust/track-format/)

Implemented and passing: ./rust/run.sh builds the container and runs cargo test, which validates the crate against tests/fixtures/track-format.json (conformance + idempotency). The Rust codec agrees with engine.js and app.py on every vector — and carries vol/cd, so it's the most spec-complete of the three. Original scope below.

(original) Stage 1 — track-format crate ← the concrete first PR

A pure, no_std-compatible crate: parse(&str) -> Track and serialize(&Track) -> String, plus a normalize() that emits the neutral structure from docs/track-format.md §5. Then a cargo test that reads tests/fixtures/track-format.json and asserts each case's norm and round-trip — i.e. a third adapter alongside js_adapter.mjs / py_adapter.py. When this is green, the Rust engine provably agrees with web + device on every groove, euclid, swing, ghost, polymeter, and the playback-flow tokens. No hardware, fully testable in the container.

This is the highest-value slice: small, gated by work already done, and it proves the toolchain.

Stage 2 — scheduler/engine ✅ DONE (rust/track-format/src/schedule.rs)

Ported the look-ahead step scheduler (the durs math from app.py tick/_prepare_next). render(track, bars) produces the deterministic click timeline; tests/schedule.rs asserts the timings — quarter-note spacing, subdivisions, swing 2/3:1/3, polymeter 5:4, accents/ghosts, mute, multi-bar looping. All green on the host, no hardware. The real-time firmware loop will just play this timeline against the wall clock.

Also done: the crate is now #![no_std] + alloc and builds for the RP2350 target (cargo build --lib --target thumbv8m.main-none-eabihf) — the codec + scheduler are firmware-ready.

Stage 3 — drivers (hardware) 🔧 IN PROGRESS (rust/pm-kit/)

✅ Milestone 1 (boot) — confirmed on Pico 2: GP25 blink. Toolchain + RP2350 boot block + flash work. 🟡 Milestone 2 (display) — draws correctly on Pico 2, but TEARS: ST7796 320×480 over SPI0 via rp235x-hal, drawing the shared pm-ui through an embedded-graphics framebuffer. Driver is a custom St7796 struct ported from pico/main.py — mipidsi was tried first and dropped: its split-transaction CS and orientation/offset math mangled the geometry; the port uses correct per-command CS framing (CS low → cmd → params → CS high) and full-width row bands (see rust-st7796-cs-gotcha). Not done: the image tears badly — no TE/vsync sync is possible because this module exposes no TE pin, so writes race scan-out (no crate fixes this; see the tearing note in the display driver matrix above). Open item before the display can be called finished. Diagnosed off-bench with host tools in rust/uisim: uisim renders pm-ui to PNG; --bin panelsim decodes mipidsi's real command/pixel stream into a PNG (proved the protocol correct → bug was physical); --bin initdump dumps the init + CASET/RASET sequence.

🟡 Milestone 3 (live metronome) — built, pending on-device check: the firmware is now an actual metronome. embedded-alloc heap → parses tracks with track-format on-device; 4 built-in grooves; Timer-driven clock; audio clicks on the master lane's hits (GP13 PWM, short edge-triggered pulses, accent louder); controls — A = play/stop, B = grid/notation view, joystick (rotated 90° CCW) up/down = tempo, left/right = groove. Renders pm-ui::draw_metronome / draw_notation, with a cheap draw_progress strip animating the bar position every frame (full redraw only on change → no flicker). All loop input reads use unwrap_or (no panics) — addresses the self-test crash. Compile + simulator verified; needs a flash to confirm audio timing, joystick directions, no crash.

pm-ui views (sim-verified, PNGs): metronome grid (accents/ghosts/polymeter), and drum notation (5-line staff, time sig, hands stem-up / feet stem-down, shared stems, beamed eighths, ledger lines).

Still to do: GT911 touch (GP8/9), WS2812 RGB (GP12), USB-MIDI, set-lists from programs.json, per-cell live playhead, the rest of the practice features. Then split pm-core out as its own crate and add pm-explorer/pm-grid binaries. HAL stays rp235x-hal (embassy later if async earns it).

On embassy / rp-hal:

• ST7789 240×320 display → mipidsi + embedded-graphics (mature; the parts are well-supported).
• I²S to the PCM5102A → RP2350 PIO.
• WS2812 → ws2812-pio. USB-MIDI → usbd-midi / embassy-usb.
• GT911 touch (Kit) over I²C.

pm-grid — Scroll Pack firmware 🟢 BUILT (LED-first milestone), pending on-device check

The Rust sibling of pico-scroll/app.py (rust/pm-grid/). Target is a plain RP2040 (Cortex-M0+, thumbv6m-none-eabi) — NOT the Pico 2 — so it has its own HAL (rp2040-hal 0.10 + rp2040-boot2), .cargo/config.toml, memory.x (BOOT2 + flash + 264 KB RAM) and build.sh+uf2.py (RP2040 family id 0xe48bff56). thumbv6m-none-eabi added to rust/Containerfile. Compiles clean → 48 KB pm-grid.uf2 (BOOTSEL drag-flash; no probe needed). Kept out of the host workspace like pm-kit.

What's implemented (faithful port of pico-scroll): the IS31FL3731 driver (vendored bulk 144-byte framebuffer, one I²C block write per frame — the right architecture, mirrors the CircuitPython Matrix; per-pixel I²C is too slow to animate), the polymeter scheduler driven by track-format::schedule::lane_durs (the cross-impl contract) with per-lane step clocks + ramp + gap-trainer, 4-button input (A tap=play/stop, hold=cycle view; B tap=next track, hold=next set list; X/Y=tempo ∓ with auto-repeat), the built-in set lists, and three LED views:

• Ticker (default): track name infinite-scrolls across the left (cols 0–10, full height); BPM is pinned right, rotated 90° CCW — a vertical hundreds dot-bar in col 11 (one dot per 100) + the last two digits rotated into cols 12–16 (tens bottom, units top). So 130 → 1 dot + rotated "30". This is the user-designed landscape readout. Rotation/geometry verified off-bench with an ASCII replica of draw_ticker.
• Grid (lanes×steps + playhead) and Pendulum (bouncing arm + beat ticks) — ports of _render_grid / _render_pendulum. Boot splash scrolls "PM-G1 GRID" (liveness + pixel-map check).

Deferred to the next milestone (matches pm-kit's order — none built yet): USB-MIDI note-out (the Scroll Pack has NO speaker, so this is the real audio path) + MIDI clock, live-sync SysEx (0x40-0x43 + version query), firmware push (0x10/0x21-0x23), on-device practice log, settings.json, and playback-flow auto-advance (rep/end/continue). An optional piezo on a free GPIO is also TODO.

Stage 4 — native A/B + secure boot

Replace the .mpy-level A/B hack (code.py loads app.mpy, rolls back to app.bak) with the RP2350 bootrom's native partition-table A/B + signed boot, configured via picotool (the chip already provides this — see the earlier hardware discussion). The Rust app is the image in the slot; rollback and version selection move into silicon.

What you keep / lose

• Gain: memory safety, native A/B + secure boot, performance headroom, one typed model instead of three hand-written parsers.
• Lose: the live one-click .mpy push (Rust is compile→flash→reboot). The editor's data live-sync (tempo/pattern/setlist mirroring) still works — it's a data protocol. Only live logic edits go away, and an embedded wasm3/script module could buy those back if wanted.

Acceptance gate

Every codec/engine change must pass tests/fixtures/track-format.json. The Rust crate joins js/py as a runner adapter, so "same groove on web, device, and the Rust build" is enforced, not hoped for.

Recommendation

Do Stage 1 in a container next — it's small, testable today (given a toolchain), reuses the suite, and produces a real artifact to judge the Rust path on before committing to drivers or a firmware rewrite. Defer Stages 3–4 until Stage 1–2 are green and you've decided the live-push tradeoff is acceptable.