PM_K-1 0.0.10: ship precompiled app.mpy (fixes boot OOM) + push .mpy over the air

The single-file app grew to ~57KB; CircuitPython compiling it at boot fragments the
RP2040 heap so badly that the fonts can't get a contiguous block (161KB free, yet a
~16KB alloc fails). Fix: precompile to app.mpy (Adafruit mpy-cross for CP 10.2.1, emits
CircuitPython mpy v6) so the device loads bytecode without compiling -> no fragmentation.

- build.sh precompiles pico-cp/app.py -> dist/app.mpy via tools/mpy-cross (gitignored
  binary); the bundle ships app.mpy (NOT app.py); serves pico-cp-app.mpy + pico-cp-app.py
  (the .py only for the editor's version regex + as readable reference).
- Loader (code.py) imports app.mpy and rolls back app.bak as .mpy.
- One-click updater now pushes the .mpy: editor base64-encodes it and sends it over the
  existing flow-controlled chunked transport (512-char = mult-of-4 chunks); the device
  base64-decodes each chunk to /app.new and verifies the CircuitPython .mpy header
  (magic 'C', v6, >=4KB) before the A/B install. Version still read from the served .py.

Verified: mpy-cross emits magic 'C'/v6; build produces a 21.8KB app.mpy; editing-logic
harness + scene render still pass; and a simulated push (base64 -> 57 chunks -> a2b_base64)
reassembles the .mpy byte-exact and passes the device's header check.

One-time recovery: delete app.py from the drive, copy app.mpy + code.py from the new zip.
After that, updates are one-click again (and can't brick: header check + A/B rollback).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Me Here 2026-05-29 14:01:57 -05:00
parent 7dd567fb44
commit 7481f91935
8 changed files with 94 additions and 71 deletions

1
.gitignore vendored
View file

@ -1,2 +1,3 @@
# Build output — assembled from index.html + assets/ by build.sh
dist/
tools/

View file

@ -11,6 +11,15 @@
set -euo pipefail
cd "$(dirname "${BASH_SOURCE[0]}")"
mkdir -p dist
# Precompile the PM_K-1 CircuitPython firmware to .mpy. CircuitPython compiles a ~56KB .py at boot,
# which fragments the heap and OOMs on the RP2040; a precompiled .mpy loads without compiling. Needs
# Adafruit's mpy-cross matching the device's CircuitPython (10.2.1) -> emits CircuitPython mpy v6.
MPYC="tools/mpy-cross"
[[ -x "$MPYC" ]] || { echo "error: $MPYC missing (Adafruit mpy-cross for CircuitPython 10.2.1)" >&2; exit 1; }
"$MPYC" pico-cp/app.py -o dist/app.mpy
echo "precompiled dist/app.mpy ($(stat -c%s dist/app.mpy) bytes <- $(stat -c%s pico-cp/app.py) source)"
python3 - <<'PY'
import os, pathlib, re
A = pathlib.Path("assets")
@ -41,14 +50,17 @@ _appsrc = pathlib.Path("pico-cp/app.py").read_text()
# The A/B updater pushes app.py over USB-MIDI as 7-bit data, so it MUST be pure ASCII -- a stray
# non-ASCII char (e.g. an em-dash in a comment) gets mangled to a NUL byte and bricks the device.
_bad = [(i, c) for i, c in enumerate(_appsrc) if ord(c) > 0x7F]
assert not _bad, "pico-cp/app.py has non-ASCII at %r -- the MIDI updater needs pure ASCII" % (_bad[:5],)
pathlib.Path("dist/pico-cp-app.py").write_text(_appsrc) # served for the editor's A/B firmware updater
print("copied pico-cp-app.py (ascii-checked)")
assert not _bad, "pico-cp/app.py has non-ASCII at %r -- keep it ASCII (version regex + clean source)" % (_bad[:5],)
pathlib.Path("dist/pico-cp-app.py").write_text(_appsrc) # served for version reading + as readable reference
# the editor pushes the PRECOMPILED .mpy (base64); serve it next to the source
pathlib.Path("dist/pico-cp-app.mpy").write_bytes(pathlib.Path("dist/app.mpy").read_bytes())
print("copied pico-cp-app.py + pico-cp-app.mpy")
import zipfile # PM_K-1 CircuitPython drive bundle (download → unzip onto CIRCUITPY)
with zipfile.ZipFile("dist/pm_k1_circuitpy.zip", "w", zipfile.ZIP_DEFLATED) as z:
for f in ("code.py", "app.py", "boot.py", "programs.json", "font_s.bin", "font_m.bin", "font_l.bin",
for f in ("code.py", "boot.py", "programs.json", "font_s.bin", "font_m.bin", "font_l.bin",
"logo.bin", "midi.bin", "usb.bin", "README.md", "protect-firmware.sh"):
z.write("pico-cp/" + f, f)
z.write("dist/app.mpy", "app.mpy") # the precompiled firmware (NOT app.py - too big to compile on-device)
z.write("dist/editor.html", "editor.html") # offline copy of the editor, on the drive
print("zipped pm_k1_circuitpy.zip")
PY

View file

@ -49,7 +49,8 @@ done
cp "$DIST_DIR/embed.js" "$DEST_DIR/embed.js"; echo " embed.js ($(stat -c '%s' "$DEST_DIR/embed.js") bytes)"
cp "$DIST_DIR/pico-main.py" "$DEST_DIR/pico-main.py"; echo " pico-main.py ($(stat -c '%s' "$DEST_DIR/pico-main.py") bytes)" # PM_K-1 firmware download
cp "$DIST_DIR/pm_k1_circuitpy.zip" "$DEST_DIR/pm_k1_circuitpy.zip"; echo " pm_k1_circuitpy.zip ($(stat -c '%s' "$DEST_DIR/pm_k1_circuitpy.zip") bytes)" # PM_K-1 CircuitPython bundle
cp "$DIST_DIR/pico-cp-app.py" "$DEST_DIR/pico-cp-app.py"; echo " pico-cp-app.py ($(stat -c '%s' "$DEST_DIR/pico-cp-app.py") bytes)" # PM_K-1 firmware for the editor's A/B updater
cp "$DIST_DIR/pico-cp-app.py" "$DEST_DIR/pico-cp-app.py"; echo " pico-cp-app.py ($(stat -c '%s' "$DEST_DIR/pico-cp-app.py") bytes)" # served for version reading + reference
cp "$DIST_DIR/pico-cp-app.mpy" "$DEST_DIR/pico-cp-app.mpy"; echo " pico-cp-app.mpy ($(stat -c '%s' "$DEST_DIR/pico-cp-app.mpy") bytes)" # precompiled firmware the editor pushes (base64)
rm -f "$DEST_DIR/player-asbuilt.html" # renamed to teacher.html
rm -f "$DEST_DIR/concepts.html" # Concepts is now the landing (/)
# info-*.html are first-class pages again: each form factor has a lean widget page

View file

@ -1200,63 +1200,66 @@ async function toggleDeviceAudio() {
function _queryDeviceVersion() { // ask the device its firmware version (SysEx 0x02 -> reply 0x03)
return new Promise((res) => { _verCb = res; _send([0xF0, 0x7D, 0x02, 0xF7]); setTimeout(() => { if (_verCb) { _verCb = null; res(null); } }, 1500); });
}
async function updateFirmware() { // A/B firmware update over USB-MIDI, with a version check
async function updateFirmware() { // A/B firmware update over USB-MIDI: push the precompiled .mpy
if (!(await _ensureMidi()) || !_midiOutputs().length)
return alert("Connect the PM_K-1 (Chrome/Edge/Firefox), then try again.");
const dev = await _queryDeviceVersion();
const src = await _firmwareSource();
if (src == null) return; // offline + user cancelled the picker
const m = src.match(/APP_VERSION\s*=\s*["']([^"']+)["']/); const latest = m && m[1];
if (!latest) return alert("That file isn't PM_K-1 firmware (no APP_VERSION line).");
let latest = null, b64 = null;
// version comes from the (text) source; the payload is the precompiled .mpy bytecode — CircuitPython
// compiles a big .py at boot, which OOMs the RP2040, so we ship + push compiled bytecode instead.
for (const base of ["", "https://metronome.varasys.io"]) {
try { const t = await (await fetch(base + "/pico-cp-app.py", { cache: "no-store" })).text();
const m = t.match(/APP_VERSION\s*=\s*["']([^"']+)["']/); if (m) latest = m[1]; } catch (_) {}
try { const r = await fetch(base + "/pico-cp-app.mpy", { cache: "no-store" });
if (r.ok) { b64 = _b64(new Uint8Array(await r.arrayBuffer())); break; } } catch (_) {}
}
if (!b64) { // offline: let the user pick app.mpy
alert("Can't reach the site.\n\nPick the firmware file (app.mpy) to flash — download it from\n" +
"metronome.varasys.io/pico-cp-app.mpy, or use the online editor at metronome.varasys.io/editor.html.");
const u8 = await _pickBinary(); if (!u8) return;
b64 = _b64(u8); if (!latest) latest = "(picked .mpy)";
}
if (!latest) latest = "?";
const upToDate = dev && dev === latest;
if (!confirm("Device firmware: " + (dev || "unknown") + "\nNew build: " + latest +
(upToDate ? "\n\nSame version. Re-install anyway?"
: "\n\nUpdate now? The device reboots, runs the new build, and auto-rolls-back if it fails to start."))) return;
const err = await _pushFirmware(src);
const err = await _pushFirmware(b64);
if (err) return alert("Update didn't complete (" + err + ").\n\nThe device kept its working firmware — nothing was installed. " +
"Make sure it's plugged in and NOT in editor mode (don't hold A), on firmware 0.0.6+, then retry. " +
"(If the device is older than 0.0.6, drag app.py onto the drive once in editor mode to get the chunked updater.)");
"Make sure it's plugged in and NOT in editor mode (don't hold A), then retry.");
alert("Update sent ✓ — the device verified v" + latest + " and is rebooting into it. It auto-confirms after a few seconds, or rolls back if it won't start.");
}
// One ACK/NAK await (the device replies 0x7F=ok / 0x7E=rejected after each SysEx); resolves true/false/null(timeout).
function _ack(timeout) {
return new Promise((res) => { _saveCb = res; setTimeout(() => { if (_saveCb) { _saveCb = null; res(null); } }, timeout); });
}
// Push app.py in small, flow-controlled chunks: a giant single SysEx overruns the device's MIDI input
// buffer and arrives corrupt. begin(0x21,len) -> data(0x22)* -> commit(0x23); wait for each ACK.
async function _pushFirmware(src) {
const data = []; for (let i = 0; i < src.length; i++) data.push(src.charCodeAt(i) & 0x7F); // 7-bit ASCII
const n = data.length;
_send([0xF0, 0x7D, 0x21, n & 0x7F, (n >> 7) & 0x7F, (n >> 14) & 0x7F, (n >> 21) & 0x7F, 0xF7]);
function _b64(u8) { let s = ""; for (let i = 0; i < u8.length; i++) s += String.fromCharCode(u8[i]); return btoa(s); }
// Push the base64-encoded .mpy in flow-controlled chunks (512 base64 chars = a multiple of 4, so each
// decodes cleanly on the device): begin(0x21) -> data(0x22)* -> commit(0x23), waiting for each ACK. The
// device base64-decodes each chunk to /app.new, verifies the .mpy header, then A/B-installs + reboots.
async function _pushFirmware(b64) {
_send([0xF0, 0x7D, 0x21, 0xF7]);
if (await _ack(3000) !== true) return "handshake";
const CH = 512;
for (let o = 0; o < n; o += CH) {
_send([0xF0, 0x7D, 0x22].concat(data.slice(o, o + CH)).concat([0xF7]));
for (let o = 0; o < b64.length; o += CH) {
const part = b64.slice(o, o + CH); const msg = [0xF0, 0x7D, 0x22];
for (let i = 0; i < part.length; i++) msg.push(part.charCodeAt(i)); // base64 is ASCII
msg.push(0xF7); _send(msg);
const a = await _ack(4000);
if (a !== true) return "transfer at " + o + "/" + n + (a === false ? " (rejected)" : " (timeout)");
if (a !== true) return "transfer at " + o + "/" + b64.length + (a === false ? " (rejected)" : " (timeout)");
}
_send([0xF0, 0x7D, 0x23, 0xF7]); // commit: device verifies the whole file, then reboots
_send([0xF0, 0x7D, 0x23, 0xF7]);
return (await _ack(6000)) === true ? null : "verify";
}
// Where the new app.py comes from: the site when online (the https editor, same-origin), else let the user
// pick the file — so the OFFLINE editor that ships ON the device can update too (file:// pages can't fetch).
async function _firmwareSource() {
for (const url of ["/pico-cp-app.py", "https://metronome.varasys.io/pico-cp-app.py"]) {
try { const r = await fetch(url, { cache: "no-store" }); if (r.ok) return await r.text(); } catch (_) {}
}
alert("Can't reach the site from this offline copy.\n\nPick the firmware file (app.py) to flash — download it from\n" +
"metronome.varasys.io/pico-cp-app.py, or just open the online editor at\nmetronome.varasys.io/editor.html (it updates automatically).");
return await _pickFile(".py,text/x-python,text/plain", "PM_K-1 firmware (app.py)", { "text/x-python": [".py"] });
}
function _pickFile(accept, desc, fsTypes) { // resolve to the chosen file's text, or null if cancelled
function _pickBinary() { // offline fallback: choose an app.mpy -> Uint8Array (or null if cancelled)
return new Promise(async (res) => {
if (window.showOpenFilePicker) {
try { const [h] = await showOpenFilePicker({ types: [{ description: desc, accept: fsTypes }] });
return res(await (await h.getFile()).text()); }
try { const [h] = await showOpenFilePicker({ types: [{ description: "PM_K-1 firmware (app.mpy)", accept: { "application/octet-stream": [".mpy"] } }] });
return res(new Uint8Array(await (await h.getFile()).arrayBuffer())); }
catch (e) { if (e.name === "AbortError") return res(null); }
}
const inp = document.createElement("input"); inp.type = "file"; inp.accept = accept;
inp.onchange = () => { inp.files[0] ? inp.files[0].text().then(res) : res(null); };
const inp = document.createElement("input"); inp.type = "file"; inp.accept = ".mpy,application/octet-stream";
inp.onchange = async () => { inp.files[0] ? res(new Uint8Array(await inp.files[0].arrayBuffer())) : res(null); };
inp.oncancel = () => res(null);
inp.click();
});

View file

@ -23,12 +23,14 @@ from the web editor over USBMIDI**, and plays through your **computer's speak
1. **Flash CircuitPython:** hold **BOOTSEL**, plug in, drop the CircuitPython `.uf2` onto `RPIRP2`
(<https://circuitpython.org/board/raspberry_pi_pico/> — Pico 2 / W builds also fine). A `CIRCUITPY`
drive appears.
2. **Copy the whole bundle** onto `CIRCUITPY`: `boot.py`, `code.py` (loader) + `app.py` (the application),
`programs.json`, `font_s.bin` / `font_m.bin` / `font_l.bin`, `logo.bin` / `midi.bin` / `usb.bin` (the
on-screen logo + MIDI/USB status icons), `editor.html` (offline editor), and the helper scripts.
(`code.py` is a tiny stable loader; `app.py` is what firmware updates replace. The `.bin` assets — like
the fonts — ride in the bundle, since the one-click updater only pushes `app.py`; if a `.bin` is missing
the firmware just falls back to text and never fails to boot.)
2. **Copy the whole bundle** onto `CIRCUITPY`: `boot.py`, `code.py` (loader) + **`app.mpy`** (the
application, **precompiled**), `programs.json`, `font_s.bin` / `font_m.bin` / `font_l.bin`,
`logo.bin` / `midi.bin` / `usb.bin` (logo + MIDI/USB status icons), `editor.html` (offline editor),
and the helper scripts. **If an old `app.py` is on the drive, delete it** — the firmware ships as
precompiled `app.mpy`. (Why: CircuitPython compiling the ~57 KB source at boot fragments the heap and
runs the RP2040 out of memory; a `.mpy` loads without compiling. `code.py` is a tiny stable loader; the
one-click updater pushes a new `app.mpy`. The `.bin` assets ride in the bundle — if one is missing the
firmware just falls back to text and never fails to boot.)
3. **Powercycle** (so `boot.py` takes effect). It boots into appliance mode and runs.
## Program it from the web (push over USBMIDI)
@ -42,13 +44,14 @@ device answers — boot the Pico in **editor mode** (hold A) and drag the file o
## Firmware updates (oneclick, A/B with autorollback)
`code.py` is a small stable **loader**; the application is `app.py` (it carries `APP_VERSION`). To update:
the editor's ⋯ menu → **⬆ Update firmware…** queries the device's version, fetches the latest from the site,
shows *device vs latest*, and on confirm **pushes the new `app.py` over USBMIDI**. The device installs it to
a **trial slot** (keeping the old build as `app.bak`) and reboots; if the new build **doesn't boot, the loader
automatically rolls back** to `app.bak`. A build that runs cleanly for ~5 s is confirmed. No BOOTSEL, no
dragging. (Updating CircuitPython *itself* still uses BOOTSEL + a `.uf2`, but that's rare. And the Pico is
unbrickable as the ultimate backstop.)
`code.py` is a small stable **loader**; the application is the precompiled **`app.mpy`** (it carries
`APP_VERSION`). To update: the editor's ⋯ menu → **⬆ Update firmware…** queries the device's version, fetches
the latest `app.mpy` from the site, shows *device vs latest*, and on confirm **pushes it over USBMIDI**
(base64, in flowcontrolled chunks; the device decodes each chunk to disk and verifies the `.mpy` header
before installing). It goes to a **trial slot** (old build kept as `app.bak`) and reboots; if the new build
**doesn't boot, the loader automatically rolls back** to `app.bak`. A build that runs cleanly for ~5 s is
confirmed. No BOOTSEL, no dragging. (Updating CircuitPython *itself* still uses BOOTSEL + a `.uf2`, but that's
rare. And the Pico is unbrickable as the ultimate backstop.)
## Play through the computer's speakers

View file

@ -18,7 +18,7 @@
import board, busio, digitalio, analogio, pwmio, displayio, vectorio, time, json, gc, os, supervisor
supervisor.runtime.autoreload = False # we write our own files (log + pushed programs); never self-restart
APP_VERSION = "0.0.9" # firmware version (the A/B updater pushes/compares this)
APP_VERSION = "0.0.10" # firmware version (the A/B updater pushes/compares this)
try:
import rtc # set from the editor's clock SysEx so the log has real timestamps
except ImportError:
@ -37,6 +37,10 @@ try:
import usb_midi # default-enabled on RP2040 - sends a MIDI note per click to the computer
except ImportError:
usb_midi = None
try:
from binascii import a2b_base64 # decode the base64-encoded .mpy pushed by the editor's one-click update
except ImportError:
a2b_base64 = None
# ============================== CONFIG (tweak if needed) ==============================
SPI_BAUD = 62_500_000 # faster SPI = smaller tearing window; drop to 40_000_000 if unstable
@ -396,7 +400,7 @@ class App:
self.midi_in = usb_midi.ports[0] if (MIDI_ENABLED and usb_midi and len(usb_midi.ports) > 0) else None
self._mbuf = bytearray(64); self.midi_host = False; self.last_midi_in = 0.0
self._sx = bytearray(); self._sxon = False # USB-MIDI SysEx assembler (clock + pushed programs)
self._fw = None; self._fw_len = 0 # chunked firmware transfer: staging file + expected size
self._fw = None # chunked firmware transfer: staging file handle
self.led = RGB(P_RGB)
self.buz = pwmio.PWMOut(P_BUZ, frequency=1600, variable_frequency=True, duty_cycle=0)
self.buz_off = 0
@ -941,37 +945,35 @@ class App:
self._ack(False) # read-only (editor mode) etc.
# A/B firmware update, sent as small flow-controlled chunks (a single huge SysEx overruns the
# USB-MIDI input buffer and arrives corrupt). begin(0x21,len) -> data(0x22)* -> commit(0x23).
elif cmd == 0x21: # BEGIN: open the staging file; sx[2:6] = expected length (7-bit)
self._fw_len = (sx[2] | (sx[3] << 7) | (sx[4] << 14) | (sx[5] << 21)) if len(sx) >= 6 else 0
elif cmd == 0x21: # BEGIN firmware transfer: open the .mpy staging file
try:
try: self._fw.close()
except Exception: pass
self._fw = open("/app.new", "wb"); self._ack(True)
except Exception: # read-only (editor mode) / no space
self._fw = None; self._ack(False)
elif cmd == 0x22: # DATA: append a chunk to the staging file
elif cmd == 0x22: # DATA: a base64 chunk (multiple of 4) -> decode -> append
try:
if self._fw is None: raise OSError()
self._fw.write(bytes(sx[2:])); self._fw.flush(); self._ack(True)
if self._fw is None or a2b_base64 is None: raise OSError()
self._fw.write(a2b_base64(bytes(sx[2:]))); self._ack(True)
except Exception:
try: self._fw.close()
except Exception: pass
self._fw = None; self._ack(False)
elif cmd == 0x23: # COMMIT: verify the whole file, then A/B install + reboot
elif cmd == 0x23: # COMMIT: verify it's a CircuitPython .mpy, then A/B install
try:
try: self._fw.close()
except Exception: pass
self._fw = None; gc.collect()
with open("/app.new", "rb") as f: data = f.read()
if (self._fw_len and len(data) != self._fw_len) or (0 in data) \
or (b"App().run()" not in data) or (b"APP_VERSION" not in data):
try: os.remove("/app.new") # corrupt/truncated -> reject, keep the working build
with open("/app.new", "rb") as f: head = f.read(2)
if os.stat("/app.new")[6] < 4000 or len(head) < 2 or head[0] != 0x43 or head[1] != 0x06:
try: os.remove("/app.new") # not a CircuitPython mpy v6 -> reject, keep the working build
except OSError: pass
self._ack(False); return
try: os.remove("/app.bak")
except OSError: pass
os.rename("/app.py", "/app.bak") # current build becomes the rollback
os.rename("/app.new", "/app.py")
os.rename("/app.mpy", "/app.bak") # current build becomes the rollback
os.rename("/app.new", "/app.mpy")
open("/trial", "w").close() # arm the trial; the loader reverts if it won't boot
self._ack(True); time.sleep(0.4); supervisor.reload()
except Exception: # catch ALL (read-only, MemoryError, ...) -> never brick

View file

@ -1,9 +1,10 @@
# code.py - PM_K-1 A/B firmware loader (stable; rarely changes).
#
# The real application lives in app.py; app.bak holds the previous known-good build. The web editor
# pushes a new app.py to a "trial" slot over USB-MIDI; this loader runs it, and if the new build
# fails to boot it AUTOMATICALLY ROLLS BACK to app.bak. (The Pico is also unbrickable: BOOTSEL ->
# drag a CircuitPython .uf2.) app.py clears the /trial marker once it has run healthily for ~5s.
# The real application is the PRECOMPILED app.mpy (CircuitPython compiles a big .py at boot, which
# fragments the heap and OOMs; a .mpy loads without compiling). app.bak holds the previous known-good
# build. The web editor pushes a new app.mpy to a "trial" slot over USB-MIDI; this loader runs it, and
# if it fails to boot it AUTOMATICALLY ROLLS BACK to app.bak. (Unbrickable: BOOTSEL -> drag a .uf2.)
# app.mpy clears the /trial marker once it has run healthily for ~5s.
import supervisor, os
supervisor.runtime.autoreload = False # updates reboot explicitly; never auto-restart on our own writes
@ -12,11 +13,11 @@ def _trial():
except OSError: return False
try:
import app # runs the application (app.py ends with App().run())
import app # runs the application (app.mpy; ends with App().run())
except Exception:
if _trial(): # a freshly-pushed build crashed on startup -> roll back
try:
os.remove("/app.py"); os.rename("/app.bak", "/app.py"); os.remove("/trial")
os.remove("/app.mpy"); os.rename("/app.bak", "/app.mpy"); os.remove("/trial")
except Exception: pass
supervisor.reload() # reboot into the restored known-good build
else: