cb54b4d689
The parallel agent's full session, committed now that it's solo: - Grammar: flam/drag/roll ornaments (f/F d/D z/Z, per-lane orns channel) across src/engine.js, pico-cp/pico-explorer/pico-scroll app.py, pico/main.py, rust/track-format, + golden vectors / conformance (tests/, rust/track-format/tests). - Live-sync deep-sync: SysEx 0x44 SLSYNC + 0x45 LOGSYNC (docs/livesync-protocol.md, src/livesync.js). - PM_E-2 notation: web engine (pm_e-2.html, build/deploy/index/embed wiring) + Rust device port (pm-ui draw_notation rewrite + LaneView.groups, pm-kit ViewMode, uisim notesim). Verified: node tests/run.mjs 47 pass / 1 known; ./rust/run.sh green; pm-kit firmware + uisim compile.
369 lines
20 KiB
Markdown
369 lines
20 KiB
Markdown
# PM Live-Sync protocol (beta)
|
||
|
||
Bidirectional live mirror between the **PM_E‑1 editor** (web) and a **PM_K‑1 device**
|
||
(firmware). When armed, either side can edit a groove, change tempo/volume,
|
||
start/stop, or select a set‑list item, and the other side reflects it in real
|
||
time.
|
||
|
||
It rides the **existing USB‑MIDI SysEx channel** (manufacturer `0x7D`) that the
|
||
device link already uses for RTC / version / programs / firmware — no new
|
||
transport, no new browser permission.
|
||
|
||
- **Editor side:** implemented in `src/livesync.js` + hooks in `editor-beta.html`.
|
||
- **Device side:** to be implemented in `pico-cp/app.py` (this document is the contract).
|
||
- **Browser support:** Web MIDI = Chrome / Edge / Firefox (no Safari), same as the existing "Device audio" feature.
|
||
|
||
---
|
||
|
||
## 1. Frames
|
||
|
||
Every message is one SysEx frame:
|
||
|
||
```
|
||
F0 7D <op> <payload ASCII bytes, each 0x00–0x7F> F7
|
||
```
|
||
|
||
`<op>` lives in the free `0x40` block (existing ops: `0x01` RTC, `0x02/0x03`
|
||
version, `0x10` programs, `0x21/22/23` firmware, `0x7E/0x7F` NAK/ACK):
|
||
|
||
| op | name | direction | payload |
|
||
|------|---------|------------------|-------------------------------------------|
|
||
| 0x40 | HELLO | either → either | `<origin>` |
|
||
| 0x41 | FULL | either → either | `<origin>;<seq>;<running>;<sl>;<item>;<patch>` |
|
||
| 0x42 | DELTA | either → either | `<origin>;<seq>;<evt>` |
|
||
| 0x43 | BYE | either → either | `<origin>` |
|
||
| 0x44 | SLSYNC | either → either | `<origin>;<seq>;<json>` — live set-list **content** merge (§8) |
|
||
| 0x45 | LOGSYNC | either → either | `<origin>;<seq>;<json>` — practice-log entry merge (§9) |
|
||
|
||
- **Payload is 7‑bit ASCII** — never emit a byte > `0x7F` (it corrupts the SysEx
|
||
stream and, per `build.sh`, would also break the firmware‑update path). All
|
||
share‑language patch strings are already ASCII.
|
||
- `<origin>` — a short per‑session id (the editor uses e.g. `e1a2b3c`). Used to
|
||
drop your own echoes (see §4).
|
||
- `<seq>` — a monotonically increasing integer per sender. Informational /
|
||
duplicate‑drop; ordering is guaranteed by USB‑MIDI so no reordering logic is
|
||
required.
|
||
- `<running>` — `0` or `1`.
|
||
- `<sl>` / `<item>` — set‑list and item index of the loaded program, or `-1`.
|
||
- `<patch>` — a share‑language patch string (see §3). It contains `;` and `/`,
|
||
so it is **always the tail**: parse the first 5 `;`‑fields, then rejoin the
|
||
rest as the patch.
|
||
|
||
---
|
||
|
||
## 2. DELTA event grammar (`<evt>`)
|
||
|
||
One mutation, no `;` inside. Reuses the share‑language tokens (see
|
||
`src/engine.js` / README "Share language").
|
||
|
||
| evt | meaning |
|
||
|------------------------------|----------------------------------------------------|
|
||
| `play` | start transport |
|
||
| `stop` | stop transport |
|
||
| `bpm=<n>` | set tempo (clamped to the firmware's BPM range) |
|
||
| `vol=<pct>` | master volume, 0–100 |
|
||
| `sel=<sl>/<item>` | cue/load a set‑list item |
|
||
| `beat=<lane>/<step>/<level>` | per‑step dynamics; level `0/1/2/3` = mute/normal/accent/ghost |
|
||
| `lane=<lane>/<field>/<value>`| lane field edit (see below) |
|
||
|
||
`<lane>` and `<step>` are **0‑based** indices into the current program's lane
|
||
list / that lane's step list (same order both sides).
|
||
|
||
`lane=` fields and values:
|
||
|
||
| field | value |
|
||
|-----------|------------------------------------------------|
|
||
| `sound` | voice name (`kick`, `snare`, `hatClosed`, …) |
|
||
| `groups` | grouping string, e.g. `2+2+3` |
|
||
| `sub` | subdivision int: 1 / 2 / 3 / 4 / 6 |
|
||
| `swing` | `0` or `1` |
|
||
| `gain` | dB int, e.g. `-3` |
|
||
| `poly` | `0` or `1` |
|
||
| `enabled` | `0` or `1` (0 = silenced lane) |
|
||
|
||
> Structural changes that re‑shape the lane list (add lane, remove lane,
|
||
> reorder) are **not** sent as deltas. Send a fresh **`0x41` FULL** instead — it
|
||
> is simpler and self‑healing. The editor does exactly this (a coalesced
|
||
> full‑state push ~150 ms after the last structural/practice edit).
|
||
|
||
---
|
||
|
||
## 3. What each side emits vs. applies
|
||
|
||
The two halves are **asymmetric in what they emit but symmetric in what they
|
||
apply** — each must apply *every* op/evt listed above.
|
||
|
||
**Editor emits:**
|
||
- fine `0x42` deltas for `play`/`stop`, `bpm`, `vol`, `sel`, `beat`
|
||
- a coalesced `0x41` FULL for any lane‑field / add / remove / practice (trainer,
|
||
ramp, segment bars, countdown) edit
|
||
- `0x41` FULL on connect and in reply to a received `0x40`
|
||
- a coalesced `0x44` SLSYNC on any set‑list **content** change (and on connect) — §8
|
||
- a `0x45` LOGSYNC after each logged session (and a full batch on connect) — §9
|
||
|
||
**Device should emit** (from its on‑device input handlers):
|
||
- `play`/`stop` when button A toggles transport
|
||
- `bpm=<n>` when the joystick / tap changes tempo (throttle to ≤ ~10/s)
|
||
- `sel=<sl>/<item>` on set‑list navigation
|
||
- `beat=<lane>/<step>/<level>` on a touch beat edit (`app.py` ~573–625)
|
||
- a `0x41` FULL after any lane add/remove/reorder or multi‑field lane edit
|
||
- a periodic `0x41` FULL **heartbeat** (~every 3–5 s) — the device is the
|
||
convergence authority (see §4)
|
||
- `0x41` FULL in reply to a received `0x40`
|
||
|
||
The `patch` in a `0x41` is produced by the device's existing program serializer
|
||
(the inverse of `parse_program()` in `app.py`). It must round‑trip through the
|
||
editor's `patchToSetup()` — i.e. the same grammar already used for
|
||
`programs.json` `prog` strings, plus a leading `t<bpm>` and optional `vol<pct>`.
|
||
|
||
---
|
||
|
||
## 4. Echo / loop suppression and conflict policy
|
||
|
||
Two rules keep the mirror from oscillating:
|
||
|
||
1. **Applying a remote change never re‑broadcasts.** Wrap every apply in an
|
||
"applying remote" flag (the editor uses `_applyingRemote`) and have all of
|
||
your broadcast hooks early‑out while it is set. This is the primary guard.
|
||
2. **Drop your own origin.** On receive, if `origin == myOrigin`, ignore the
|
||
frame. (Belt‑and‑suspenders; also lets the editor's `?loopback=1` self‑test
|
||
work by relabeling echoes as a peer.)
|
||
|
||
**Convergence:** the **device is authoritative**. Its periodic `0x41` heartbeat
|
||
is treated as ground truth, so if both sides edited the same field in the same
|
||
instant, they reconcile within one heartbeat. To avoid flicker, a receiver
|
||
should **diff the incoming `patch` against its current state and skip the
|
||
rebuild if they're equal** (the editor does this in `_applyFull`), only
|
||
reconciling transport.
|
||
|
||
This is single‑user‑friendly (last‑writer‑wins per field). True simultaneous
|
||
multi‑editor use is out of scope for the beta.
|
||
|
||
---
|
||
|
||
## 5. Handshake & lifecycle
|
||
|
||
```
|
||
editor "Live sync" ON ─► 0x40 HELLO ─────────────► device
|
||
◄──────────── 0x41 FULL ◄── (device's current state)
|
||
editor 0x41 FULL ──────────────────────────────► (editor's current state)
|
||
… steady state: 0x42 deltas both ways, device 0x41 heartbeat …
|
||
editor "Live sync" OFF ─► 0x43 BYE ────────────► device
|
||
```
|
||
|
||
On connect the editor sends **both** a `0x40` (asking for the device's state)
|
||
and a `0x41` (offering its own), so whichever side the user considers "source of
|
||
truth" wins immediately. A device that boots with sync idle should simply answer
|
||
`0x40` with a `0x41` and start emitting deltas once it has heard from a peer.
|
||
|
||
---
|
||
|
||
## 6. Firmware checklist (`pico-cp/app.py`)
|
||
|
||
- [ ] **Dispatch** `0x40/0x41/0x42` in the SysEx handler (~`app.py:1361‑1415`,
|
||
alongside `0x01/0x02/0x10/0x21‑23`). Ignore frames whose origin is your own.
|
||
- [ ] **HELLO (`0x40`)** → reply `0x41` FULL built from current `App` state
|
||
(running, sl/idx, serialized program).
|
||
- [ ] **FULL (`0x41`)** → diff vs. current program; if different, load it
|
||
(reuse `parse_program()` / the `programs.json` load path); then reconcile
|
||
`running` (start/stop). Wrap in your remote‑apply flag.
|
||
- [ ] **DELTA (`0x42`)** → apply `play/stop/bpm/vol/sel/beat/lane` to `App`
|
||
state, wrapped in the remote‑apply flag so the on‑device handlers don't
|
||
re‑broadcast.
|
||
- [ ] **Broadcast** a `0x42` from each on‑device input handler (button A,
|
||
joystick tempo, touch beat edit, set‑list nav, lane editor), guarded by
|
||
the remote‑apply flag. Structural lane changes → `0x41` FULL.
|
||
- [ ] **Heartbeat:** emit `0x41` FULL every ~3–5 s while a peer is connected.
|
||
- [ ] **BYE (`0x43`)** → mark the peer gone (stop heartbeating/emitting until
|
||
the next HELLO).
|
||
- [ ] **SLSYNC (`0x44`)** → merge the JSON manifest of user lists by normalized
|
||
title (replace‑per‑list, append unknown, never delete), reusing
|
||
`load_user_setlists()`‑shaped parsing; `rebuild_setlists()` + reload.
|
||
Emit one after `_persist_user()` and in reply to `0x40`. (§8)
|
||
- [ ] **LOGSYNC (`0x45`)** → merge practice entries by (`at`,`name`); append +
|
||
cap + re‑sort. Emit a one‑entry batch after `_log_play()` and a full batch
|
||
in reply to `0x40`. Record `at` (epoch ms) on each play when the RTC is
|
||
set. (§9)
|
||
- [ ] **Throttle** high‑rate sources (joystick tempo) and keep frames small —
|
||
the RP2040 USB‑MIDI RX buffer is tiny (the firmware updater already chunks
|
||
at 64 bytes), and live traffic shares the bus with MIDI clock, note‑out,
|
||
and the editor's Active‑Sensing heartbeat. Don't let a flood stall a
|
||
concurrent firmware push.
|
||
|
||
### Built‑in vs. user set lists (must match the editor)
|
||
|
||
The PM_K‑1's built‑in playlists (Styles / Practice / Song) are **baked into
|
||
firmware and read‑only**; on‑device edits **copy‑on‑write** into the user
|
||
"My edits" list. The editor follows the same rule (`userSetlists()` excludes the
|
||
seeded titles). So a **remote edit that targets a built‑in must follow the same
|
||
copy‑on‑write semantics** on the receiving side, or the two halves will disagree
|
||
about where the edit landed. When in doubt, after such an edit send a `0x41`
|
||
FULL with the resulting (copied) program so both sides converge on the same
|
||
target.
|
||
|
||
### Out of scope for the beta
|
||
- Mirroring device `settings.json` (LED brightness, MIDI config, etc.).
|
||
- Multi‑peer / multi‑editor arbitration beyond last‑writer‑wins.
|
||
|
||
> **No longer out of scope** (now specced in §8 / §9): live set‑list **content**
|
||
> sync (`0x44`) and streaming the device practice log up to the browser and back
|
||
> (`0x45`). The old `0x10` programs push (Save/Load to device) still exists as the
|
||
> explicit, full‑overwrite path; `0x44` is the *incremental, merge‑by‑title* live
|
||
> mirror that runs automatically while sync is armed.
|
||
|
||
---
|
||
|
||
## 7. Per‑device emit/apply matrix
|
||
|
||
Both targets implement the **full apply path** for every verb. They differ in what
|
||
they **emit**, because on‑device editing differs:
|
||
|
||
| Device | Emits | Applies |
|
||
|-------------|----------------------------------------------------|---------------------------------------------|
|
||
| **PM_K‑1** Kit (touchscreen + joystick) | `play` / `stop` / `bpm` / `sel` / `beat` / `lane` (FULL on structural lane edits) | all of the above |
|
||
| **PM_X‑1** Explorer (6 buttons, read‑only beats) | `play` / `stop` / `bpm` / `sel` only (no on‑device beat/lane editing) | all of the above |
|
||
| **PM_G‑1** Grid (17×7 LED matrix, 4 buttons, read‑only beats) | `play` / `stop` / `bpm` / `sel` only (no on‑device beat/lane editing) | all of the above |
|
||
|
||
Editors don't need to special‑case the source — both DELTA streams look identical on
|
||
the wire, and the **device id is only exposed on the version query** (SysEx `0x02`
|
||
→ `0x03` reply, `<id>;<version>`; pre‑0.0.23 firmware sends bare version → assume
|
||
`K`).
|
||
|
||
---
|
||
|
||
## 8. Set‑list content sync (`0x44` SLSYNC)
|
||
|
||
The `0x41` FULL only carries the *one loaded program* (`<patch>`) plus the
|
||
*selection* indices. `0x44` carries **set‑list content** — titles + every item's
|
||
name + program string — so the two halves converge on the same library while
|
||
sync is armed, without the user pressing "Save to device".
|
||
|
||
**Frame:** `F0 7D 44 <origin>;<seq>;<json> F7`
|
||
|
||
- `<origin>` / `<seq>` — same as the other ops (echo‑drop + duplicate info).
|
||
- `<json>` — a **7‑bit‑safe JSON** manifest of the sender's **user** set lists,
|
||
in the *exact same shape `0x10` already uses* so the firmware can reuse its
|
||
`programs.json` parser:
|
||
|
||
```json
|
||
{"setlists":[{"title":"My set list","programs":[{"name":"Funk","prog":"t120;..."}]}]}
|
||
```
|
||
|
||
Non‑ASCII in titles/names is escaped `\uXXXX` (the editor's existing
|
||
`programsJSON()` 7‑bit‑safe path); the firmware stores it verbatim. The whole
|
||
manifest rides **one SysEx frame** (same as `0x10` — user libraries are a few
|
||
KB and the RX assembler holds 60 000 bytes). It is **never chunked**; if it
|
||
ever grew past the buffer, fall back to the explicit `0x10` push.
|
||
|
||
### What's included
|
||
- **Only user set lists.** Built‑in / seeded lists (firmware `BUILTIN_SETLISTS`;
|
||
editor `SEED_SETLISTS` titles) are read‑only on both halves and **never
|
||
transmitted** — both sides already have identical copies baked in. (Same filter
|
||
as `userSetlists()` / `load_user_setlists()`.)
|
||
|
||
### Identity & merge rule
|
||
- **Set lists match by title** (normalized: lower‑case, alphanumerics only — the
|
||
firmware's `_slkey()`), independent of index. Indices diverge freely between
|
||
halves (the device prepends built‑ins; the web orders differently), so a
|
||
positional match is wrong — **title is the key.**
|
||
- **Items match by name** within a list (case‑sensitive, as both UIs key
|
||
practice history by exact name).
|
||
- Merge is a **per‑list replace**: a received user list **replaces** the local
|
||
user list of the same normalized title wholesale (its items become the
|
||
received items, in the received order). Lists present locally but absent from
|
||
the message are **left untouched** (additive — sync never deletes a list the
|
||
peer simply didn't send). A received list with no matching local title is
|
||
**appended** as a new user list.
|
||
- This is **last‑writer‑wins per list** (consistent with the rest of the
|
||
protocol). The receiver applies under its remote‑apply guard and does **not**
|
||
re‑broadcast a `0x44` in response (no echo storm); the next heartbeat / FULL
|
||
still reconciles the loaded program.
|
||
|
||
### Copy‑on‑write for built‑ins
|
||
A `0x44` never targets a built‑in: it only carries user lists, and the receiver
|
||
only ever writes user lists. If a user **edits a built‑in item** on either half,
|
||
that edit must first be **forked into a user list** (the firmware's `_save_edit`
|
||
already forks built‑in edits into the "My edits" user list; the editor keeps a
|
||
separate user list). The fork then rides `0x44` as an ordinary user list. So
|
||
copy‑on‑write happens **before** the sync, and the wire only ever sees user
|
||
content — the built‑ins on both sides stay pristine and identical.
|
||
|
||
### When it's emitted
|
||
- **Editor:** coalesced ~150 ms after any set‑list structural edit (add/rename/
|
||
reorder list, add/remove/rename item, capture/update an item), and once on
|
||
connect right after the first FULL. Reuses `syncPatchSoon()`‑style debouncing.
|
||
- **Device:** after `_persist_user()` succeeds (a save that wrote
|
||
`programs.json`), guarded by the remote‑apply flag, and once in reply to a
|
||
`0x40` HELLO. The device's per‑item *program* edits still ride `0x41` FULL;
|
||
`0x44` is specifically for **library shape** (which lists/items exist).
|
||
|
||
> The device is the **convergence authority** for the *loaded program* (§4), but
|
||
> set‑list content is last‑writer‑wins per list — there is no periodic `0x44`
|
||
> heartbeat (it would clobber concurrent edits on the other half). Send it only
|
||
> on an actual content change or on connect.
|
||
|
||
---
|
||
|
||
## 9. Practice‑log sync (`0x45` LOGSYNC)
|
||
|
||
Both halves keep a practice history (web: `localStorage` `metronome.logs`;
|
||
device: `/history.json`). `0x45` streams entries between them and **merges by a
|
||
stable key**, so a session played on the device shows up in the editor's history
|
||
graph and vice‑versa.
|
||
|
||
**Frame:** `F0 7D 45 <origin>;<seq>;<json> F7`
|
||
|
||
`<json>` is a 7‑bit‑safe JSON batch of **normalized** entries:
|
||
|
||
```json
|
||
{"log":[{"at":1733059200000,"name":"Funk","dur":92,"bpm":120}]}
|
||
```
|
||
|
||
| field | type | meaning |
|
||
|--------|-------------|------------------------------------------------------------|
|
||
| `at` | int (ms) | session start, **Unix epoch milliseconds** — the dedup key |
|
||
| `name` | string | set‑list item name the session was logged against |
|
||
| `dur` | int (sec) | session duration in **whole seconds** |
|
||
| `bpm` | int | tempo at the end of the session |
|
||
|
||
This is the **intersection** of the two native schemas (the web's
|
||
`{at,name,durationSec,bpm,lanes}` and the device's `{t,bpm,dur,bars,name}`):
|
||
`at` ↔ `at`, `dur` ↔ `round(durationSec)` / `dur`, `bpm` ↔ `bpm`, `name` ↔
|
||
`name`. Fields each side keeps privately (web `lanes`; device `t`/`bars`) are
|
||
**not** transmitted; the receiver fills them from what it has (`t` from `at` via
|
||
the RTC, `bars`/`lanes` left absent).
|
||
|
||
### Timestamps & the device clock
|
||
The dedup key is `at` (epoch ms). The editor already pushes the RTC over `0x01`
|
||
on connect / heartbeat, so the device **can** produce a real epoch. The firmware
|
||
therefore records an `at` (epoch **seconds** × 1000 → ms) on each logged play
|
||
*in addition to* its existing `t:"HH:MM"` field, computed from `time.time()`
|
||
when the RTC is set; if the RTC is unset (no editor ever connected) it falls
|
||
back to `at = 0`, which the merge treats as "no stable key" (see dedup).
|
||
|
||
### Direction & dedup/merge
|
||
- **Bidirectional.** Each half emits its **own** local entries; each applies the
|
||
peer's.
|
||
- **Dedup key = `at` + `name`.** On receive, an entry whose (`at`,`name`) already
|
||
exists locally is dropped. Entries with `at == 0` (device logged before any RTC
|
||
sync) are **always appended** (can't be deduped — better a possible duplicate
|
||
than a dropped session); these are rare and the user can delete them.
|
||
- Merge is **additive only** — `0x45` never deletes history. (Deleting a stale
|
||
entry on one half does not propagate; out of scope, matches the
|
||
last‑writer‑wins philosophy and avoids a delete echoing into a re‑add.)
|
||
- The receiver **caps** its merged log (web keeps all; device keeps newest 200,
|
||
its existing cap) and re‑sorts newest‑first by `at`.
|
||
|
||
### When it's emitted
|
||
- **Editor:** after `logFinalize()` writes a new session (one‑entry batch), and
|
||
a **full batch** once on connect (after the first FULL) so the device catches
|
||
up on everything it missed. Guarded so an *applied* remote entry doesn't
|
||
re‑broadcast.
|
||
- **Device:** after `_log_play()` appends a session (one‑entry batch), guarded by
|
||
the remote‑apply flag, and a full batch once in reply to a `0x40` HELLO.
|
||
|
||
> **Batch size caution (firmware):** a full‑history batch (up to 200 entries) is
|
||
> small JSON but still allocates; the device sends its on connect/HELLO only, and
|
||
> the editor's on‑connect batch is bounded by the SysEx buffer (60 KB ≈ a few
|
||
> thousand minimal entries). If a log ever exceeded that, the editor truncates to
|
||
> the newest entries that fit. Per‑session emits are a single entry — negligible.
|