metronome/docs/track-format.md

# PM track format — specification

This is the **single source of truth** for the metronome's track ("program" / "patch")
format. The grammar is currently implemented by hand in three places that have quietly
drifted:

- **Web** — `src/engine.js` (`patchToSetup` / `laneStrToCfg` / `setupToPatch` / `laneCfgToStr`)
- **Firmware** — `pico-cp/app.py` (`parse_program` / `_parse_lane` / `lane_to_str` / `_prog_str`)
- (`editor.html` inlines `engine.js` at build time)

`tests/fixtures/track-format.json` holds golden vectors that pin every feature to a single
expected meaning. `tests/run.mjs` runs both the JS and Python implementations against them
and reports divergences. **Any new implementation (e.g. a Rust engine) must pass the same
vectors** — that is what keeps "the same groove on the device and in the browser" true.

Status legend used below and in the fixtures:

- **stable** — implemented and (intended to be) identical on web + firmware.
- **divergence** — a real cross-implementation disagreement that exists *today*; the vector
  encodes the spec's intended behavior and the runner flags the side that is wrong.
- **new** — defined here but not yet implemented anywhere (the playback-flow model below);
  the vector is the acceptance test for building it.

---

## 1. Container: `programs.json`

The device reads set-lists from `/programs.json` (pushed by the editor over USB-MIDI SysEx,
or dragged onto the CIRCUITPY drive). Built-in set-lists are baked into firmware and are *not*
in this file.

```jsonc
{
  "format": 2,                       // NEW. absent ⇒ treat as format 1 (legacy)
  "setlists": [
    {
      "title": "My set",
      "description": "optional",
      "onEnd": "stop",               // NEW: stop | nextList | loop   (default: stop)
      "defaultEnd": "next",          // NEW: items without their own end= inherit this
      "programs": [
        { "name": "Intro", "prog": "t88;b8;kick:4=X.x.;end=next" },
        { "name": "Groove", "prog": "t88;kick:4;snare:4=.X.X" }
      ]
    }
  ]
}
```

**Legacy (format 1):** `{ "setlists":[...] }` with no `format`, or the older flat
`{ "programs":[...] }` (a single list). Loaders MUST still accept both. Migration to format 2:
set `format: 2`; a previously "continuous" set-list gets `onEnd` per the author's intent and
`defaultEnd: "next"` (or `end=next` on each item).

---

## 2. Patch string grammar

A patch is `;`-separated tokens. **Rule that prevents token collisions:** a token containing
`:` is a **lane**; every other token is a **keyword directive** matched against the reserved
set `{ v1, t, vol, cd, b, tr, rmp, rep, end }`. Unknown tokens are ignored (see §6).

```ebnf
patch      = [ "v1" ";" ] directive *( ";" directive ) ;

directive  = tempo | volume | countin | bars | trainer | ramp | rep | end | lane ;

tempo      = "t" int ;                       (* beats per minute, clamped to 5..300 *)
volume     = "vol" int ;                      (* 0..100; web-authoring only — device has a hardware volume knob *)
countin    = "cd" int ;                       (* count-in seconds; web-authoring only — device has no count-in *)
bars       = "b" int ;                         (* cycle length in bars; drives Continue/rep *)
trainer    = "tr" int "/" int ;                (* playBars "/" muteBars (gap trainer) *)
ramp       = "rmp" int "/" signed "/" int ;    (* startBpm "/" amount "/" everyBars *)

rep        = "rep=" int ;                       (* cycles before end fires; default 1 *)
end        = "end" "=" ( "stop" | "next" | signed ) ;  (* NEW: see §3 *)

lane       = sound ":" groups [ "/" sub [ "s" ] ] [ euclid ] [ "=" pattern ]
                    [ "@" signed ] [ "~" ] [ "!" ] ;
sound      = name | int ;                       (* int = GM percussion note number alias *)
groups     = int *( "+" int ) ;                 (* "4"  or  "2+2+3" → beats per bar *)
sub        = int ;                              (* subdivision; trailing "s" = swing *)
euclid     = "(" int [ "," int [ "," int ] ] ")" ;   (* k [, n [, rot ]] — even distribution *)
pattern    = *( "X" | "x" | "g" | "." | "-" | "_" ) ; (* per-step dynamics *)
```

### Lane semantics

- **groups** — `2+2+3` means a 7-beat bar grouped 2+2+3; the first step of each group is
  accented by default. `beatsPerBar = sum(groups)`.
- **sub** — steps per beat (`/2` eighths, `/3` triplets, `/4` sixteenths). `steps = beats*sub`.
- **swing** — `/2s` lays the off-beat on the last triplet (≈ 2/3).
- **pattern** — one char per step: `X`=accent (level 2), `x`/`1`=normal (1), `g`=ghost (3),
  `.`/`-`/`_`/anything else = rest (0). Short patterns are right-padded with rests to `steps`.
  **With no pattern (the default):** every step sounds at normal level and accents fall **only
  on group starts** — the grouping *is* the accent map. So `4` accents beat 1; `2+2` accents
  beats 1 & 3; `4/2` is a steady 8th lane with an accent on beat 1. To accent every beat,
  write the grouping (`1+1+1+1`) rather than relying on the default.
- **euclid** `(k,n,rot)** — `k` hits spread as evenly as possible over `n` steps (rotated by
  `rot`), first hit accented. Replaces an explicit `=pattern`.
- **`@<db>`** — per-lane gain in dB. (Host-optional: applied in the browser; see §6.)
- **`~`** — polymeter: this lane keeps its own bar length independent of the master lane.
- **`!`** — lane present but silenced/disabled.

### Top-level directives

- **`t`** tempo, clamped to `[5, 300]`.
- **`b<n>`** the segment's **cycle length** in bars — used for Continue/`rep` accounting and
  song auto-advance. Absent ⇒ the cycle is one bar of the **master lane** (the first lane).
- **`rmp<start>/<amt>/<every>`** tempo ramp: every `<every>` bars, change tempo by `<amt>`
  (may be negative), starting from `<start>`.
- **`tr<play>/<mute>`** gap trainer: play `<play>` bars, silence `<mute>` bars, repeat.
- **`vol`, `cd`** web-authoring only — the device has a hardware volume knob and no count-in, so
  it parses past these and does not carry them (see §6).

---

## 3. Playback flow

Per-track playback behavior (parsed + serialized by both engines; firmware runtime implemented).
**Implementation note:** the device keeps the global `Continue` toggle as a *default* — a track
with an explicit `end=` governs itself; a track without one falls back to `end=next` while
Continue is on (and still needs `b<bars>`), else it loops. So per-track `end=` overrides the
global toggle rather than replacing the UI.

**Default (no `end=` token) — loop forever**, exactly like a metronome. Manual advance
(joystick / footswitch) always moves to the next track. "Vamp until cue" is therefore the
default and needs no special token.

`end=` exists only to make flow **automatic** after a finite number of cycles:

| Token            | Meaning                                                              |
|------------------|---------------------------------------------------------------------|
| *(absent)*       | Loop forever. Manual advance → next track.                          |
| `end=stop`       | After `rep` cycles, stop.                                            |
| `end=next`       | After `rep` cycles, auto-advance one track. (Sugar for `end=+1`.)    |
| `end=<±N>`       | **Relative goto**: after `rep` cycles, jump `N` tracks (e.g. `end=-2` repeats a section, D.S.). |
| `rep=<N>`        | Cycles to play before `end` fires. Default `1`. Only meaningful with `end`. |

A **cycle** = `b<bars>` bars if `b` is present, else one bar of the master lane.

**Normalization** (what the golden vectors compare): `end` normalizes to `"stop"`, or an
integer offset (`next` ⇒ `1`); absent ⇒ `null`. `rep` ⇒ the integer (defaults to `1` when
`end` is present), else `null`.

### Resolution & boundaries

- **Manual override always wins.** `end=` is only the hands-off behavior.
- **Seam is gapless.** An automatic `next`/goto does **not** re-trigger count-in and inserts
  no gap (honors the existing `_seam_t` seam). Count-in (`cd`) fires only on manual/initial
  start. Tempo jumps to the destination track's `t` at the seam.
- **Goto past the last item** (or `end=next` on the last item) triggers the set-list's
  `onEnd` policy: `stop` | `nextList` | `loop`.
- **Goto before the first item** clamps to the first item.
- **`defaultEnd`** on a set-list is inherited by any item whose patch has no `end=`.

---

## 4. Canonical form & round-tripping

Serializing a parsed patch MUST be **idempotent**: `serialize(parse(serialize(parse(x))))`
equals `serialize(parse(x))`. Host-optional fields that a host does not act on
(`vol`, `cd`, `@db`) MUST still survive the round-trip rather than being dropped — silent
data loss is the failure mode this spec exists to prevent.

The runner checks **semantic equality** (the normalized structure — what actually plays)
across implementations, plus per-implementation idempotency. It does **not** require the two
serializers to emit byte-identical strings (they legitimately differ on the optional `v1`
prefix and on whether a default pattern is written out).

---

## 5. Normalized structure (the comparison target)

Each implementation's adapter parses a patch and emits this neutral shape; vectors store the
expected value in `norm`:

```jsonc
{
  "bpm": 120, "bars": 0, "volume": null, "countMs": 0,
  "ramp":    null,            // or { "start": 80, "amt": 4, "every": 4 }
  "trainer": null,            // or { "play": 2, "mute": 2 }
  "rep":     null,            // NEW
  "end":     null,            // NEW: "stop" | <int> | null
  "lanes": [
    { "sound": "kick", "groups": [4], "sub": 1, "swing": false,
      "poly": false, "mute": false, "gainDb": 0, "levels": [2,1,1,1] }
  ]
}
```

`levels` is the resolved per-step dynamics array (0 rest / 1 normal / 2 accent / 3 ghost) —
the real audible payload, and the most important thing two implementations must agree on.

---

## 6. Divergences — status

Surfaced by the runner. **Resolved** (now identical on web + firmware, verified by the suite):

- **Default (no-pattern) groove** — every subdivision sounds; accent only on group starts.
- **Euclid `(k,n,rot)`** — now parsed by both engines (`kick:4(3,8)` etc.).
- **Unknown sound name** — falls back to `beep` on both.
- **GM note-number aliases** — `36:4` resolves to the voice name on both.

**Intentional / permanent host differences** (not bugs — the device is a host that lacks these):

- **`vol` (master volume) / `cd` (count-in)** — web-authoring fields. The device has a hardware
  volume knob and no count-in, so it parses past them and does not carry them. (Contrast `@db`
  gain, which the device *does* round-trip as a per-lane field even though it doesn't apply it.)

**Resolved on the web side** (`engine.js` now matches the firmware):

- **Tempo clamp** — `patchToSetup` clamps `t` to `[5,300]`.
- **Empty patch** — `patchToSetup` defaults to a `beep:4` lane when no lanes are given. (The
  editor still shows its "no lanes" hint by checking the *raw* input for a `:` token.)

The capability/version handshake (the device already replies with its firmware version over
SysEx) should gate features so the editor can warn when a track uses something the connected
firmware is too old to play, instead of letting it degrade silently.