Formalize track format: spec + golden-vector conformance suite

Single source of truth for the track ("program"/"patch") grammar, which was
implemented by hand in src/engine.js and pico-cp/app.py with no cross-check and
had quietly drifted.

- docs/track-format.md: formal grammar, container (programs.json) schema with a
  version field, the new per-track playback-flow model (rep/end + relative goto;
  default = loop forever), normalization rules, and a list of known divergences.
- tests/: golden vectors + a runner that loads the REAL engine.js and app.py
  grammar (no copies; app.py via ast extraction) and compares both against the
  spec. Exit non-zero on unexpected mismatch or round-trip break -> usable as CI.

Surfaces real divergences for follow-up: default accent pattern (no =pattern)
differs web vs device and affects shipped presets; euclid not parsed on device;
vol/cd dropped on device; unknown-sound fallback; tempo clamp; empty patch.
The rep/end playback-flow vectors are the acceptance test for building that.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Me Here 2026-05-30 23:54:08 -05:00
parent 25d0c57d79
commit 754ed1c22d
7 changed files with 673 additions and 0 deletions

207
docs/track-format.md Normal file
View file

@ -0,0 +1,207 @@
# PM track format — specification
This is the **single source of truth** for the metronome's track ("program" / "patch")
format. The grammar is currently implemented by hand in three places that have quietly
drifted:
- **Web**`src/engine.js` (`patchToSetup` / `laneStrToCfg` / `setupToPatch` / `laneCfgToStr`)
- **Firmware**`pico-cp/app.py` (`parse_program` / `_parse_lane` / `lane_to_str` / `_prog_str`)
- (`editor.html` inlines `engine.js` at build time)
`tests/fixtures/track-format.json` holds golden vectors that pin every feature to a single
expected meaning. `tests/run.mjs` runs both the JS and Python implementations against them
and reports divergences. **Any new implementation (e.g. a Rust engine) must pass the same
vectors** — that is what keeps "the same groove on the device and in the browser" true.
Status legend used below and in the fixtures:
- **stable** — implemented and (intended to be) identical on web + firmware.
- **divergence** — a real cross-implementation disagreement that exists *today*; the vector
encodes the spec's intended behavior and the runner flags the side that is wrong.
- **new** — defined here but not yet implemented anywhere (the playback-flow model below);
the vector is the acceptance test for building it.
---
## 1. Container: `programs.json`
The device reads set-lists from `/programs.json` (pushed by the editor over USB-MIDI SysEx,
or dragged onto the CIRCUITPY drive). Built-in set-lists are baked into firmware and are *not*
in this file.
```jsonc
{
"format": 2, // NEW. absent ⇒ treat as format 1 (legacy)
"setlists": [
{
"title": "My set",
"description": "optional",
"onEnd": "stop", // NEW: stop | nextList | loop (default: stop)
"defaultEnd": "next", // NEW: items without their own end= inherit this
"programs": [
{ "name": "Intro", "prog": "t88;b8;kick:4=X.x.;end=next" },
{ "name": "Groove", "prog": "t88;kick:4;snare:4=.X.X" }
]
}
]
}
```
**Legacy (format 1):** `{ "setlists":[...] }` with no `format`, or the older flat
`{ "programs":[...] }` (a single list). Loaders MUST still accept both. Migration to format 2:
set `format: 2`; a previously "continuous" set-list gets `onEnd` per the author's intent and
`defaultEnd: "next"` (or `end=next` on each item).
---
## 2. Patch string grammar
A patch is `;`-separated tokens. **Rule that prevents token collisions:** a token containing
`:` is a **lane**; every other token is a **keyword directive** matched against the reserved
set `{ v1, t, vol, cd, b, tr, rmp, rep, end }`. Unknown tokens are ignored (see §6).
```ebnf
patch = [ "v1" ";" ] directive *( ";" directive ) ;
directive = tempo | volume | countin | bars | trainer | ramp | rep | end | lane ;
tempo = "t" int ; (* beats per minute, clamped to 5..300 *)
volume = "vol" int ; (* 0..100, host-optional — device ignores *)
countin = "cd" int ; (* count-in seconds, host-optional *)
bars = "b" int ; (* cycle length in bars; drives Continue/rep *)
trainer = "tr" int "/" int ; (* playBars "/" muteBars (gap trainer) *)
ramp = "rmp" int "/" signed "/" int ; (* startBpm "/" amount "/" everyBars *)
rep = "rep" int ; (* NEW: cycles before end fires; default 1 *)
end = "end" "=" ( "stop" | "next" | signed ) ; (* NEW: see §3 *)
lane = sound ":" groups [ "/" sub [ "s" ] ] [ euclid ] [ "=" pattern ]
[ "@" signed ] [ "~" ] [ "!" ] ;
sound = name | int ; (* int = GM percussion note number alias *)
groups = int *( "+" int ) ; (* "4" or "2+2+3" → beats per bar *)
sub = int ; (* subdivision; trailing "s" = swing *)
euclid = "(" int [ "," int [ "," int ] ] ")" ; (* k [, n [, rot ]] — even distribution *)
pattern = *( "X" | "x" | "g" | "." | "-" | "_" ) ; (* per-step dynamics *)
```
### Lane semantics
- **groups**`2+2+3` means a 7-beat bar grouped 2+2+3; the first step of each group is
accented by default. `beatsPerBar = sum(groups)`.
- **sub** — steps per beat (`/2` eighths, `/3` triplets, `/4` sixteenths). `steps = beats*sub`.
- **swing**`/2s` lays the off-beat on the last triplet (≈ 2/3).
- **pattern** — one char per step: `X`=accent (level 2), `x`/`1`=normal (1), `g`=ghost (3),
`.`/`-`/`_`/anything else = rest (0). Short patterns are right-padded with rests to `steps`.
With no pattern, the first step of each beat is accented and the rest are normal.
- **euclid** `(k,n,rot)** — `k` hits spread as evenly as possible over `n` steps (rotated by
`rot`), first hit accented. Replaces an explicit `=pattern`.
- **`@<db>`** — per-lane gain in dB. (Host-optional: applied in the browser; see §6.)
- **`~`** — polymeter: this lane keeps its own bar length independent of the master lane.
- **`!`** — lane present but silenced/disabled.
### Top-level directives
- **`t`** tempo, clamped to `[5, 300]`.
- **`b<n>`** the segment's **cycle length** in bars — used for Continue/`rep` accounting and
song auto-advance. Absent ⇒ the cycle is one bar of the **master lane** (the first lane).
- **`rmp<start>/<amt>/<every>`** tempo ramp: every `<every>` bars, change tempo by `<amt>`
(may be negative), starting from `<start>`.
- **`tr<play>/<mute>`** gap trainer: play `<play>` bars, silence `<mute>` bars, repeat.
- **`vol`, `cd`** host-optional (the device has its own volume and no count-in).
---
## 3. Playback flow (NEW)
Replaces the old **global** `Continue` toggle with **per-track** behavior.
**Default (no `end=` token) — loop forever**, exactly like a metronome. Manual advance
(joystick / footswitch) always moves to the next track. "Vamp until cue" is therefore the
default and needs no special token.
`end=` exists only to make flow **automatic** after a finite number of cycles:
| Token | Meaning |
|------------------|---------------------------------------------------------------------|
| *(absent)* | Loop forever. Manual advance → next track. |
| `end=stop` | After `rep` cycles, stop. |
| `end=next` | After `rep` cycles, auto-advance one track. (Sugar for `end=+1`.) |
| `end=<±N>` | **Relative goto**: after `rep` cycles, jump `N` tracks (e.g. `end=-2` repeats a section, D.S.). |
| `rep=<N>` | Cycles to play before `end` fires. Default `1`. Only meaningful with `end`. |
A **cycle** = `b<bars>` bars if `b` is present, else one bar of the master lane.
**Normalization** (what the golden vectors compare): `end` normalizes to `"stop"`, or an
integer offset (`next` ⇒ `1`); absent ⇒ `null`. `rep` ⇒ the integer (defaults to `1` when
`end` is present), else `null`.
### Resolution & boundaries
- **Manual override always wins.** `end=` is only the hands-off behavior.
- **Seam is gapless.** An automatic `next`/goto does **not** re-trigger count-in and inserts
no gap (honors the existing `_seam_t` seam). Count-in (`cd`) fires only on manual/initial
start. Tempo jumps to the destination track's `t` at the seam.
- **Goto past the last item** (or `end=next` on the last item) triggers the set-list's
`onEnd` policy: `stop` | `nextList` | `loop`.
- **Goto before the first item** clamps to the first item.
- **`defaultEnd`** on a set-list is inherited by any item whose patch has no `end=`.
---
## 4. Canonical form & round-tripping
Serializing a parsed patch MUST be **idempotent**: `serialize(parse(serialize(parse(x))))`
equals `serialize(parse(x))`. Host-optional fields that a host does not act on
(`vol`, `cd`, `@db`) MUST still survive the round-trip rather than being dropped — silent
data loss is the failure mode this spec exists to prevent.
The runner checks **semantic equality** (the normalized structure — what actually plays)
across implementations, plus per-implementation idempotency. It does **not** require the two
serializers to emit byte-identical strings (they legitimately differ on the optional `v1`
prefix and on whether a default pattern is written out).
---
## 5. Normalized structure (the comparison target)
Each implementation's adapter parses a patch and emits this neutral shape; vectors store the
expected value in `norm`:
```jsonc
{
"bpm": 120, "bars": 0, "volume": null, "countMs": 0,
"ramp": null, // or { "start": 80, "amt": 4, "every": 4 }
"trainer": null, // or { "play": 2, "mute": 2 }
"rep": null, // NEW
"end": null, // NEW: "stop" | <int> | null
"lanes": [
{ "sound": "kick", "groups": [4], "sub": 1, "swing": false,
"poly": false, "mute": false, "gainDb": 0, "levels": [2,1,1,1] }
]
}
```
`levels` is the resolved per-step dynamics array (0 rest / 1 normal / 2 accent / 3 ghost) —
the real audible payload, and the most important thing two implementations must agree on.
---
## 6. Known divergences (today) — the work list
Surfaced by the runner; each has a `divergence` vector:
1. **Euclid `(k,n,rot)`** — parsed by `engine.js`, **not** by firmware (`_parse_lane` can't
strip `(...)`, so it falls back to a plain unaccented bar). → Implement on device.
2. **`vol` / `cd`** — round-tripped by `engine.js`, **dropped** by firmware. → Firmware should
preserve them on serialize even though it ignores them at runtime.
3. **Unknown sound name**`engine.js` falls back to `beep`; firmware keeps the raw name.
→ Spec: unknown sound ⇒ `beep` everywhere.
4. **Tempo clamp** — firmware clamps `t` to `[5,300]`; `engine.js` does not. → Clamp everywhere.
5. **Empty patch** — firmware injects a default `beep:4` lane; `engine.js` returns zero lanes
(the editor guards emptiness separately). → Spec: ≥1 lane; empty ⇒ `beep:4`.
6. **`@db` gain** — parsed on both sides but **applied** only in the browser (runtime, not a
parse divergence; not a fixture failure — listed for completeness).
The capability/version handshake (the device already replies with its firmware version over
SysEx) should gate features so the editor can warn when a track uses something the connected
firmware is too old to play, instead of letting it degrade silently.

2
tests/.gitignore vendored Normal file
View file

@ -0,0 +1,2 @@
__pycache__/
*.pyc

40
tests/README.md Normal file
View file

@ -0,0 +1,40 @@
# Track-format conformance tests
Golden-vector suite that pins the track ("program"/"patch") format to a single meaning and
checks that both implementations agree:
- **web**`src/engine.js`
- **firmware**`pico-cp/app.py`
The spec is `docs/track-format.md`. Any new implementation (e.g. a Rust engine) must pass the
same vectors — that is what keeps "the same groove on the device and in the browser" true.
## Run
```sh
node tests/run.mjs # table of pass / known-divergence / FAIL per case
node tests/run.mjs -v # also print expected-vs-actual diffs for unexpected failures
```
Exit code is non-zero on any **unexpected** failure or round-trip (idempotency) break, so it
works as a CI gate.
## Layout
- `fixtures/track-format.json` — the vectors. Each has `in` (a patch), `norm` (expected
normalized meaning, see spec §5), a `status`, and optional `expectFail` listing impls known
to differ today.
- `adapters/js_adapter.mjs` — loads the real `src/engine.js` grammar (no copy) and normalizes.
- `adapters/py_adapter.py` — extracts the real `pico-cp/app.py` grammar functions via `ast`
(no copy) and normalizes.
- `run.mjs` — runs every vector through both adapters and reports.
## Reading the result
- `✓ pass` — implementation matches the spec for that vector.
- `· known` — a divergence/feature listed in `expectFail`; expected, not a failure.
- `✗ FAIL` — an **unexpected** mismatch (a regression). Investigate.
- `★ fixed` — an impl listed in `expectFail` now passes; remove it from `expectFail`.
When you fix a divergence in code, delete that impl from the case's `expectFail`. When you
implement the `new` playback-flow tokens (`rep` / `end`), those cases flip to `pass`.

View file

@ -0,0 +1,51 @@
// JS adapter: loads the REAL grammar out of src/engine.js (no copy) and emits the
// neutral normalized structure defined in docs/track-format.md §5.
//
// engine.js has no top-level side effects (only const/function declarations) and references
// `window` only inside ensureAudio(), which we never call — so it loads cleanly in Node.
import { readFileSync } from "node:fs";
import { fileURLToPath } from "node:url";
import { dirname, join } from "node:path";
const here = dirname(fileURLToPath(import.meta.url));
const src = readFileSync(join(here, "..", "..", "src", "engine.js"), "utf8");
// Expose the pure codec functions from engine.js's top-level scope.
globalThis.window = globalThis.window || {};
const api = new Function(src + "\n;return { patchToSetup, setupToPatch };")();
const groupsArr = (s) => String(s).split(/[^0-9]+/).filter(Boolean).map(Number);
export function normalize(patch) {
const s = api.patchToSetup(patch);
return {
bpm: s.bpm,
bars: s.bars || 0,
volume: s.volume == null ? null : s.volume,
countMs: s.countMs || 0,
ramp: s.ramp && s.ramp.on ? { start: s.ramp.startBpm, amt: s.ramp.amount, every: s.ramp.everyBars } : null,
trainer: s.trainer && s.trainer.on ? { play: s.trainer.playBars, mute: s.trainer.muteBars } : null,
rep: s.rep == null ? null : s.rep, // not parsed yet → undefined → null
end: s.end == null ? null : s.end, // not parsed yet → undefined → null
lanes: (s.lanes || []).map((c) => ({
sound: c.sound,
groups: groupsArr(c.groupsStr),
sub: c.stepsPerBeat || 1,
swing: !!c.swing,
poly: !!c.poly,
mute: c.enabled === false,
gainDb: c.gainDb || 0,
levels: (c.beatsOn || []).map((v) => v | 0),
})),
};
}
// serialize(parse(x)) — used for the idempotency check.
export function canonical(patch) {
return api.setupToPatch(api.patchToSetup(patch));
}
// CLI: `node js_adapter.mjs '<patch>'` prints normalized JSON.
if (process.argv[1] && fileURLToPath(import.meta.url) === process.argv[1]) {
process.stdout.write(JSON.stringify(normalize(process.argv[2] ?? "")));
}

View file

@ -0,0 +1,93 @@
#!/usr/bin/env python3
"""Python adapter: extracts the REAL grammar functions out of pico-cp/app.py (no copy) and
emits the neutral normalized structure from docs/track-format.md §5.
app.py can't be imported directly (it does hardware init at module load), so we pull just the
pure codec nodes by name via `ast` and exec them in isolation. This tests the actual firmware
code and survives edits as long as the function/constant names persist.
"""
import ast
import json
import os
import sys
APP = os.path.join(os.path.dirname(__file__), "..", "..", "pico-cp", "app.py")
WANT = {"PAT", "PRIO", "PAT_CH", "parse_program", "_parse_lane", "lane_to_str"}
with open(APP) as f:
_src = f.read()
_segs = []
for node in ast.parse(_src).body:
name = None
if isinstance(node, ast.Assign) and len(node.targets) == 1 and isinstance(node.targets[0], ast.Name):
name = node.targets[0].id
elif isinstance(node, ast.FunctionDef):
name = node.name
if name in WANT:
_segs.append(ast.get_source_segment(_src, node))
NS = {}
exec("\n".join(_segs), NS)
def _prog_str(bpm, lanes, bars, ramp, trainer):
# Mirrors app.py App._prog_str (app.py:577) using the real lane_to_str.
parts = ["t" + str(bpm)]
if bars:
parts.append("b" + str(bars))
if ramp:
parts.append("rmp%d/%d/%d" % (ramp.get("start", bpm), ramp["amt"], ramp["every"]))
if trainer:
parts.append("tr%d/%d" % (trainer["play"], trainer["mute"]))
for L in lanes:
parts.append(NS["lane_to_str"](L))
return ";".join(parts)
def _gain_db(g):
if not g:
return 0
try:
return float(str(g).lstrip("@"))
except ValueError:
return 0
def normalize(patch):
bpm, lanes, bars, ramp, trainer = NS["parse_program"](patch)
return {
"bpm": bpm,
"bars": bars,
"volume": None, # device has no master-volume token
"countMs": 0, # device has no count-in
"ramp": {"start": ramp.get("start", bpm), "amt": ramp["amt"], "every": ramp["every"]} if ramp else None,
"trainer": {"play": trainer["play"], "mute": trainer["mute"]} if trainer else None,
"rep": None, # not parsed yet
"end": None, # not parsed yet
"lanes": [
{
"sound": L["sound"],
"groups": list(L.get("groups", [4])),
"sub": L["sub"],
"swing": bool(L["swing"]),
"poly": bool(L["poly"]),
"mute": bool(L["mute"]),
"gainDb": _gain_db(L.get("gain", "")),
"levels": [int(v) for v in L["levels"]],
}
for L in lanes
],
}
def canonical(patch):
return _prog_str(*NS["parse_program"](patch))
if __name__ == "__main__":
patch = sys.argv[1] if len(sys.argv) > 1 else ""
# Re-parse the canonical form too, so the runner can check idempotency in one call.
c1 = canonical(patch)
c2 = canonical(c1)
sys.stdout.write(json.dumps({"norm": normalize(patch), "canonical": c1, "idempotent": c1 == c2}))

183
tests/fixtures/track-format.json vendored Normal file
View file

@ -0,0 +1,183 @@
{
"_comment": "Golden vectors for the PM track format. See docs/track-format.md. Each case: parse `in`, normalize, compare to `norm`. status: stable|divergence|new. expectFail lists impls (js/py) known to mismatch today; an UNlisted mismatch is a regression, and a listed PASS means the gap was fixed. NOTE: stable cases use EXPLICIT =patterns on every lane, because lanes with no =pattern currently produce DIFFERENT defaults on web vs device (see the 'default-pattern' case + docs §6).",
"cases": [
{
"id": "minimal",
"status": "stable",
"in": "t120;kick:4=Xxxx",
"norm": { "bpm": 120, "bars": 0, "volume": null, "countMs": 0, "ramp": null, "trainer": null, "rep": null, "end": null,
"lanes": [ { "sound": "kick", "groups": [4], "sub": 1, "swing": false, "poly": false, "mute": false, "gainDb": 0, "levels": [2,1,1,1] } ] }
},
{
"id": "backbeat",
"status": "stable",
"in": "t120;kick:4=Xxxx;snare:4=.x.x;hatClosed:4/2=X.x.x.x.",
"norm": { "bpm": 120, "bars": 0, "volume": null, "countMs": 0, "ramp": null, "trainer": null, "rep": null, "end": null,
"lanes": [
{ "sound": "kick", "groups": [4], "sub": 1, "swing": false, "poly": false, "mute": false, "gainDb": 0, "levels": [2,1,1,1] },
{ "sound": "snare", "groups": [4], "sub": 1, "swing": false, "poly": false, "mute": false, "gainDb": 0, "levels": [0,1,0,1] },
{ "sound": "hatClosed", "groups": [4], "sub": 2, "swing": false, "poly": false, "mute": false, "gainDb": 0, "levels": [2,0,1,0,1,0,1,0] }
] }
},
{
"id": "odd-meter-2+2+3",
"status": "stable",
"in": "t130;kick:2+2+3=x..x..x;hatClosed:2+2+3/2=X.x.X.x.X.x.x.",
"norm": { "bpm": 130, "bars": 0, "volume": null, "countMs": 0, "ramp": null, "trainer": null, "rep": null, "end": null,
"lanes": [
{ "sound": "kick", "groups": [2,2,3], "sub": 1, "swing": false, "poly": false, "mute": false, "gainDb": 0, "levels": [1,0,0,1,0,0,1] },
{ "sound": "hatClosed", "groups": [2,2,3], "sub": 2, "swing": false, "poly": false, "mute": false, "gainDb": 0, "levels": [2,0,1,0,2,0,1,0,2,0,1,0,1,0] }
] }
},
{
"id": "swing",
"status": "stable",
"in": "t150;ride:4/2s=X.x.x.x.;kick:4=X..x",
"norm": { "bpm": 150, "bars": 0, "volume": null, "countMs": 0, "ramp": null, "trainer": null, "rep": null, "end": null,
"lanes": [
{ "sound": "ride", "groups": [4], "sub": 2, "swing": true, "poly": false, "mute": false, "gainDb": 0, "levels": [2,0,1,0,1,0,1,0] },
{ "sound": "kick", "groups": [4], "sub": 1, "swing": false, "poly": false, "mute": false, "gainDb": 0, "levels": [2,0,0,1] }
] }
},
{
"id": "ghost-notes",
"status": "stable",
"in": "t92;snare:4/3=..gg.gX.gg.g",
"norm": { "bpm": 92, "bars": 0, "volume": null, "countMs": 0, "ramp": null, "trainer": null, "rep": null, "end": null,
"lanes": [ { "sound": "snare", "groups": [4], "sub": 3, "swing": false, "poly": false, "mute": false, "gainDb": 0, "levels": [0,0,3,3,0,3,2,0,3,3,0,3] } ] }
},
{
"id": "polymeter",
"status": "stable",
"in": "t100;kick:4=Xxxx;claves:5=Xxxxx~",
"norm": { "bpm": 100, "bars": 0, "volume": null, "countMs": 0, "ramp": null, "trainer": null, "rep": null, "end": null,
"lanes": [
{ "sound": "kick", "groups": [4], "sub": 1, "swing": false, "poly": false, "mute": false, "gainDb": 0, "levels": [2,1,1,1] },
{ "sound": "claves", "groups": [5], "sub": 1, "swing": false, "poly": true, "mute": false, "gainDb": 0, "levels": [2,1,1,1,1] }
] }
},
{
"id": "disabled-lane",
"status": "stable",
"in": "t120;kick:4=Xxxx;hatClosed:4=Xxxx!",
"norm": { "bpm": 120, "bars": 0, "volume": null, "countMs": 0, "ramp": null, "trainer": null, "rep": null, "end": null,
"lanes": [
{ "sound": "kick", "groups": [4], "sub": 1, "swing": false, "poly": false, "mute": false, "gainDb": 0, "levels": [2,1,1,1] },
{ "sound": "hatClosed", "groups": [4], "sub": 1, "swing": false, "poly": false, "mute": true, "gainDb": 0, "levels": [2,1,1,1] }
] }
},
{
"id": "ramp",
"status": "stable",
"in": "t80;woodblock:4=Xxxx;rmp80/4/4",
"norm": { "bpm": 80, "bars": 0, "volume": null, "countMs": 0, "ramp": { "start": 80, "amt": 4, "every": 4 }, "trainer": null, "rep": null, "end": null,
"lanes": [ { "sound": "woodblock", "groups": [4], "sub": 1, "swing": false, "poly": false, "mute": false, "gainDb": 0, "levels": [2,1,1,1] } ] }
},
{
"id": "gap-trainer",
"status": "stable",
"in": "t100;kick:4=Xxxx;tr2/2",
"norm": { "bpm": 100, "bars": 0, "volume": null, "countMs": 0, "ramp": null, "trainer": { "play": 2, "mute": 2 }, "rep": null, "end": null,
"lanes": [ { "sound": "kick", "groups": [4], "sub": 1, "swing": false, "poly": false, "mute": false, "gainDb": 0, "levels": [2,1,1,1] } ] }
},
{
"id": "segment-bars",
"status": "stable",
"in": "t88;b8;kick:4=X.x.",
"norm": { "bpm": 88, "bars": 8, "volume": null, "countMs": 0, "ramp": null, "trainer": null, "rep": null, "end": null,
"lanes": [ { "sound": "kick", "groups": [4], "sub": 1, "swing": false, "poly": false, "mute": false, "gainDb": 0, "levels": [2,0,1,0] } ] }
},
{
"id": "gm-note-number-alias",
"status": "stable",
"in": "t120;36:4=Xxxx",
"note": "GM note 36 = kick. engine.js maps it; firmware keeps the numeric sound but plays it via SOUND_GM. Names should resolve the same.",
"expectFail": ["py"],
"norm": { "bpm": 120, "bars": 0, "volume": null, "countMs": 0, "ramp": null, "trainer": null, "rep": null, "end": null,
"lanes": [ { "sound": "kick", "groups": [4], "sub": 1, "swing": false, "poly": false, "mute": false, "gainDb": 0, "levels": [2,1,1,1] } ] }
},
{
"id": "default-pattern",
"status": "divergence",
"expectFail": ["py"],
"note": "DESIGN DECISION PENDING. A lane with no =pattern produces different defaults: web plays every subdivision (off-beats normal) and accents every beat -> [2,1,2,1...]; device rests off-beats -> [2,0,1,0...]. This affects shipped presets (e.g. 'Four-on-the-floor' hatClosed:4/2). Vector pins the web behavior as provisional reference; revise once the rule is chosen.",
"in": "t120;hatClosed:4/2",
"norm": { "bpm": 120, "bars": 0, "volume": null, "countMs": 0, "ramp": null, "trainer": null, "rep": null, "end": null,
"lanes": [ { "sound": "hatClosed", "groups": [4], "sub": 2, "swing": false, "poly": false, "mute": false, "gainDb": 0, "levels": [2,1,2,1,2,1,2,1] } ] }
},
{
"id": "euclid",
"status": "divergence",
"expectFail": ["py"],
"note": "Euclid (k,n) parsed by engine.js; firmware falls back to a plain bar. Spec = engine behavior.",
"in": "t120;kick:4(3,8)",
"norm": { "bpm": 120, "bars": 0, "volume": null, "countMs": 0, "ramp": null, "trainer": null, "rep": null, "end": null,
"lanes": [ { "sound": "kick", "groups": [4], "sub": 2, "swing": false, "poly": false, "mute": false, "gainDb": 0, "levels": [2,0,0,1,0,0,1,0] } ] }
},
{
"id": "vol-and-countin",
"status": "divergence",
"expectFail": ["py"],
"note": "engine.js round-trips vol/cd; firmware drops them. Spec = preserve them.",
"in": "t120;vol80;cd2;kick:4=Xxxx",
"norm": { "bpm": 120, "bars": 0, "volume": 0.8, "countMs": 2000, "ramp": null, "trainer": null, "rep": null, "end": null,
"lanes": [ { "sound": "kick", "groups": [4], "sub": 1, "swing": false, "poly": false, "mute": false, "gainDb": 0, "levels": [2,1,1,1] } ] }
},
{
"id": "unknown-sound",
"status": "divergence",
"expectFail": ["py"],
"note": "engine.js maps unknown sounds to beep; firmware keeps the raw name. Spec = beep.",
"in": "t120;blorp:4=Xxxx",
"norm": { "bpm": 120, "bars": 0, "volume": null, "countMs": 0, "ramp": null, "trainer": null, "rep": null, "end": null,
"lanes": [ { "sound": "beep", "groups": [4], "sub": 1, "swing": false, "poly": false, "mute": false, "gainDb": 0, "levels": [2,1,1,1] } ] }
},
{
"id": "tempo-clamp-high",
"status": "divergence",
"expectFail": ["js"],
"note": "Firmware clamps t to [5,300]; engine.js does not. Spec = clamp everywhere.",
"in": "t999;kick:4=Xxxx",
"norm": { "bpm": 300, "bars": 0, "volume": null, "countMs": 0, "ramp": null, "trainer": null, "rep": null, "end": null,
"lanes": [ { "sound": "kick", "groups": [4], "sub": 1, "swing": false, "poly": false, "mute": false, "gainDb": 0, "levels": [2,1,1,1] } ] }
},
{
"id": "empty-defaults-to-beep",
"status": "divergence",
"expectFail": ["js"],
"note": "Firmware injects a default beep:4; engine.js returns no lanes (editor guards emptiness). Spec = beep:4.",
"in": "",
"norm": { "bpm": 120, "bars": 0, "volume": null, "countMs": 0, "ramp": null, "trainer": null, "rep": null, "end": null,
"lanes": [ { "sound": "beep", "groups": [4], "sub": 1, "swing": false, "poly": false, "mute": false, "gainDb": 0, "levels": [2,1,1,1] } ] }
},
{
"id": "end-stop",
"status": "new",
"expectFail": ["js", "py"],
"note": "Per-track playback flow — not yet implemented anywhere.",
"in": "t120;kick:4=Xxxx;end=stop",
"norm": { "bpm": 120, "bars": 0, "volume": null, "countMs": 0, "ramp": null, "trainer": null, "rep": 1, "end": "stop",
"lanes": [ { "sound": "kick", "groups": [4], "sub": 1, "swing": false, "poly": false, "mute": false, "gainDb": 0, "levels": [2,1,1,1] } ] }
},
{
"id": "rep-then-next",
"status": "new",
"expectFail": ["js", "py"],
"note": "Play 4 cycles then auto-advance. next ⇒ +1.",
"in": "t120;kick:4=Xxxx;rep=4;end=next",
"norm": { "bpm": 120, "bars": 0, "volume": null, "countMs": 0, "ramp": null, "trainer": null, "rep": 4, "end": 1,
"lanes": [ { "sound": "kick", "groups": [4], "sub": 1, "swing": false, "poly": false, "mute": false, "gainDb": 0, "levels": [2,1,1,1] } ] }
},
{
"id": "relative-goto-back-two",
"status": "new",
"expectFail": ["js", "py"],
"note": "Relative goto (D.S.): after 1 cycle, jump back two tracks.",
"in": "t120;kick:4=Xxxx;end=-2",
"norm": { "bpm": 120, "bars": 0, "volume": null, "countMs": 0, "ramp": null, "trainer": null, "rep": 1, "end": -2,
"lanes": [ { "sound": "kick", "groups": [4], "sub": 1, "swing": false, "poly": false, "mute": false, "gainDb": 0, "levels": [2,1,1,1] } ] }
}
]
}

97
tests/run.mjs Normal file
View file

@ -0,0 +1,97 @@
#!/usr/bin/env node
// Conformance runner for the PM track format.
// node tests/run.mjs run all golden vectors against engine.js + app.py
// node tests/run.mjs -v also print the expected/actual diff for every failure
//
// For each vector it parses `in` with both implementations, normalizes, and compares to
// `norm`. A mismatch on an impl listed in the vector's `expectFail` is "known" (expected);
// any other mismatch is a regression and fails the run. See docs/track-format.md.
import { readFileSync } from "node:fs";
import { fileURLToPath } from "node:url";
import { dirname, join } from "node:path";
import { execFileSync } from "node:child_process";
import * as js from "./adapters/js_adapter.mjs";
const here = dirname(fileURLToPath(import.meta.url));
const verbose = process.argv.includes("-v");
const fixtures = JSON.parse(readFileSync(join(here, "fixtures", "track-format.json"), "utf8"));
const pyAdapter = join(here, "adapters", "py_adapter.py");
// stable, key-sorted JSON so deep-equality is a string compare.
const stable = (o) => JSON.stringify(o, (k, v) =>
v && typeof v === "object" && !Array.isArray(v)
? Object.fromEntries(Object.keys(v).sort().map((kk) => [kk, v[kk]]))
: v);
function runJs(patch) {
try {
return { norm: js.normalize(patch), canonical: js.canonical(patch), error: null };
} catch (e) {
return { norm: null, canonical: null, error: String(e.message || e) };
}
}
function runPy(patch) {
try {
const out = execFileSync("python3", [pyAdapter, patch], { encoding: "utf8" });
return { ...JSON.parse(out), error: null };
} catch (e) {
const msg = (e.stderr || "").toString().trim().split("\n").pop() || e.message;
return { norm: null, canonical: null, error: msg };
}
}
const want = stable; // alias
let regressions = 0, fixedNowCount = 0, nonIdempotent = 0;
const rows = [];
function jsIdempotent(patch) {
try { const c1 = js.canonical(patch); return c1 === js.canonical(c1); } catch { return false; }
}
for (const c of fixtures.cases) {
const expected = want(c.norm);
const expectFail = new Set(c.expectFail || []);
const r = { id: c.id, status: c.status };
for (const [impl, res] of [["js", runJs(c.in)], ["py", runPy(c.in)]]) {
// serialize(parse(x)) must be stable under re-parsing (no silent drift on round-trip).
const idem = impl === "js" ? jsIdempotent(c.in) : res.idempotent !== false;
if (res.error == null && !idem) { nonIdempotent++; console.log(` ! non-idempotent serialize: ${c.id} [${impl}]`); }
const ok = res.error == null && want(res.norm) === expected;
const known = expectFail.has(impl);
let mark;
if (ok && !known) mark = "PASS";
else if (ok && known) { mark = "FIXED"; fixedNowCount++; } // listed as failing but now passes
else if (!ok && known) mark = "known"; // expected divergence/not-built
else { mark = "FAIL"; regressions++; } // unexpected → regression
r[impl] = mark;
r[impl + "_res"] = res;
if (mark === "FAIL" && verbose) {
console.log(`\n--- ${c.id} [${impl}] expected vs actual ---`);
console.log("expected:", expected);
console.log("actual: ", res.error ? "ERROR " + res.error : want(res.norm));
}
}
rows.push(r);
}
// ---- report ----
const pad = (s, n) => String(s).padEnd(n);
console.log("\n PM track-format conformance\n");
console.log(" " + pad("case", 26) + pad("status", 13) + pad("engine.js", 11) + "app.py");
console.log(" " + "-".repeat(58));
const glyph = { PASS: "✓ pass", known: "· known", FAIL: "✗ FAIL", FIXED: "★ fixed" };
for (const r of rows) {
console.log(" " + pad(r.id, 26) + pad(r.status, 13) + pad(glyph[r.js], 11) + glyph[r.py]);
}
const counts = rows.reduce((a, r) => { a[r.js] = (a[r.js] || 0) + 1; a[r.py] = (a[r.py] || 0) + 1; return a; }, {});
console.log("\n " + Object.entries(counts).map(([k, v]) => `${glyph[k] || k}: ${v}`).join(" "));
if (fixedNowCount) console.log(`\n ${fixedNowCount} case(s) marked expectFail now PASS — update the fixture (remove them from expectFail).`);
if (nonIdempotent) console.log(` ${nonIdempotent} non-idempotent serialize(s) above.`);
if (regressions || nonIdempotent) {
console.log(`\n${regressions} unexpected failure(s), ${nonIdempotent} round-trip issue(s). Run with -v for diffs.\n`);
process.exit(1);
}
console.log("\n ✓ no unexpected failures; serialize round-trips are stable.\n");