Rust port Stage 1: track-format codec crate (passes the golden vectors)

A third implementation of the track DSL alongside engine.js and app.py, validated
against the same tests/fixtures/track-format.json:

- rust/track-format/: pure parse()/serialize() codec (std + alloc for now; no_std is
  a later refinement). Ports the app.py/engine.js semantics exactly — grouping,
  subdivisions, swing, ghost, polymeter, euclid, GM note-number aliases, unknown->beep,
  default groove (group-start accents), tempo clamp, empty->beep, and the playback-flow
  tokens (rep/end/relative-goto). Carries vol/cd too, so it's the most spec-complete
  of the three.
- tests/conformance.rs: the Rust adapter — reads the shared fixtures, asserts each
  case's normalized form (number-tolerant deep-equal) + serialize idempotency.
- rust/Containerfile + run.sh: Rust toolchain in a container (mirrors hardware/eda/),
  with the thumbv8m.main-none-eabihf target for the eventual RP2350 firmware. Never
  on the host.

Verified: ./rust/run.sh -> cargo test -> conformance + idempotent both pass.
docs/rust-port.md Stage 1 marked done.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

docs/rust-port.md

@@ -20,7 +20,13 @@ Add a Rust toolchain image (mirroring `hardware/eda/`): a `Containerfile` with `
 `thumbv8m.main-none-eabihf` target (RP2350 is Cortex-M33), `flip-link`, `probe-rs`, `elf2uf2`.
 Driven by a `run.sh` like the EDA one. **Never on the host.**
 
-### Stage 1 — `track-format` crate  ← the concrete first PR
+### Stage 1 — `track-format` crate  ✅ DONE (`rust/track-format/`)
+Implemented and **passing**: `./rust/run.sh` builds the container and runs `cargo test`, which
+validates the crate against `tests/fixtures/track-format.json` (conformance + idempotency). The
+Rust codec agrees with `engine.js` and `app.py` on every vector — and carries `vol`/`cd`, so it's
+the most spec-complete of the three. Original scope below.
+
+#### (original) Stage 1 — `track-format` crate  ← the concrete first PR
 A pure, `no_std`-compatible crate: `parse(&str) -> Track` and `serialize(&Track) -> String`,
 plus a `normalize()` that emits the neutral structure from `docs/track-format.md` §5. Then a
 `cargo test` that reads `tests/fixtures/track-format.json` and asserts each case's `norm` and

rust/Containerfile

@@ -0,0 +1,8 @@
+# Rust toolchain for the PM track-format crate (Stage 1 of the Rust port).
+# Host-tested codec for now; the RP2350 firmware target is added for later stages.
+FROM docker.io/library/rust:1-slim
+
+# Cortex-M33 target for the eventual RP2350 firmware. Harmless for the host tests.
+RUN rustup target add thumbv8m.main-none-eabihf
+
+WORKDIR /work/rust

rust/README.md

@@ -0,0 +1,13 @@
+# Rust port — `track-format` crate (Stage 1)
+
+Pure parse/serialize codec for the track DSL, validated against the shared golden
+vectors (`tests/fixtures/track-format.json`) — the third implementation alongside
+`engine.js` and `app.py`. See `docs/rust-port.md` for the staged plan.
+
+All tooling runs in a container (per the develop-in-container rule):
+
+```sh
+./rust/run.sh            # cargo test — runs the conformance + idempotency suite
+./rust/run.sh cargo build
+./rust/run.sh bash       # interactive shell in the crate
+```

rust/run.sh

@@ -0,0 +1,25 @@
+#!/usr/bin/env bash
+# Build (first time) and run the PM Rust container with the repo mounted.
+#
+#   ./run.sh                 # cargo test (runs the golden-vector conformance suite)
+#   ./run.sh cargo build
+#   ./run.sh bash            # interactive shell in rust/track-format/
+#
+# Override the runtime with RUNTIME=docker ./run.sh ...
+set -euo pipefail
+
+IMG="pm-rust:1"
+RUST_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"   # rust/
+REPO_DIR="$(cd "$RUST_DIR/.." && pwd)"                     # repo root
+RUNTIME="${RUNTIME:-podman}"
+
+if ! "$RUNTIME" image inspect "$IMG" >/dev/null 2>&1; then
+  echo ">> building $IMG (first run, a few minutes)…" >&2
+  "$RUNTIME" build -t "$IMG" "$RUST_DIR"
+fi
+
+# Mount the whole repo so the crate's tests can read tests/fixtures/ ; land in the crate dir.
+flags=(--rm -v "$REPO_DIR":/work:Z -w /work/rust/track-format)
+[[ -t 0 ]] && flags+=(-it)
+if [[ $# -eq 0 ]]; then set -- cargo test; fi   # default command (multi-word, so set positional args)
+exec "$RUNTIME" run "${flags[@]}" "$IMG" "$@"

rust/track-format/.gitignore

@@ -0,0 +1,2 @@
+target/
+Cargo.lock

rust/track-format/Cargo.toml

@@ -0,0 +1,10 @@
+[package]
+name = "track-format"
+version = "0.1.0"
+edition = "2021"
+description = "PM track-format codec — Stage 1 of the Rust port; validated against the shared golden vectors."
+
+[dependencies]
+
+[dev-dependencies]
+serde_json = "1"

rust/track-format/src/lib.rs

@@ -0,0 +1,384 @@
+//! PM track-format codec — Stage 1 of the Rust port (see `docs/rust-port.md`).
+//!
+//! Pure parse/serialize for the track DSL, validated against
+//! `tests/fixtures/track-format.json` — the same golden vectors `engine.js` and
+//! `app.py` pass. This is the third implementation; it must agree with them.
+//!
+//! Uses `std` (String/Vec/BTreeSet) for now. A `no_std` + `alloc` version is a later
+//! refinement (swap the collections for `alloc` equivalents); the logic is unchanged.
+
+use std::collections::BTreeSet;
+
+#[derive(Debug, Clone)]
+pub struct Lane {
+    pub sound: String,
+    pub groups: Vec<u32>,
+    pub sub: u32,
+    pub swing: bool,
+    pub poly: bool,
+    pub mute: bool,
+    pub gain_db: f64,
+    pub levels: Vec<u8>,
+}
+
+#[derive(Debug, Clone)]
+pub struct Ramp {
+    pub start: i64,
+    pub amt: i64,
+    pub every: i64,
+}
+
+#[derive(Debug, Clone)]
+pub struct Trainer {
+    pub play: i64,
+    pub mute: i64,
+}
+
+#[derive(Debug, Clone, PartialEq)]
+pub enum End {
+    Stop,
+    Goto(i64), // relative track offset; +1 == "next"
+}
+
+#[derive(Debug, Clone)]
+pub struct Track {
+    pub bpm: i64,
+    pub bars: i64,
+    pub volume: Option<f64>,
+    pub count_ms: i64,
+    pub ramp: Option<Ramp>,
+    pub trainer: Option<Trainer>,
+    pub rep: Option<i64>,
+    pub end: Option<End>,
+    pub lanes: Vec<Lane>,
+}
+
+/// General-MIDI percussion note number -> voice name (matches GM_NUM in engine.js / app.py).
+fn gm_num(n: u32) -> Option<&'static str> {
+    Some(match n {
+        35 | 36 => "kick",
+        37 => "rim",
+        38 | 40 => "snare",
+        39 => "clap",
+        41 | 43 => "tomLow",
+        42 | 44 => "hatClosed",
+        45 | 47 => "tomMid",
+        46 => "hatOpen",
+        48 | 50 => "tomHigh",
+        49 => "crash",
+        51 | 53 => "ride",
+        54 => "tambourine",
+        56 => "cowbell",
+        75 => "claves",
+        76 | 77 => "woodblock",
+        _ => return None,
+    })
+}
+
+/// Known voices (matches SOUND_GM keys in app.py); anything else falls back to `beep`.
+fn known_sound(s: &str) -> bool {
+    matches!(
+        s,
+        "kick" | "kick808" | "kick909" | "snare" | "snare808" | "snare909"
+            | "clap" | "clap808" | "clap909" | "rim" | "hatClosed" | "hat808" | "hat909"
+            | "hatOpen" | "openHat808" | "ride" | "ride909" | "crash" | "crash909"
+            | "tomLow" | "tom808" | "tomMid" | "tomHigh" | "tambourine" | "cowbell"
+            | "cowbell808" | "woodblock" | "jamblock" | "claves" | "beep"
+    )
+}
+
+/// Euclidean distribution: k hits over n steps, rotated by rot (matches euclid() in engine.js).
+fn euclid(k: i64, n: i64, rot: i64) -> Vec<u8> {
+    let n = n.max(1);
+    let k = k.clamp(0, n);
+    let rot = ((rot % n) + n) % n;
+    (0..n)
+        .map(|i| {
+            let j = (i + rot) % n;
+            if (j * k) % n < k { 1 } else { 0 }
+        })
+        .collect()
+}
+
+fn pat(c: char) -> u8 {
+    match c {
+        'X' => 2,
+        'x' | '1' => 1,
+        'g' => 3,
+        _ => 0,
+    }
+}
+
+fn parse_lane(tok: &str) -> Lane {
+    let poly = tok.contains('~');
+    let mute = tok.contains('!');
+    let cleaned: String = tok.chars().filter(|&c| c != '~' && c != '!').collect();
+
+    let mut gain_db = 0.0;
+    let body = if let Some(at) = cleaned.find('@') {
+        gain_db = cleaned[at + 1..].parse::<f64>().unwrap_or(0.0);
+        cleaned[..at].to_string()
+    } else {
+        cleaned
+    };
+
+    let (sound0, rest0) = match body.find(':') {
+        Some(i) => (body[..i].to_string(), body[i + 1..].to_string()),
+        None => (body.clone(), String::new()),
+    };
+    let mut sound = sound0;
+    if let Ok(n) = sound.parse::<u32>() {
+        if let Some(name) = gm_num(n) {
+            sound = name.to_string();
+        }
+    }
+
+    let mut rest = rest0;
+
+    // euclid (k[,n[,rot]]) shorthand — pulled before the =/ splits
+    let mut euc: Option<Vec<i64>> = None;
+    if let Some(lp) = rest.find('(') {
+        if let Some(rel) = rest[lp..].find(')') {
+            let rp = lp + rel;
+            let nums: Vec<i64> = rest[lp + 1..rp]
+                .split(',')
+                .filter_map(|x| x.trim().parse::<i64>().ok())
+                .collect();
+            rest = format!("{}{}", &rest[..lp], &rest[rp + 1..]);
+            if !nums.is_empty() {
+                euc = Some(nums);
+            }
+        }
+    }
+
+    let mut pattern: Option<String> = None;
+    if let Some(eq) = rest.find('=') {
+        pattern = Some(rest[eq + 1..].to_string());
+        rest = rest[..eq].to_string();
+    }
+
+    let mut sub: u32 = 1;
+    let mut swing = false;
+    if let Some(sl) = rest.find('/') {
+        let sd = &rest[sl + 1..];
+        swing = sd.ends_with('s');
+        sub = sd.trim_end_matches('s').parse::<u32>().unwrap_or(1);
+        rest = rest[..sl].to_string();
+    }
+
+    let mut groups: Vec<u32> = rest.split('+').filter_map(|g| g.parse::<u32>().ok()).collect();
+    if groups.is_empty() {
+        groups = vec![4];
+    }
+    let beats: u32 = groups.iter().sum();
+    let mut starts = BTreeSet::new();
+    let mut acc = 0u32;
+    for &g in &groups {
+        starts.insert(acc);
+        acc += g;
+    }
+
+    let levels: Vec<u8> = if let Some(e) = euc {
+        // euclidean: k hits over n steps, first hit accented
+        let k = e[0];
+        let n = if e.len() > 1 { e[1] } else { (beats * sub) as i64 };
+        let rot = if e.len() > 2 { e[2] } else { 0 };
+        if e.len() > 1 {
+            if n % (beats as i64) == 0 {
+                sub = (n / beats as i64) as u32;
+            } else {
+                groups = vec![n as u32];
+                sub = 1;
+            }
+        }
+        let mut first = true;
+        euclid(k, n, rot)
+            .into_iter()
+            .map(|h| {
+                if h != 0 {
+                    let v = if first { 2 } else { 1 };
+                    first = false;
+                    v
+                } else {
+                    0
+                }
+            })
+            .collect()
+    } else if let Some(p) = pattern {
+        let steps = (beats * sub) as usize;
+        let mut lv: Vec<u8> = p.chars().map(pat).collect();
+        if lv.len() < steps {
+            lv.resize(steps, 0);
+        }
+        lv
+    } else {
+        // default: every subdivision sounds at normal, accent only on group starts
+        let steps = beats * sub;
+        (0..steps)
+            .map(|i| {
+                if i % sub == 0 {
+                    if starts.contains(&(i / sub)) { 2 } else { 1 }
+                } else {
+                    1
+                }
+            })
+            .collect()
+    };
+
+    if !known_sound(&sound) {
+        sound = "beep".to_string();
+    }
+    Lane { sound, groups, sub, swing, poly, mute, gain_db, levels }
+}
+
+pub fn parse(s: &str) -> Track {
+    let mut bpm = 120i64;
+    let mut bars = 0i64;
+    let mut ramp = None;
+    let mut trainer = None;
+    let mut rep: Option<i64> = None;
+    let mut end: Option<End> = None;
+    let mut volume: Option<f64> = None;
+    let mut count_ms = 0i64;
+    let mut lanes = Vec::new();
+
+    for raw in s.trim().split(';') {
+        let tok = raw.trim();
+        if tok.is_empty() {
+            continue;
+        }
+        if tok.len() > 1 && tok.starts_with('t') && tok[1..].bytes().all(|b| b.is_ascii_digit()) {
+            bpm = tok[1..].parse().unwrap_or(120);
+            continue;
+        }
+        if tok.len() > 1 && tok.starts_with('b') && tok[1..].bytes().all(|b| b.is_ascii_digit()) {
+            bars = tok[1..].parse().unwrap_or(0);
+            continue;
+        }
+        if tok.starts_with("rmp") {
+            let p: Vec<&str> = tok[3..].split('/').collect();
+            if p.len() == 3 {
+                if let (Ok(a), Ok(b), Ok(c)) =
+                    (p[0].parse::<i64>(), p[1].parse::<i64>(), p[2].parse::<i64>())
+                {
+                    ramp = Some(Ramp { start: a, amt: b, every: c.max(1) });
+                }
+            }
+            continue;
+        }
+        if tok.starts_with("tr") && tok.contains('/') && !tok.contains(':') {
+            let p: Vec<&str> = tok[2..].split('/').collect();
+            if p.len() == 2 {
+                if let (Ok(a), Ok(b)) = (p[0].parse::<i64>(), p[1].parse::<i64>()) {
+                    trainer = Some(Trainer { play: a.max(0), mute: b.max(0) });
+                }
+            }
+            continue;
+        }
+        if tok.starts_with("vol") {
+            volume = Some(tok[3..].parse::<i64>().unwrap_or(0) as f64 / 100.0);
+            continue;
+        }
+        if tok.starts_with("cd") {
+            count_ms = tok[2..].parse::<i64>().unwrap_or(0) * 1000;
+            continue;
+        }
+        if tok.starts_with("rep=") {
+            rep = Some(tok[4..].parse::<i64>().unwrap_or(1).max(1));
+            continue;
+        }
+        if tok.starts_with("end=") {
+            let v = &tok[4..];
+            end = Some(if v == "stop" {
+                End::Stop
+            } else if v == "next" {
+                End::Goto(1)
+            } else {
+                End::Goto(v.parse::<i64>().unwrap_or(0))
+            });
+            continue;
+        }
+        if !tok.contains(':') {
+            continue;
+        }
+        lanes.push(parse_lane(tok));
+    }
+    if lanes.is_empty() {
+        lanes.push(parse_lane("beep:4"));
+    }
+    Track {
+        bpm: bpm.clamp(5, 300),
+        bars,
+        volume,
+        count_ms,
+        ramp,
+        trainer,
+        rep,
+        end,
+        lanes,
+    }
+}
+
+fn lane_to_str(l: &Lane) -> String {
+    let mut s = format!(
+        "{}:{}",
+        l.sound,
+        l.groups.iter().map(|g| g.to_string()).collect::<Vec<_>>().join("+")
+    );
+    if l.sub != 1 || l.swing {
+        s.push_str(&format!("/{}{}", l.sub, if l.swing { "s" } else { "" }));
+    }
+    s.push('=');
+    s.extend(l.levels.iter().map(|&v| match v {
+        2 => 'X',
+        1 => 'x',
+        3 => 'g',
+        _ => '.',
+    }));
+    if l.gain_db != 0.0 {
+        s.push_str(&format!("@{}", l.gain_db));
+    }
+    if l.poly {
+        s.push('~');
+    }
+    if l.mute {
+        s.push('!');
+    }
+    s
+}
+
+pub fn serialize(t: &Track) -> String {
+    let mut parts = vec![format!("t{}", t.bpm)];
+    if let Some(v) = t.volume {
+        parts.push(format!("vol{}", (v * 100.0).round() as i64));
+    }
+    if t.count_ms > 0 {
+        parts.push(format!("cd{}", t.count_ms / 1000));
+    }
+    if t.bars != 0 {
+        parts.push(format!("b{}", t.bars));
+    }
+    if let Some(r) = &t.ramp {
+        parts.push(format!("rmp{}/{}/{}", r.start, r.amt, r.every));
+    }
+    if let Some(tr) = &t.trainer {
+        parts.push(format!("tr{}/{}", tr.play, tr.mute));
+    }
+    for l in &t.lanes {
+        parts.push(lane_to_str(l));
+    }
+    if let Some(e) = &t.end {
+        if let Some(rep) = t.rep {
+            if rep > 1 {
+                parts.push(format!("rep={}", rep));
+            }
+        }
+        parts.push(match e {
+            End::Stop => "end=stop".to_string(),
+            End::Goto(1) => "end=next".to_string(),
+            End::Goto(n) if *n > 0 => format!("end=+{}", n),
+            End::Goto(n) => format!("end={}", n),
+        });
+    }
+    parts.join(";")
+}

rust/track-format/tests/conformance.rs

@@ -0,0 +1,75 @@
+//! The Rust adapter for the shared golden vectors — the third implementation alongside
+//! `tests/adapters/js_adapter.mjs` and `tests/adapters/py_adapter.py`. Reads the same
+//! `tests/fixtures/track-format.json` and asserts each case's normalized form + round-trip.
+
+use serde_json::{json, Value};
+use track_format::{parse, serialize, End, Track};
+
+/// Build the neutral normalized structure (docs/track-format.md §5) from a parsed Track.
+fn norm(t: &Track) -> Value {
+    json!({
+        "bpm": t.bpm,
+        "bars": t.bars,
+        "volume": t.volume,
+        "countMs": t.count_ms,
+        "ramp": t.ramp.as_ref().map(|r| json!({"start": r.start, "amt": r.amt, "every": r.every})),
+        "trainer": t.trainer.as_ref().map(|x| json!({"play": x.play, "mute": x.mute})),
+        "rep": match t.end { None => Value::Null, Some(_) => json!(t.rep.unwrap_or(1)) },
+        "end": match &t.end {
+            None => Value::Null,
+            Some(End::Stop) => json!("stop"),
+            Some(End::Goto(n)) => json!(n),
+        },
+        "lanes": t.lanes.iter().map(|l| json!({
+            "sound": l.sound, "groups": l.groups, "sub": l.sub, "swing": l.swing,
+            "poly": l.poly, "mute": l.mute, "gainDb": l.gain_db, "levels": l.levels
+        })).collect::<Vec<_>>(),
+    })
+}
+
+/// Deep-equal that treats all JSON numbers as f64 (so 0 == 0.0 across the int/float boundary).
+fn json_eq(a: &Value, b: &Value) -> bool {
+    match (a, b) {
+        (Value::Number(x), Value::Number(y)) => x.as_f64() == y.as_f64(),
+        (Value::Array(x), Value::Array(y)) => {
+            x.len() == y.len() && x.iter().zip(y).all(|(p, q)| json_eq(p, q))
+        }
+        (Value::Object(x), Value::Object(y)) => {
+            x.len() == y.len() && x.iter().all(|(k, v)| y.get(k).map_or(false, |w| json_eq(v, w)))
+        }
+        _ => a == b,
+    }
+}
+
+fn fixtures() -> Value {
+    let path = concat!(env!("CARGO_MANIFEST_DIR"), "/../../tests/fixtures/track-format.json");
+    let raw = std::fs::read_to_string(path).expect("read fixtures json");
+    serde_json::from_str(&raw).expect("parse fixtures json")
+}
+
+#[test]
+fn conformance() {
+    let doc = fixtures();
+    let mut fails = 0;
+    for case in doc["cases"].as_array().unwrap() {
+        let id = case["id"].as_str().unwrap_or("?");
+        let patch = case["in"].as_str().unwrap_or("");
+        let got = norm(&parse(patch));
+        if !json_eq(&case["norm"], &got) {
+            fails += 1;
+            eprintln!("FAIL {id}\n  expected: {}\n  got:      {got}", case["norm"]);
+        }
+    }
+    assert_eq!(fails, 0, "{fails} conformance mismatch(es)");
+}
+
+#[test]
+fn idempotent() {
+    let doc = fixtures();
+    for case in doc["cases"].as_array().unwrap() {
+        let patch = case["in"].as_str().unwrap_or("");
+        let c1 = serialize(&parse(patch));
+        let c2 = serialize(&parse(&c1));
+        assert_eq!(c1, c2, "non-idempotent serialize for {}", case["id"]);
+    }
+}