Four rungs, each cheaper than the next
Testing
The device is slow to reach and easy to crash, so the cheap tests have to be good enough to keep you off it. This is the ladder we actually run, and the bugs each rung caught.
1 · The framework host test (seconds, x86)
./test.sh # → tools/test_port.sh: ASan + UBSan, two instances, every parameter
Two instances, names, set/get on every parameter, option select and nudge, wheel and
Q-Link stepping, popup open/close, note → audio, chunk round-trip. Caught: a
parameter that was an enum in the JSON but missing from the engine’s text table (the control flipped,
the sound did nothing), and a buffer overrun in the wrapper’s display read-back.
2 · Per-parameter round-trips you write yourself
The framework test only exercises parameter 0 of each kind. Ours sets every option of every
enum through the real wrapper and checks value and display text — 43 checks.
Caught: labels beginning with a digit being read as an index
(atoi("1/16") == 1), which made two switches snap back while the sound changed.
set option k of enum E → read back value and effGetParamDisplay → compare with the label
3 · DSP tests with numbers, not ears
Rendered checks per feature: bit-exactness when a stage is at zero, a null test against bypass, THD ordering, level within a band, no NaN/Inf, determinism across runs, no sample step above the signal’s own rise. Caught: an ADAA guard 1000× too small — in a decaying signal the sample difference fell to 1e-6 and float rounding turned into 411 audible jumps in half a second. The fix was a constant; finding it took three wrong attempts at the output stage.
4 · Lifetime tests (offline, but about the host)
Build the plugin’s worker code as a shared object, dlopen it, run the plugin lifecycle,
dlclose it, idle. Caught: the shape of the crash that took the device
down twice — a thread still parked in poll() when the library is unloaded dies with
SIGSEGV. The test asserts both directions: with the stop-and-join it survives, without it it must die,
or the test proves nothing.
5 · The page, headless
node web/pagetest.mjs web/index.html
The real page script runs against a fake DOM and a recording canvas: the drawing must not throw in either layout, every control must register a hit region, the geometry must fit and must not overlap (all pages × both viewports), knob drags and switch holds must post the right values, and a swipe must turn the page. Caught: a function and a variable with the same name, an unbounded hit list, switches without keys, a lost helper, adjacent hit boxes sharing an exact edge, and a negative arc radius when a timestamp went backwards.
6 · The device, by hand or by agent
Insert, play, save, reload: state must survive. Then the things no test on a host can see — knob feel, Q-Link pickup, screen legibility, whether a switch behaves like a switch.
Drive it: the remote addin
jacob-sabella/mpc-addin-remote
(MIT) is an addin: a small shared library MPC loads through LD_PRELOAD, so it
runs inside the MPC process. It captures the real screen over DRM, writes touches into the
touchscreen's evdev node, presses hardware buttons, opens an ALSA MIDI port and reports device
status — as HTTP routes and as an MCP server on /mcp.
ssh -N -L 6720:127.0.0.1:6720 root@<ip> # it listens on the device only
curl -s localhost:6720/info # screen size, pixel format, touch device
curl -s "localhost:6720/tap?x=640&y=300" # a real touch, seen by MPC as a finger
curl -s "localhost:6720/screen.png" -o shot.png
curl -s localhost:6720/mcp -d '{"jsonrpc":"2.0","id":1,"method":"tools/list"}'
The MCP side is what an agent should use: screenshot (with a labelled pixel grid),
tap / drag / scroll, wait_for_screen,
play_notes and send_midi, device_status. The workspace wraps
it in vst/device_test.sh (shot, tap, drag,
call).
How it combines with a companion screen: the addin sees and touches the
device, the companion answers questions about the plugin. So the loop is
screenshot → tap the control where it is actually drawn → read /state to prove the
parameter moved. That is the difference between "the switch looks wrong" and "the switch is at
cy=658, which is outside the touchable area, and tapping it changes nothing". Both of those cost
real time here; only the second one is a fix.
Treat it as a development tool. It runs as root inside MPC and has no login;
it binds to loopback for that reason, and bind=0.0.0.0 gives anyone on the LAN your
screen, your touches and a MIDI port into MPC. Never install it on a device you gig with.
The rule behind the ladder
Every bug that reached the device cost a deploy and, twice, a crash. Everything that was caught on rung 1–5 cost a rebuild. If a class of bug has bitten you once, add a rung — the test is cheaper than the next occurrence, and it is the only thing that keeps a regression from coming back.
Bench, in one table
| Stage | What it answers |
|---|---|
| idle | does the plugin cost anything when silent? |
| audio in | the honest baseline |
| Q-Link sweep | cost while parameters move (smoothing, ramps) |
| release tail | decay paths and denormals |
| trigger fire | one-shots, measured separately so they cannot decide the verdict |
| background threads | a listener or worker that should be asleep |