MPC VST · field notes

Letting a model drive the bench

Agents

A model writes a plugin quickly. What it cannot do, today, is see the device. That gap is where the time goes: coordinates guessed from a screenshot, taps that silently miss, and a crash nobody can explain because the device keeps no symbols.

What already exists

PieceWhat it gives you
addin remote (MIT) the live screen, touch injection, MIDI, and a JSON-RPC MCP endpoint on 127.0.0.1:6720 — screenshot, tap, drag, scroll, wait_for_screen, play_notes, device_status, list_files
addin commander the loaded plugins with their skins, MPC’s transport and parameter changes over HTTP/WS — no tapping required
addin filebrowser browse the device drives in a browser (deploy without scp acrobatics)
addin usb-audio the MPC as a USB audio interface — worth knowing before you assume “no loopback” forever
addin skipback records the main output all the time, saves the last 30–60 s
mpc-store.sh, Plugin Manager, MPC Link installing, updating, and getting files from a computer into the device
patches: drum-pad-layout, drive-exec, button-remap what a patch can change without touching a plugin at all

What a development agent actually needs

From one real port — a plugin built, tested, deployed, driven and twice crashed — the missing pieces are not “more tools”, they are named handles and evidence:

Say the name, not the coordinate

The skin already declares every widget and its bounds. A helper that parses the installed TUI.json can answer “where is the RESONANCE knob, what is its value” and tap it by name. MPC’s own screens (plugin list, dialogs) are fixed-pitch lists: record their geometry once, with a human watching, and replay it with assertions instead of re-deriving pixels every session.

Every action returns evidence

Before/after screenshot hash, the changed rectangle, the parameter value read back. “The tap landed and nothing happened” has to be a visible result, not a story the model tells itself.

Health and crash facts

MPC pid, whether the .so is still mapped, the /data/sentry tail, audio-worker CPU from /proc, xruns. That is what turned “it crashed twice” into a rule in the device page.

Safe staging

Snapshot and restore of MPC.settings, the plugin folder and the project; a single “remove and re-insert” verb; a watchdog that restarts acvs when a plugin took it down.

Proposal: a development bench as a plugin

A resident addin (plus a thin host CLI) that turns the loop into verbs: screen.describe, screen.act(name), param.set(instance, id, value), plugin.reset, device.health, crash.explain. It composes the remote and commander addins instead of reimplementing them, speaks JSON-RPC on 127.0.0.1 with the same names as its MCP tools, and every answer carries {ok, before, after, evidence[]}.

First release, read-only: describe a screen, act on one binding, report health and the plugin’s own diagnostics. Writes come second, behind snapshot/rollback.

Do not build these

When the agent is the one laying out the screen

Skin work is where an agent burns an afternoon fastest, because the feedback loop is the device. This week it did exactly that: switches that did not react were “fixed” five times — bigger art, padded hit boxes, a different widget — and every attempt needed a rebuild, a deploy and a human finger. The cause was one number that had been written down all along: a widget is drawn 86 px above its cy, so in a 1280×628 area nothing below art y ≈ 542 has a reachable touch box. The switches sat at cy=658. Every “fix” changed the thing that was not broken.

The loop that converges, in this order:

  1. Read the layout skill first (.opencode/skills/mpc-ui-builder). It carries the offsets, the canvas budget, and which widget to use for which kind of value.
  2. tools/skin_check.py "<skin>/Plugin Skins" — reports EDGE (box past 1280×628), TOUCH (two touch boxes overlapping), QLINK, OPTS per page, in about a second. gen_vst.py prints the same findings after every build. A skin with any finding is not deployed.
  3. studio.py preview for the picture, including the variant per stateful widget — that is the only way to see a pressed switch without touching the device.
  4. Only then build and deploy. A skin needs remove/re-insert; a .so needs an acvs restart. Removing and re-inserting a plugin does not reload a new library.

Two facts that cost the most, and neither is about taste: a toggle never fired on a 2-option enum on this OS build (enum_h did, immediately), and MPC acts on release, not on touch-down. If a control must show its state, it has to latch.

How to drive it today, without the bench

ssh -N -L 6720:127.0.0.1:6720 root@<ip>        # the addin listens on the device only
curl -s localhost:6720/info                    # {"width":1280,"height":800,"touch":"/dev/input/event0"}
# screenshot → tap → screenshot; assert with a hash or a region diff, never by hope
# and read /proc/<pid>/maps when a deploy “did nothing”

The proposal above was drafted by a model (DeepSeek V4.1 Flash) from this week’s session notes and then cut down to what the evidence supports. The “inference” markers are gone on purpose: everything left here was either measured or is labelled as not proven.