~5 min read

The learned loop

Two PPO policies were trained in MuJoCo on this arm's own geometry. Fomo carries them as ONNX and can run them on the real bus and the real eyes — today in shadow: the net watches, its deltas are drawn, no servo is written. The words that change that are the owner's, not mine.

What runs

The Nicla's FOMO detector (fomo_face_detection.tflite) reports boxes at the edge; the head's own lens is the only camera. Every 10 Hz tick the policy receives the servo positions from the bus and the largest box from the board, and answers with a delta per servo, clipped to ±5° per step.

policy	servos	observation	trained as
track	pan, tilt	box centre, visible, pan/tilt normalised to ±60°, their velocities, the last action	cagataydev/strands-arm-track-2026-09-06
frame	all six	six joints as sin/cos, box centre + size, visible, where it was last seen and how long ago, the last action	cagataydev/strands-arm-frame-2026-09-06

The observation builders are pinned to each policy's contract.json by tests, and the deltas go through the same guard as every other write — a policy is a caller, not an exception.

Stage 0 · shadow — today

fomo rl shadow frame --seconds 20          # or: the RL card in the cockpit's dock

The shadow reads the bus and the eyes, runs the net, logs one JSON row per tick to ~/.fomo/rl/, and reports: ticks, how often a face was in frame, the largest delta asked, how many ticks would have hit a hard limit — and written: false. On the bench with no face the frame net asks for at most 2.65° and touches no limit; the six bars in the cockpit show it thinking.

The dash's RL seat runs in-process (GET /api/rl/status, POST /api/rl/shadow, WS /ws/rl): the While a shadow runs, the cockpit's twin follows the net's ghost — the pose the head would be in if the policy were driving: it starts at the real pose, integrates the clipped deltas inside the hard windows, and holds while no face has been seen for a second, exactly the live rule. The metal is still; the badge under the twin says so. Face → Nicla FOMO → policy → motion closes in the sim first.

shadow paces itself to the bus reads, about 6 Hz on the metal — one observation per fresh position, which is also the ceiling for any live loop.

Stages 1 and 2 · the owner's words

A stage opens only when a line beginning OWNER: in archive/lanes/RL-LIVE.md carries the exact phrase. Documenting the phrase — as this page does — opens nothing; a test reads the real journal on every run to make sure. The stage-1 path is built and waiting:

fomo rl live track --seconds 20     # today: refused (403): head servos wait for 'BENCH OK FOR RL HEAD' …

Once the word is there, that command's first move is the sign check — pan +5° and back, tilt +5° and back, and each time the box must have moved the way the net itself expects (the env's error is tan(target − pos), so in the policy's frame a positive step always lowers it; a sign here maps that frame onto this head's servo directions, on position and action only — the image error is never flipped). A wrong sign centres the head, releases it, never starts the run, and names the value that fixes it. Every tick after, the net's deltas (clipped to ±5°) are integrated into a pending goal the way the training env's guard model does; once that goal is a servo deadband away (1°) it becomes one guard.goto on pan and tilt inside the ±60° window. The guard ramps, then waits for the servo to hold still, so a tick that writes costs 0.3–0.5 s — expect 2–5 Hz while the head is moving and the policy's own rate while it holds, which is inside the lag and latency the policy was trained with. The summary reports the measured rate and how many ticks wrote; 30 s at most, head centred and released after. On the bench simulation a face at ex 0.30 settles to 0.02 in 20 ticks with 8 writes — the 0.02 floor is one deadband at this field of view.

stage	phrase	what it allows	and still requires
1 · head	BENCH OK FOR RL HEAD	track on pan and tilt, ±60° about home	the head's direction signs not yet verified on this head — the first live step is one +5° pan, then the box must have moved the way the net expects
2 · arm	BENCH OK FOR RL ARM	frame on all six	rail ≥ 7 V — at 5.6 V the elbow cannot lift (guard rule 8)

Growing the detector

The policies were trained to follow a subject; the board today only knows faces. A new class — the follower's gripper, a fiducial, a cup — starts as stills from this lens:

fomo rl dataset capture gripper --n 40 --every 1 [--box X Y W H]
fomo rl dataset stats
fomo rl dataset export gripper             # Edge Impulse bounding_boxes.labels

Stills come through the dash (the board serves one stream client), near-duplicates are dropped, and every row in labels.jsonl keeps the six joints, the ToF and the detector's own boxes at that instant, so a training set can be filtered by pose later. Flash the new model to the board's ROMFS and the same policies follow the new thing.