Drive ~15 min read

at the endYou know which motions are learned and which stay geometry, what each policy beat in simulation and by how much, why the reach policy was not shipped, and how the whole-arm cameraman policies are built on a MuJoCo twin of this exact arm.

Learned motion

Two kinds of motion live in strands-arm. Scripted kinematics (shot orbit|dolly|reveal, nod, the re-base in watch) are plain geometry and stay that way: nothing to learn. Learned policies earn their place where a controller has to cope with latency, servo lag, a moving target and occlusions at once. They are trained in simulation on a laptop, exported to ONNX, and run through the same motion guard as every other command. Nothing about the guard changes for a policy: 30 deg/s ramp, +-60 deg head window, torque off after idle, one JSON line per command in motion.log.

What ships

policy	robot	trained in	result	repo
track	leader head, pan (servo 5) + tilt (servo 6)	analytic 2-DOF sim, PPO, 1 min on an M4 Pro CPU	sim eval 6.11 deg RMSE vs 8.89 for the shipped P step; live dry run done	cagataydev/strands-arm-track-2026-09-06 (private)
reach	SO-101 follower, 6 joints	MuJoCo so101_new_calib.xml, PPO, 27 min	best run 94 % success / 1.5 cm vs 96 % / 0.4 cm for a Jacobian-P controller: not published, IK wins	none
gaze_stabilize	leader head vs its IMU while the base turns	analytic 2-DOF sim + base model, PPO, 2 min	sim eval 3.63 deg RMSE vs 4.50 for drive-command feed-forward	cagataydev/strands-arm-gaze_stabilize-2026-09-06 (private)
frame	the whole leader arm, all six servos	MuJoCo twin of this arm + printed head, identified servo model, PPO, 31 M steps in 20 min	sim eval: subject visible 69 % vs 39 % for the head-only P step, centre error 0.20 vs 0.66, 20 % of steps fully framed vs 1 %; shadow mode on the live robot done, never written to the arm	cagataydev/strands-arm-frame-2026-09-06 (private)
frame_mobile	the same, riding the omni base	MuJoCo twin + base velocity feed-forward, PPO, 16 M steps	sim eval: visible 68 % vs 29 % head-only P, vs 58 % for the still-base frame policy on the same moving base	cagataydev/strands-arm-frame_mobile-2026-09-06 (private)
frame_dolly	the arm plus the omni base as its dolly (6 servos + vx, vy, wz)	MuJoCo twin, base follows the command through a ramp, size spec = a dolly offset, PPO, 32 M steps in 18 min	sim eval: visible 87 %, fully framed 70 %, size error 0.045, vs 72 % / 30 % / 0.080 for frame_mobile parked in the same env and 34 % for the head-only P step	cagataydev/strands-arm-frame_dolly-2026-09-06 (private)
follow_gripper	the leader arm filming the SO-101 follower's gripper while it replays real LeRobot episodes	MuJoCo twin with the follower in the scene, follower joint state in the observation, PPO, 5.7 M steps in 11 min (stopped on plateau)	sim eval: gripper visible 85 % vs 25 % head-only P, 18 % for frame on the same env; occluded by the follower's own links 10 % of frustum time (the geometric ceiling); jerk 0.84 deg/step after a smoothness curriculum (1.27 flat)	cagataydev/strands-arm-follow_gripper-2026-09-06 (private)

Left: the policy. Right: the P step watch uses. Same target script, same latency and servo lag.

track

The head sees a face as a FOMO box at QVGA. watch turns the box centre into a pan/tilt increment with a proportional gain (head_tool.track_step, 0.05 deg per pixel, 5 deg per step, 5 Hz). It works, and it overshoots when the detection is 100..300 ms old: the same stale error gets acted on twice. The policy sees the error, the head's own angle and velocity, its last action and the previous error, and outputs the increment at 10 Hz. Trained with 1..3 steps of observation latency, 0..1 step of action delay, servo lag 0.05..0.3 s, half-FOV 25..40 deg (the lens FOV is unsourced, so the policy cannot depend on it), three target motions and occlusion bursts.

Numbers, 4 seeded scripts x 64 envs x 300 steps, true bearing RMSE in degrees:

controller	rmse	visible
still	19.11	0.878
P step at 10 Hz	10.15	0.892
P step at 5 Hz (deployed)	8.89	0.906
policy	6.11	0.907

The sim holds each detection for 100..200 ms because the Nicla node was measured publishing at 148 ms while the policy runs at 10 Hz. Five runs landed between 5.97 and 6.52; a 1500-update run did not beat a 400-update one. The smoothness terms in the reward trade against bearing error once the policy's noise collapses.

Run it:

strands-arm watch face --policy rl/policies/track/policy.onnx --dry      # reads only, prints what it would do
strands-arm watch face --policy rl/policies/track/policy.onnx            # through the guard, needs the bench gate

--dry reads the Nicla node's /detect stream and the dashboard's /api/state; it never opens the USB or serial port, so it runs beside a live dashboard. The obs/act contract, the onnxruntime snippet and the training config are on the model card, and rl/policies/track/contract.json in the repo.

gaze_stabilize

The base will turn under the head (docs/base.md). Without a camera target the head has two things to go on: the drive command it can see, and the Nicla IMU riding on the head. Dead-reckoning from the command alone is a strong controller (4.50 deg RMSE in the sim) because an integrated gyro drifts over 30 s. The policy fuses both: 3.63 deg over the same 4 seeded scripts, 19 % lower, without touching the +-60 deg window. Slip, yaw-rate gain and IMU bias ranges are assumptions until the base exists; the card says so.

The twin, and the whole arm as a cameraman

The 2-DOF policies above treat the head as two angles. The robot's actual job is to place a camera: the leader arm is the tripod, jib and dolly of the Nicla on its wrist, and the interesting motions use all six servos at once. That needed a simulation of this arm, not of the SO-101 in general.

rl/sim/build_mjcf.py builds it from the vendor MJCF plus our printed head: the SO-101 leader with the gripper stripped, the yoke and tilt cradle from hardware/cad at their real offsets, the Nicla lens as a MuJoCo camera with the firmware's field of view (33 deg horizontal half-angle, so 51.9 deg vertical on 4:3), the fold pose the arm rests in, and the calibrated servo signs from kine.py. rl/sim/servo.py is the servo, fitted from motion.log: the guard's 20 Hz staircase, the Feetech speed profile, a first-order lag with a range of time constants, a dead band. rl/sim/validation.py puts the sim camera next to a real snapshot at the same servo readings.

frame is the first whole-arm task. A head-sized subject wanders 0.3..1.5 m from the base; the policy gets what the Nicla would report (box centre, box size, visible flag, ToF, one frame late, with pixel noise), the six servo readings, its last command and a shot spec: where in the frame the subject should sit and how big. It commands six servo deltas per 100 ms, each within the head tool's 5 deg cap; the servo limits are the calibrated EEPROM windows and the guard's +-60 deg head window. Contacts between the arm and itself or the floor cost reward, so do steps near a limit. The sim never integrates dynamics: the servo model is the dynamics and MuJoCo does kinematics, camera, occlusion rays and contacts, which is why 128 arms run at 28 000 steps per second on a laptop.

Top row is watch, arm frozen at the fold; bottom is the policy, same subject. Left is the Nicla's view (green = the shot asked for, yellow = the detection), right is third person. Over 256 seeded episodes the policy keeps the subject visible 69 % of the time against 39 % for the head alone, centres it 3.3x better (0.20 vs 0.66 frame units), holds the full shot (centre and size) 20 % of the time against 1 %, and touches itself 24x less (0.04 vs 0.85 contacts per step). Freeze the arm under the same network and contacts jump above 2 per step: the arm is what keeps the head off the shoulder. Two runs got here: the first (visible 72 %, centre 0.27) moved 8.9 deg of servo per step, a jitter no real servo should be asked for; pricing action and its rate of change 5x higher gave the shipped one at 5.4 deg per step with better centring and fewer contacts for 3 points of visibility. One gap is structural: distance. The size error (0.18) barely beats doing nothing (0.19); this policy does not dolly, and the next section says why an arm cannot. The policy has been run in shadow mode against the live robot, real pose, real detections, real ToF, nothing written: with no face in view it asks for the unfolding motion it learned to search with.

frame_mobile is the same task with the arm riding the omni base, whose drive loop random-walks inside the base guard's caps; the policy is told the commanded base velocity, the one signal the robot always has exactly. On the moving base the still-base frame policy keeps the subject visible 58 % of the time; frame_mobile, fed the base velocity, 68 %, against 29 % for the head-only P step. Centring is close between the two (0.32 vs 0.34), so what the feed-forward buys is mostly not losing the subject when the base swings; it moves more (8.7 deg per step) because the base makes it.

The arm is a jib, not a dolly. Three frame variants (v3, v4 and a static-subject diagnostic) tried to make the arm learn the size half of the shot spec, and every one of them ignored it. The numbers say why (rl/sim/sensitivity.py, a numerical Jacobian over 200 visible poses): one servo degree moves the subject about 0.04 of the frame sideways and 0.002 in size, a 17.6 : 1 ratio. Changing the subject size by 0.3 costs about 150 degrees of coordinated motion while keeping it in view; under any action price the policy is right to skip it. The SO-101's 0.3 m of usable reach frames, it does not dolly. The base does. frame_dolly hands the policy the base: nine actions, six servo deltas plus (vx, vy, wz) mapped onto the drive guard caps, the size spec a dolly offset the base can always reach. It keeps the subject visible 87 % of the time and fully framed 70 %, with a size error of 0.045, against 72 % / 30 % / 0.080 for frame_mobile dropped into the same env with the base parked. Driving is what bought the size; the arm keeps the centre.

follow_gripper is the first policy whose subject is the other arm. A stock SO-101 follower stands on a deck 0.9 m in front of the leader and replays recorded trajectories (LeRobot so101_molmoact_e2e, the action column at 30 Hz, resampled with a random time scale, start and pan mirror, under a joint speed cap); the leader has to keep the fingertip framed. Two things are new. The subject is a real mesh: the follower's own wrist and jaws hide the fingertip from some angles, so the eval separates in frustum from visible and reports the difference as occlusion (10 % of frustum time for the policy, the same as the head-only P step). That 10 % is geometry, not a training gap: rl/sim/vantage.py casts a ray from 600 collision-free camera positions the leader can reach to the fingertip through the replayed episodes and finds the fingertip unoccluded 82..86 % of the time from every one of them, flat across height and side. All reachable viewpoints sit in a 0.4 m ball 0.9 m from the follower; the baseline is too short to look around the wrist. Seeing more means moving the base, or a second eye. And the follower is on the same servo bus, so its joint positions and velocities are in the observation for free; the policy learns to lead a target whose next move it can read. That is where most of the gain comes from: the head-only P step sees the gripper 25 % of the time, frame dropped into this env 18 %, this policy 85 %, and the same network with the arm frozen still 70 %. The arm buys centring (0.28 vs 1.12 frame units) and the remaining visibility. It moves 6 deg of servo per step with 0.84 deg of jerk; the first run, with smoothness priced flat from step one, landed at the same visibility with 1.27 deg of jerk. Ramping the action and jerk prices in only after the skill is learned (the microduck rule) cut jerk by a third and improved size and framing for free. Three assumptions ride on this release and are listed in the card: the deck placement, the LeRobot-to-URDF joint direction of the recording (the sample's state column is half dropouts, and only one sign set keeps the recorded arm above the table), and a gripper class for the Nicla's detector, which knows faces and persons.

None of these run on the arm until the bench says so: a line under RL.md ## RL-2 NEED, head first, arm joints second, always through the guard.

reach, and what RL is not for

reach is the standard sim demo: the 6-joint SO-101 in MuJoCo, end effector to a random reachable point, smooth joints. Three PPO runs, the last 8.3 M steps in 27 minutes on this Mac: 94 % of targets within 2 cm, 1.5 cm final error. A ten-line Jacobian pseudo-inverse controller with the same joint-velocity cap reaches 96 % at 0.4 cm. So reach is not shipped as a controller: when the target position and the kinematics are known, IK is the right tool and a policy only approximates it. What the run proves is the path: MuJoCo model with the XML's position actuators and joint limits, PPO, ONNX export with a contract, the same evaluation harness and the same "beat the hand-written baseline or stay in rl/runs/" rule that shipped track and gaze_stabilize.

Manipulation on strands-arm will not come from reward shaping. It will come from imitation learning on teleop datasets (record --lerobot, LeKiwi-shaped episodes, ACT or SmolVLA), where the leader is the teacher and the Nicla on the follower is the camera. RL here is for reflexes: tracking, stabilizing, keeping a horizon.

Layout

rl/sim/ the MuJoCo twin (build_mjcf.py, servo.py, validation.py), rl/envs/ the tasks, rl/ppo.py PPO in pure torch, rl/train.py, rl/export_onnx.py, rl/eval.py and rl/eval_frame.py (fixed seeded scripts, every number above), rl/render.py / rl/render_frame.py (MP4 + GIF), rl/make_model_card.py (card + private upload). Released policies: rl/policies/<name>/. Runner on the robot: strands_arm/policy.py. Journal with every iteration's numbers: RL.md.

shot: a negative result that became six lines

rl/envs/shot_env.py re-frames mid-take: every 4..8 s the director draws a new (ex, ey, size*) and the spec the policy sees glides there at cinematic speed (0.03 frame units and 0.01 size per 0.1 s step). A policy trained on it for 900 updates scored framed 0.21 (0.24 while holding, 0.16 while the ramp moved). The shipped frame policy, which never saw a cut, scored 0.25 / 0.28 / 0.19 on the same env with the same jerk (0.49 deg/step). The spec is an input, so a ramped spec is just a target that moves slowly, and the reactive policy already follows it. shot therefore is not a policy: it is FramePolicy.direct(ex, ey, size) in the runtime, which glides the spec at the same slew. The shot-trained run stays in rl/runs/shot/ unpublished.

What did not beat the shipped frame policy

Every sim2real trick in rl/PLAYBOOK.md (borrowed from Pollen's microduck_rl) was run as an A/B against the shipped frame (same PPO config, same 2500 updates, same seeds for evaluation). Numbers are framed fraction / jerk in deg per step; the shipped policy scores 0.17 / 0.45 on the same env.

lever	result	why
backlash + servo randomisation	shipped in follow_gripper v2	the real bus has both; the policy stopped chattering at the goal
smoothness curriculum (prices ramp in after skill)	win on follow_gripper (jerk 1.27 → 0.84), loss on frame (0.13 / 0.45, 6× the self-contacts)	cheap early actions teach big swings through the collision zone when the subject is free; fine when the subject is structured
slewed target in the env (shot)	0.21 vs 0.25 for the shipped policy on the same env	the spec is an observation, so a gliding target is the runtime's job, not the policy's
symmetry augmentation	dropped before training	the SO-101 is not mirror-symmetric: arm plane 18 mm off the pan axis, lens 18 mm off the roll axis. Mirroring would train in a 0.05–0.1 centre bias (tests/test_rl_frame.py::test_arm_is_not_mirror_symmetric)
asymmetric critic (value net sees the true subject)	0.16 / 0.53, null	the observation already carries the subject; the critic was never the bottleneck

The honest reading: the shipped frame sits at the ceiling of its task (a free random-walk subject at 10 Hz behind a detector delay). Better cameramen come from structured subjects, the follower, a walking person, and from the real bench, not from another PPO knob. RL.md has each run's numbers.