scout / a small robot with a big curiosity
sees / thinks / drives / remembers#
scout is a Strands agent living on a Jetson AGX Thor that drives a FrodoBots Earth Rover Mini+. It looks through the rover's two cameras, talks through its speaker, listens on its mic, takes orders from a passkey-gated web cockpit, Telegram or your voice โ and records every drive as a LeRobot dataset with the agent's own reasoning aligned frame-by-frame.
a stylised stand-in, drawn from primitives โ the real Earth Rover Mini+ CAD is linked on the hardware page (not redistributed: no license on the source repo)
What is scout, in five bullets#
- One agent, many faces. The same Strands agent answers in the REPL, the web cockpit's Ask box, Telegram, a bidirectional voice session over the rover's own mic and speaker, and a slow background thinker that wakes every minute to do one useful thing. โ Personas
- Real vision, real safety. Camera tools return actual image blocks the model sees; every motion tool clamps to
[-1, 1], re-streams/controlframes (the firmware wants a continuous stream) and always auto-stops โ even on exceptions. โ Tools - A cockpit you can trust from anywhere. FastAPI + vanilla JS PWA, sealed behind WebAuthn passkeys and HTTPS, exposed through a Cloudflare Tunnel. Joystick, cameras, live agent stream, persona switches, dataset replay. โ Dashboard
- Every drive is data. A LeRobot v3 recorder captures 10 Hz video + 27-D state + 2-D action, with the agent's reasoning trace as an Embodied-Chain-of-Thought sidecar and YOLO detections aligned to the same frames. โ Datasets
- Fleet-aware. With
TINY_MCP=1the agents mount tiny.technology's MCP tools and can ask sibling robots (Reachy, Fomo, Qโฆ) for help โ allow-listed, self-invoke refused, one hop deep. โ Fleet
How it fits together#
flowchart LR
subgraph clients["๐ clients"]
PWA[๐ฑ cockpit PWA]; TG[โ Telegram]; MIC[๐ voice]; REPL[๐ง REPL / MCP]
end
subgraph edge["๐ก edge"]
CF[โ๏ธ cloudflared tunnel]; PK[๐ passkey session]
end
subgraph thor["๐ง Jetson AGX Thor ยท docker compose slim"]
DASH["dashboard :8080 TLS<br/>FastAPI ยท agent ยท replay ยท personas"]
AG[strands agent<br/>ROVER_ALL_TOOLS]
SDK["earth-rovers-sdk :8002<br/>headless Chromium ยท Agora"]
HUB["media hub :8090<br/>one drainer, N subscribers"]
YOLO[yolo detector]
THK[๐ข thinker]; TEL[telegram listener]; VOX[voice agent]
SUP["scout-supervisor<br/>(host, unix socket)"]
REC[(LeRobot v3 datasets<br/>+ ECoT + detections)]
MEM[(.memory SQLite)]
end
ROVER((๐ Earth Rover Mini+))
PWA -. HTTPS / WS .-> CF --> PK --> DASH
TG --> TEL --> AG; MIC --> VOX --> AG; REPL --> AG; DASH --> AG
AG <--> SDK; HUB --> SDK; YOLO --> HUB; VOX --> HUB
AG --> REC; YOLO --> REC; AG <--> MEM
DASH -. start/stop .-> SUP -. docker compose .-> THK & TEL & VOX
SDK <==>|WebRTC| ROVER
Ports and services come straight from docker-compose.slim.yml + docker-compose.slim.override.yml โ see Architecture.
Where to next#
-
Start
What you need, installing on the brain, docker compose slim, the
.envwalkthrough, first drive. -
Dashboard
Passkey gate, camera tiles, glass joystick, Ask stream, persona pills, dataset replay.
-
Perception
MediaHub fan-out, YOLO sidecars, the Locate-Anything "who to follow" idea.
-
Datasets & replay
LeRobot v3, ECoT traces, why episodes are sealed, how to repair an index.
-
Tools reference
Every
@tool, generated from the code at build time โ signatures and docstrings. -
Operations
Bring-up order, restart recipes, tunnel, watchdogs, what to check before a demo.