Skip to content

scout / a small robot with a big curiosity

sees / thinks / drives / remembers#

scout is a Strands agent living on a Jetson AGX Thor that drives a FrodoBots Earth Rover Mini+. It looks through the rover's two cameras, talks through its speaker, listens on its mic, takes orders from a passkey-gated web cockpit, Telegram or your voice โ€” and records every drive as a LeRobot dataset with the agent's own reasoning aligned frame-by-frame.

docs last commit license dataset strands

a stylised stand-in, drawn from primitives โ€” the real Earth Rover Mini+ CAD is linked on the hardware page (not redistributed: no license on the source repo)

What is scout, in five bullets#

  • One agent, many faces. The same Strands agent answers in the REPL, the web cockpit's Ask box, Telegram, a bidirectional voice session over the rover's own mic and speaker, and a slow background thinker that wakes every minute to do one useful thing. โ†’ Personas
  • Real vision, real safety. Camera tools return actual image blocks the model sees; every motion tool clamps to [-1, 1], re-streams /control frames (the firmware wants a continuous stream) and always auto-stops โ€” even on exceptions. โ†’ Tools
  • A cockpit you can trust from anywhere. FastAPI + vanilla JS PWA, sealed behind WebAuthn passkeys and HTTPS, exposed through a Cloudflare Tunnel. Joystick, cameras, live agent stream, persona switches, dataset replay. โ†’ Dashboard
  • Every drive is data. A LeRobot v3 recorder captures 10 Hz video + 27-D state + 2-D action, with the agent's reasoning trace as an Embodied-Chain-of-Thought sidecar and YOLO detections aligned to the same frames. โ†’ Datasets
  • Fleet-aware. With TINY_MCP=1 the agents mount tiny.technology's MCP tools and can ask sibling robots (Reachy, Fomo, Qโ€ฆ) for help โ€” allow-listed, self-invoke refused, one hop deep. โ†’ Fleet

How it fits together#

flowchart LR
    subgraph clients["๐ŸŒ clients"]
        PWA[๐Ÿ“ฑ cockpit PWA]; TG[โœˆ Telegram]; MIC[๐ŸŽ™ voice]; REPL[๐Ÿง  REPL / MCP]
    end
    subgraph edge["๐Ÿ›ก edge"]
        CF[โ˜๏ธ cloudflared tunnel]; PK[๐Ÿ” passkey session]
    end
    subgraph thor["๐Ÿง  Jetson AGX Thor ยท docker compose slim"]
        DASH["dashboard :8080 TLS<br/>FastAPI ยท agent ยท replay ยท personas"]
        AG[strands agent<br/>ROVER_ALL_TOOLS]
        SDK["earth-rovers-sdk :8002<br/>headless Chromium ยท Agora"]
        HUB["media hub :8090<br/>one drainer, N subscribers"]
        YOLO[yolo detector]
        THK[๐Ÿข thinker]; TEL[telegram listener]; VOX[voice agent]
        SUP["scout-supervisor<br/>(host, unix socket)"]
        REC[(LeRobot v3 datasets<br/>+ ECoT + detections)]
        MEM[(.memory SQLite)]
    end
    ROVER((๐Ÿ›ž Earth Rover Mini+))
    PWA -. HTTPS / WS .-> CF --> PK --> DASH
    TG --> TEL --> AG; MIC --> VOX --> AG; REPL --> AG; DASH --> AG
    AG <--> SDK; HUB --> SDK; YOLO --> HUB; VOX --> HUB
    AG --> REC; YOLO --> REC; AG <--> MEM
    DASH -. start/stop .-> SUP -. docker compose .-> THK & TEL & VOX
    SDK <==>|WebRTC| ROVER

Ports and services come straight from docker-compose.slim.yml + docker-compose.slim.override.yml โ€” see Architecture.

sdk :8002earth-rovers-sdk โ€” headless Chromium joined to the rover's Agora channel; `/control` `/data` `/v2/front` `/speak`
dashboard :8080FastAPI cockpit (TLS) โ€” agent, cameras, joystick, personas, replay; the only thing the tunnel exposes
media :8090MediaHub โ€” single drainer for mic + cameras, fan-out to recorder / YOLO / voice
yoloUltralytics on the hub's front stream โ†’ `detections/episode_N.jsonl` sidecars
telegram ยท thinker ยท voicepersona containers (profiles), switched live from the cockpit via the host supervisor
cloudflaredpublic hostname โ†’ `https://localhost:8080`, real TLS at the edge

Where to next#

  • Start


    What you need, installing on the brain, docker compose slim, the .env walkthrough, first drive.

    Start here

  • Dashboard


    Passkey gate, camera tiles, glass joystick, Ask stream, persona pills, dataset replay.

    The cockpit

  • Perception


    MediaHub fan-out, YOLO sidecars, the Locate-Anything "who to follow" idea.

    Perception

  • Datasets & replay


    LeRobot v3, ECoT traces, why episodes are sealed, how to repair an index.

    Datasets

  • Tools reference


    Every @tool, generated from the code at build time โ€” signatures and docstrings.

    Reference

  • Operations


    Bring-up order, restart recipes, tunnel, watchdogs, what to check before a demo.

    Runbook