Skip to content

🛞 scout · a small robot with a big curiosity

sees · thinks · drives · remembers·

scout is a Strands agent living on a Jetson AGX Thor that drives a FrodoBots Earth Rover Mini+. It looks through the rover's two cameras, talks through its speaker, listens on its mic, takes orders from a passkey-gated web cockpit, Telegram or your voice — and records every drive as a LeRobot dataset with the agent's own reasoning aligned frame-by-frame.

docs last commit license dataset strands

a stylised stand-in, drawn from primitives — the real Earth Rover Mini+ CAD is linked on the hardware page (not redistributed: no license on the source repo)

First drive in 60 s Open the cockpit Tools reference

What is scout, in five bullets·

  • One agent, many faces. The same Strands agent answers in the REPL, the web cockpit's Ask box, Telegram, a bidirectional voice session over the rover's own mic and speaker, and a slow background thinker that wakes every minute to do one useful thing. → Personas
  • Real vision, real safety. Camera tools return actual image blocks the model sees; every motion tool clamps to [-1, 1], re-streams /control frames (the firmware wants a continuous stream) and always auto-stops — even on exceptions. → Tools
  • A cockpit you can trust from anywhere. FastAPI + vanilla JS PWA, sealed behind WebAuthn passkeys and HTTPS, exposed through a Cloudflare Tunnel. Joystick, cameras, live agent stream, persona switches, dataset replay. → Dashboard
  • Every drive is data. A LeRobot v3 recorder captures 10 Hz video + 27-D state + 2-D action, with the agent's reasoning trace as an Embodied-Chain-of-Thought sidecar and YOLO detections aligned to the same frames. → Datasets
  • Fleet-aware. With TINY_MCP=1 the agents mount tiny.technology's MCP tools and can ask sibling robots (Reachy, Fomo, Q…) for help — allow-listed, self-invoke refused, one hop deep. → Fleet

How it fits together·

flowchart LR
    subgraph clients["🌐 clients"]
        PWA[📱 cockpit PWA]; TG[✈ Telegram]; MIC[🎙 voice]; REPL[🧠 REPL / MCP]
    end
    subgraph edge["🛡 edge"]
        CF[☁️ cloudflared tunnel]; PK[🔐 passkey session]
    end
    subgraph thor["🧠 Jetson AGX Thor · docker compose slim"]
        DASH["dashboard :8080 TLS<br/>FastAPI · agent · replay · personas"]
        AG[strands agent<br/>ROVER_ALL_TOOLS]
        SDK["earth-rovers-sdk :8002<br/>headless Chromium · Agora"]
        HUB["media hub :8090<br/>one drainer, N subscribers"]
        YOLO[yolo detector]
        THK[🐢 thinker]; TEL[telegram listener]; VOX[voice agent]
        SUP["scout-supervisor<br/>(host, unix socket)"]
        REC[(LeRobot v3 datasets<br/>+ ECoT + detections)]
        MEM[(.memory SQLite)]
    end
    ROVER((🛞 Earth Rover Mini+))
    PWA -. HTTPS / WS .-> CF --> PK --> DASH
    TG --> TEL --> AG; MIC --> VOX --> AG; REPL --> AG; DASH --> AG
    AG <--> SDK; HUB --> SDK; YOLO --> HUB; VOX --> HUB
    AG --> REC; YOLO --> REC; AG <--> MEM
    DASH -. start/stop .-> SUP -. docker compose .-> THK & TEL & VOX
    SDK <==>|WebRTC| ROVER

Ports and services come straight from docker-compose.slim.yml + docker-compose.slim.override.yml — see Architecture.

sdk :8002earth-rovers-sdk — headless Chromium joined to the rover's Agora channel; `/control` `/data` `/v2/front` `/speak`
dashboard :8080FastAPI cockpit (TLS) — agent, cameras, joystick, personas, replay; the only thing the tunnel exposes
media :8090MediaHub — single drainer for mic + cameras, fan-out to recorder / YOLO / voice
yoloUltralytics on the hub's front stream → `detections/episode_N.jsonl` sidecars
telegram · thinker · voicepersona containers (profiles), switched live from the cockpit via the host supervisor
cloudflaredpublic hostname → `https://localhost:8080`, real TLS at the edge

Where to next·

  • Start

    What you need, installing on the brain, docker compose slim, the .env walkthrough, first drive.

    Start here

  • Dashboard

    Passkey gate, camera tiles, glass joystick, Ask stream, persona pills, dataset replay.

    The cockpit

  • Perception

    MediaHub fan-out, YOLO sidecars, the Locate-Anything "who to follow" idea.

    Perception

  • Datasets & replay

    LeRobot v3, ECoT traces, why episodes are sealed, how to repair an index.

    Datasets

  • Tools reference

    Every @tool, generated from the code at build time — signatures and docstrings.

    Reference

  • Operations

    Bring-up order, restart recipes, tunnel, watchdogs, what to check before a demo.

    Runbook