Skip to content

WebXR teleop (Quest 3)·

90s

Teleoperate the G1 straight from the Quest 3 browser: no APK, no sideload. A WebXR page tracks your hands/controllers/head and streams poses to a bridge on the robot, which runs IK and publishes to the arm controller over DDS.

Open the live page

With the bridge running, point the Quest browser at https://<robot-ip>:8013/ (docs/teleop/index.html).

architecture·

flowchart LR
    Q["Quest 3 browser<br/>native WebXR"]
      -- "WSS, Vuer-format<br/>column-major SE(3) 4×4" --> B
    subgraph ROBOT["G1 / Jetson"]
        B["xr_bridge<br/>WSS :8012 + HTTPS :8013"] --> PS["WebXRPoseSource"]
        PS --> TV["televuer wrapper"] --> IK["xr_teleoperate IK"] --> DDS["ArmController → DDS"]
    end

Why it works without Vuer: WebXR hands out each pose as a column-major Float32Array(16) in the OpenXR basis, exactly what Unitree's televuer consumes, so the upstream transform and IK run unchanged over our WSS transport.

you need·
headset Quest 3 / Pro / 3S, hand tracking enabled
robot G1 (29/23 DoF) reachable over DDS (eth0, 192.168.123.x)
network Quest and robot on the same LAN
TLS make xr-cert (WebXR needs a secure context)

The page pulls the G1's binocular head cam over WebRTC from Unitree's teleimager (/offer, port 60001, side-by-side stereo 480×1280) and renders each half to one eye, stereo parallax for free.

mode you see
Immersive VR robot's stereo first-person view fills the headset
AR passthrough robot view floats as a panel in your room

Paste teleimager's /offer URL (auto-filled to https://<host>:60001/offer). Test robot camera shows a 2D preview before you enter XR.

setup·

# 1 · IK/robot stack on the robot (reuses Unitree's xr_teleoperate):
git clone --recurse-submodules https://github.com/unitreerobotics/xr_teleoperate
pip install -r xr_teleoperate/requirements.txt
pip install -e xr_teleoperate/teleop/televuer
pip install -e xr_teleoperate/teleop/robot_control/dex-retargeting
export PYTHONPATH="$PWD/xr_teleoperate:$PWD/xr_teleoperate/teleop/televuer/src:$PYTHONPATH"
pip install -r requirements-teleop.txt

# 2 · cert (secure context)
make xr-cert

# 3 · start the bridge
make xr-teleop-dry                          # verify headset stream, NO robot
make xr-teleop ARM=G1_29 IFACE=eth0         # live — IK + DDS (hand tracking)
# controllers + walking:
python -m neon.teleop.xr_bridge --input-mode controller --motion --network-interface eth0

on the Quest·

  1. Quest Browser → https://<robot-ip>:8013/, accept the cert once.
  2. Pick input (hand/controller) + send rate (30-90 Hz).
  3. Enter XR & Teleop, grant tracking.
  4. Wrist poses drive the arms; pinch (hands) / trigger (controllers) = gripper.

wire format·

Three JSON messages over WS, HAND_MOVE, CONTROLLER_MOVE, CAMERA_MOVE, decoded by WebXRPoseSource into the surface televuer.TeleVuer exposes.

safety·

  • Control loop starts only after the first valid motion frame.
  • Arm speed ramps gradually, no jump on connect.
  • No walking by default: opt-in (--motion), clamped ±0.3, right-A = stop.
  • FSM gating + arm mutex + battery refusal inherited from the DDS layer.
  • Keep clear. Have an e-stop ready.

troubleshooting·

symptom fix
"WebXR not available" use Quest Browser over HTTPS
bridge dot grey open https://<ip>:8013 first, accept cert
hands not tracked enable hand tracking in Settings; grant permission
arms don't move (poses shown) IK stack not on PYTHONPATH, or DDS down
jittery lower send rate to 30 Hz; check Wi-Fi RSSI

safety model architecture

Edit page