Part II: Teleoperation and data collection · §17 of 43
LeRobot is Hugging Face’s Python framework for real-robot learning. One toolchain covers:
teleoperate → record datasets (with cameras) → share on the Hub → train policies (ACT, Diffusion, SmolVLA, Pi0, …) → run policies on the robot
It has first-class OpenArm support: an openarm_follower robot plus leader/teleoperator devices.
The important thing to understand is that LeRobot doesn’t use ROS 2 at all. It is a second, independent control stack that talks to the very same Damiao motors directly:
Two independent stacks, one set of motors. Only one may own a follower bus at a time.
flowchart TB subgraph ros["ROS 2 · openarm container"] direction TB code["your code / MoveIt"] -.-> ctrl["controllers"] -.-> ohw["OpenArmHW"] -.-> ocan["openarm_can (C++)"] end subgraph lr["LeRobot · host, conda env lerobot"] direction TB leader["leader arms (Minis)<br/>or a policy (NN)"] -.-> robot["Robot.send_action()"] -.-> dbus["DamiaoMotorsBus (python-can)"] end ocan -.->|SocketCAN| can["can0 / can1"] dbus -.->|SocketCAN| can can ==> arms["follower arms · Damiao, MIT mode"]
The same diagram as text, as written in the guide
┌──────────────── Part I: ROS 2 stack ────────────────┐ your code / MoveIt → │ controllers → OpenArmHW → openarm_can (C++) │ ─┐ └──────────────────────────────────────────────────────┘ │ SocketCAN ├─► can0 / can1 ─► follower arms ┌──────────────── Part II: LeRobot stack ─────────────┐ │ (Damiao, MIT mode) leader arm(s) ─────► │ Teleoperator.get_action() → Robot.send_action() │ ─┘ policy (NN) ──────► │ DamiaoMotorsBus (python-can) — MIT commands, deg │ cameras ───────────► │ → LeRobotDataset (parquet + mp4) → Hub / training │ └──────────────────────────────────────────────────────┘
Consequences:
- Only one stack may own a follower CAN bus at a time. Both send MIT commands at high rate to the same motor IDs. Running both means two masters fighting over the arm. See §25.
- What you build in Part I (ros2_control, MoveIt, RViz) and Part II (LeRobot teleop, datasets, learned policies) are parallel paths to the same hardware, not layers of one system. They share:
- LeRobot runs on the host in its own conda env (
lerobot). ROS 2 runs in theopenarmcontainer. Both reachcan0/can1because the container uses--network host.
When to use which:
| Task | Use |
|---|---|
| Teleoperation with a leader arm, demonstration recording, imitation learning, running learned policies | LeRobot |
| Motion planning, collision checking, Cartesian goals, scripted trajectories, integration with other ROS nodes (perception, navigation) | ROS 2 / MoveIt |
| Visualising the robot model, checking frames, simulating before hardware | ROS 2 (RViz, mock hardware) |
LeRobot’s own visualisation is Rerun (--display_data=true), which shows joint values and camera streams as time series and images. It doesn’t show a 3D robot model.