Prandium Docs

24. Recording, training and running policies

4 min read

Part II: Teleoperation and data collection · §24 of 43

24.1 Record a dataset

lerobot-record \
  --robot.type=bi_openarm_follower \
  --robot.left_arm_config.port=can0  --robot.left_arm_config.side=left  --robot.left_arm_config.max_relative_target=5.0 \
  --robot.right_arm_config.port=can1 --robot.right_arm_config.side=right --robot.right_arm_config.max_relative_target=5.0 \
  --robot.id=my_bimanual_follower \
  --robot.cameras='{top: {type: opencv, index_or_path: /dev/video0, width: 640, height: 480, fps: 30}}' \
  --teleop.type=bi_openarm_mini \
  --teleop.left_arm_config.port=/dev/serial/by-id/usb-1a86_USB_Single_Serial_5876043720-if00  --teleop.left_arm_config.side=left \
  --teleop.right_arm_config.port=/dev/serial/by-id/usb-1a86_USB_Single_Serial_5876043592-if00 --teleop.right_arm_config.side=right \
  --teleop.id=my_mini \
  --dataset.repo_id=<hf_user>/openarm_pick_cube \
  --dataset.single_task="Pick up the red cube and place it in the bowl" \
  --dataset.num_episodes=20 \
  --dataset.episode_time_s=60 --dataset.reset_time_s=20 \
  --dataset.fps=30 \
  --dataset.no_stamp=true \
  --dataset.push_to_hub=false \
  --display_data=false

Dataset names (new in V3)

  • Without --dataset.no_stamp=true, lerobot-record appends a date-time tag to the name: <hf_user>/openarm_pick_cube becomes <hf_user>/openarm_pick_cube_20261010_143000. Every later command (lerobot-train, lerobot-dataset-viz, --resume) then needs that full name. ls ~/.cache/huggingface/lerobot/<hf_user>/ shows the real names. With no_stamp, the name stays as typed. To add episodes to it later, add --resume=true.
  • Names starting with eval_ are refused by lerobot-record (“reserved for policy evaluation”). Policy evaluation is done with lerobot-rollout and needs names starting with rollout_ (§37).

Recording uses stock LeRobot, so it has the start-up snap behaviour described in §23.3 (safe_teleop.py doesn’t record). Before every lerobot-record: run compare_mini_follower.py, put the Minis in the followers’ pose (all diffs ≈ 0), keep max_relative_target, and keep --display_data=false. A recording mode for safe_teleop.py (gate first, then hand over to LeRobot’s dataset writer) is the obvious next improvement.

The docs page you pasted uses short flags (--repo-id, --num-episodes, --fps). The v0.6.2 CLI expects the --dataset.* names above.

  • Data goes to ~/.cache/huggingface/lerobot/<repo_id>/ (override with --dataset.root=…): parquet for joint data, mp4 per camera.
  • Keyboard during recording (needs an X session; pynput): → ends the episode early, ← re-records it, Esc stops.
  • --resume=true appends to an existing dataset. --dataset.push_to_hub=true uploads (needs hf auth login).
  • Recorded observation = follower joint positions in degrees (+ images). Action = the (clipped) command sent to the follower, i.e. the leader’s pose.

24.2 Inspect / replay

lerobot-dataset-viz --repo-id <hf_user>/openarm_pick_cube --episode-index 0      # Rerun viewer
lerobot-replay --robot.type=bi_openarm_follower … --dataset.repo_id=<hf_user>/openarm_pick_cube --dataset.episode=0

lerobot-replay drives the real follower through the recorded actions. Same safety rules as teleop: start pose matched, max_relative_target set.

24.3 Train a policy (on the RTX 5050 or a bigger machine)

lerobot-train \
  --dataset.repo_id=<hf_user>/openarm_pick_cube \
  --policy.type=act \
  --output_dir=outputs/train/act_openarm_pick_cube \
  --job_name=act_openarm_pick_cube \
  --policy.device=cuda \
  --policy.push_to_hub=false

ACT is a good first policy for ~20–50 demos. Diffusion, SmolVLA and Pi0 need their extras and much more GPU memory. An 8 GB laptop GPU is tight for anything beyond ACT/Diffusion with small batch sizes.

The trained model ends up in outputs/train/act_openarm_pick_cube/checkpoints/last/pretrained_model/ (relative to where you ran lerobot-train). That directory is what --policy.path points to in Part III (§31.2). It contains the weights, the policy config (which inputs it expects) and the normalisation statistics of the training dataset. --policy.push_to_hub=true --policy.repo_id=<hf_user>/act_openarm_pick_cube uploads it so you can use --policy.path=<hf_user>/act_openarm_pick_cube instead.

24.4 Which policy to train

Policy--policy.typeExtra to installInference on zeus (RTX 5050, 8 GB)Notes
ACTactnone✅ fast, --inference.type=syncBest first choice: 20–50 demos of one task
Diffusiondiffusionpip install -e ".[diffusion]"✅ slower per callSmoother, multimodal behaviour; needs more demos
SmolVLAsmolvlapip install -e ".[smolvla]"⚠ probably fits; use RTCLanguage-conditioned (--task matters); fine-tune from lerobot/smolvla_base
Pi0 / Pi0.5pi0 / pi05pip install -e ".[pi]"❌ too big for 8 GB, use a bigger GPU + async inference (§40)Strongest generalist VLAs

Install extras inside the lerobot env, in ~/Documents/lerobot. The training command (§24.3) is the same apart from --policy.type (VLAs: --policy.path=lerobot/smolvla_base to fine-tune instead of --policy.type).

24.5 Running the trained policy

Running a policy on the arms has its own part, written for a first-time user: Part III.

You want to…Go to
Understand what running a policy means, and the vocabulary§30
Find the trained model, check what it expects, find the training start pose§31
Understand every flag of lerobot-rollout (+ a ready-made script)§32
Know what happens, and when the arms move§33
Run it the first time, safely§34, §35
Make it smoother or more reactive§36
Measure the success rate§37
Correct it with the Minis (DAgger)§38
Fix an error§41

Graph View

Backlinks