A robot demo needs a failure log

The useful part of a robot demo starts when the robot misses.

Hugging Face's July 7 LeRobot v0.6.0 announcement describes six simulation benchmarks behind a common evaluation command, plus a rollout tool that supports human corrections. The project has since published v0.6.1, dated August 3. This is a development worth revisiting, not a new release this week.

For a small robotics group, the interesting possibility is a repeatable loop: choose a task, record attempts, inspect failures, change something, try again. A successful video supplies one observation. A record of attempts helps other builders understand the conditions that produced it.

Our proposed test: use one supported simulation task and retain unsuccessful runs alongside the best one. Record the software version and settings. Compare results before making a claim about improvement.

We have checked the release documentation, not run the robot stack. Simulation results would still need separate testing on physical hardware. The question for a follow-up is whether a community team can reproduce the result with the equipment and time it actually has.

Source documentation rechecked November 3, 2026. This brief revisits July and August releases, not a new release this week.