# Open-source tool for detecting issues in robot-learning datasets **URL:** \ **Category:** 🤗Datasets\ **Created:** [August 12, 2026, 9:37am UTC](https://discuss.huggingface.co/t/open-source-tool-for-detecting-issues-in-robot-learning-datasets/178602 "2026-08-12T09:37:48Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![omert27](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/omert27/32/63145_2.png) [@omert27](https://discuss.huggingface.co/u/omert27)\ **Post date:** [August 12, 2026, 9:37am UTC](https://discuss.huggingface.co/t/open-source-tool-for-detecting-issues-in-robot-learning-datasets/178602/1 "2026-08-12T09:37:48Z")
**Open-source tool for robot dataset issues** I’ve been working on **Calibra** , an open-source toolkit for analyzing robot-learning datasets. While working with demo data, I noticed a common issue: we often waste compute training on data that shouldn’t be there in the first place. Datasets can include: - duplicate or redundant demos - frozen camera frames - jittery robot motion - calibration drift - corrupted or inconsistent data So I built Calibra to help catch these issues before training. Would love to hear thoughts from anyone working with **LeRobot or imitation learning.** GitHub: [GitHub - omertt27/Calibra: Dataset observability and coreset selection for robotics imitation learning · GitHub](https://github.com/omertt27/Calibra) > **[Calibra — Dataset Integrity - a Hugging Face Space by omert27](https://huggingface.co/spaces/omert27/robot-dataset-health-check)** > > Integrity, quality & coverage checks for robot datasets I would appreciate any feedbacks. ---
**Author:** ![thordata](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/thordata/32/62596_2.png) [@thordata](https://discuss.huggingface.co/u/thordata)\ **Post date:** [September 4, 2026, 9:12am UTC](https://discuss.huggingface.co/t/open-source-tool-for-detecting-issues-in-robot-learning-datasets/178602/3 "2026-09-04T09:12:51Z")
This is a useful problem. One thing I’d want to see is a clear split between corrupted data and unusual but valid demonstrations, since rare movements can still be valuable in real-world data. A few visual examples for each failure type would make the tool much easier to trust. ---
**Author:** ![omert27](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/omert27/32/63145_2.png) [@omert27](https://discuss.huggingface.co/u/omert27)\ **Post date:** [September 4, 2026, 10:36am UTC](https://discuss.huggingface.co/t/open-source-tool-for-detecting-issues-in-robot-learning-datasets/178602/4 "2026-09-04T10:36:30Z")
Thank you for the feedback, i will consider. ---
**Author:** ![Sattyam](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/sattyam/32/62925_2.png) [@Sattyam](https://discuss.huggingface.co/u/Sattyam)\ **Post date:** [September 6, 2026, 12:45pm UTC](https://discuss.huggingface.co/t/open-source-tool-for-detecting-issues-in-robot-learning-datasets/178602/5 "2026-09-06T12:45:57Z")
The corrupted-versus-unusual-but-valid distinction raised upthread is the whole problem, and it has a measurable form: what is Calibra’s false-positive rate on a dataset you know is clean? Concretely. Take a LeRobot dataset you are confident in, run the detector, and publish the per-check flag rate. Jittery motion and calibration drift will both flag on some genuinely good demonstrations, because a human demonstrator is jittery and a real rig does drift. That rate is not a bug to tune to zero; it is the number a user needs in order to decide what to do with 40 flags on their own data. The reason this matters more here than in most tooling: a detector at the dataset layer sits upstream of every policy trained on it. If it silently removes 8 percent of valid demonstrations on a particular task family, the downstream success rate moves and nobody attributes it to the cleaner. Same discipline we hold ourselves to one layer down, at policy evaluation: every detection rate gets published beside its firing rate on known-benign input, and if the firings concentrate on a subset rather than spreading, that is a signal about the detector, not the data. ---
**Author:** ![omert27](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/omert27/32/63145_2.png) [@omert27](https://discuss.huggingface.co/u/omert27)\ **Post date:** [September 6, 2026, 1:12pm UTC](https://discuss.huggingface.co/t/open-source-tool-for-detecting-issues-in-robot-learning-datasets/178602/6 "2026-09-06T13:12:24Z")
Thanks for the feedback Sattyam, i will implement these benchmarks. ---
**Author:** ![Sattyam](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/sattyam/32/62925_2.png) [@Sattyam](https://discuss.huggingface.co/u/Sattyam)\ **Post date:** [September 12, 2026, 4:50pm UTC](https://discuss.huggingface.co/t/open-source-tool-for-detecting-issues-in-robot-learning-datasets/178602/7 "2026-09-12T16:50:33Z")
Picking up the point about where the reporting minimum should live. I have come round to thinking it belongs in the dataset card, not in the tool. A tool-side check can be skipped and leaves no trace. A card field is visible to the next person who downloads the dataset, whether or not they ran anything. Three fields would cover most of what I keep having to reconstruct by hand: 1. `episodes_total` and `episodes_used`, separately. Almost every dataset I have looked at has a filtering step, and the filtered count is what a success rate is actually divided by. Right now that denominator is usually not recoverable. 2. `simulator` and its version, or `hardware` and the platform. Not both, one or the other, and stated. A dataset that does not say which it is cannot be compared with one that does. 3. `known_defects`, free text, allowed to be empty but not allowed to be absent. The forcing function is having to write “none found”, which is a different claim from silence. What I am not proposing: a quality score, a badge, or anything that ranks datasets. Those invite gaming and I do not think anyone wants to maintain the rubric. Our own cards do not fully do this yet, so this is a proposal I am also behind on.