A careful examination of large behavior models for multitask dexterous manipulation

Robot manipulation has seen tremendous progress in recent years, with imitation learning policies enabling successful performance of dexterous and hard-to-model tasks. Concurrently, scaling data and model size has led to the development of capable language and vision foundation models, motivating large-scale efforts to create general-purpose robot foundation models. While these models have garnered significant enthusiasm and investment, meaningful evaluation of real-world performance remains a challenge, limiting both the pace of development and inhibiting a nuanced understanding of current capabilities. In this paper, we rigorously evaluate multitask robot manipulation policies, referred to as Large Behavior Models (LBMs), by extending the Diffusion Policy paradigm across a corpus of simulated and real-world robot data. We propose and validate an evaluation pipeline to rigorously analyze the capabilities of these models with statistical confidence. We compare against single-task baseli..., , , # Data from: A careful examination of large behavior models for multitask dexterous manipulation

This dataset contains the raw evaluation data used to generate the Compact Letter Display (CLD) plots in the paper. The paper rigorously evaluates multitask robot manipulation policies — Large Behavior Models (LBMs) obtained through multitask pretraining on approximately 1,700 hours of demonstration data — and compares them against single-task baselines through blind, randomized A/B trials in both simulation and the real world (1,800 real-world rollouts and over 47,000 simulation rollouts). Two performance metrics are used: success rate (SR), a binary outcome indicating whether the task was completed, and task completion (TC), a continuous measure of the fraction of task-specific milestones achieved per rollout. Each CSV file corresponds to one or more figure panels, recording per-rollout outcomes for comparing these policies. Statistical comparisons between methods use sequential A/B testi...,