shi-labs/physical-ai-bench-conditional-generation

--- language: - en pretty_name: PAI-Bench-Transfer configs: - config_name: benchmark data_files: - split: PAIBenchTransfer path: metadata.csv task_categories: - video-to-video license: mit --- # Physical AI Bench - Conditional Generation [Paper](https://huggingface.co/papers/2512.01989) | [Code](https://github.com/SHI-Labs/physical-ai-bench) This dataset (Phsical AI benchmark, PAI-Bench) consisting of 600 examples across three key scenarios: robotic arm operations, driving, and ego-centric everyday life scenes, each representing a critical aspect of Physical AI. This dataset is constructed by sampling a number of videos from three different datasets. The specific details are provided below. | Dataset | Category | Sample Nums | | ------------------------------------------------------------ | ------------------ | ----------- | | [Agibot World](https://github.com/OpenDriveLab/AgiBot-World) | Robotics | 200 | | [OpenDV](https://github.com/OpenDriveLab/DriveAGI.git) | Autonomous Driving | 200 | | [Ego-Exo4D](https://ego-exo4d-data.org/) | Ego-centric | 200 | ## Dataset Summary - **Dataset Size**: 600 video samples - **Video Format**: MP4 files with various processing variants - **Annotations**: Text captions for each video - **Processing Variants**: Blur, Canny edge detection, Depth estimation, SAM2 segmentation ## File Organization ```text physical-ai-bench-transfer/ ├── videos/ # Original video files ├── blur/ # Blur-processed videos ├── canny/ # Edge detection videos ├── depth_vids/ # Depth estimation videos ├── depth_npzs/ # Depth estimation numpy arrays ├── sam2_vids/ # SAM2 segmentation videos ├── sam2_pkls/ # SAM2 segmentation pickle files └── captions/ # JSON files with video descriptions ``` ## Citation If you use Physical AI Bench in your research, please cite: ```bibtex @misc{zhou2025paibenchcomprehensivebenchmarkphysical, title={PAI-Bench: A Comprehensive Benchmark For Physical AI}, author={Fengzhe Zhou and Jiannan Huang and Jialuo Li and Deva Ramanan and Humphrey Shi}, year={2025}, eprint={2512.01989}, archivePrefix={arXiv}, primaryClass={cs.CV}, url={https://arxiv.org/abs/2512.01989}, } ```--- 语言： - 英语 展示名称：PAI-Bench-Transfer 配置项： - 配置名称：基准测试 数据文件： - 拆分集：PAIBenchTransfer 路径：metadata.csv 任务类别： - 视频到视频生成 (video-to-video) 许可证：MIT --- # 物理AI基准测试：条件生成任务 [论文](https://huggingface.co/papers/2512.01989) | [代码](https://github.com/SHI-Labs/physical-ai-bench) 本数据集（物理人工智能基准测试，PAI-Bench）包含600个样本，涵盖三大核心场景：机械臂操作、自动驾驶以及第一人称日常生活场景，每一场景均代表物理人工智能的关键研究方向。本数据集通过从三个不同数据集抽取视频样本构建而成，具体细节如下： | 数据集名称 | 类别 | 样本数量 | | ------------------------------------------------------------ | ------------------ | -------- | | [Agibot World](https://github.com/OpenDriveLab/AgiBot-World) | 机器人技术 | 200 | | [OpenDV](https://github.com/OpenDriveLab/DriveAGI.git) | 自动驾驶 | 200 | | [Ego-Exo4D](https://ego-exo4d-data.org/) | 第一人称视角 | 200 | ## 数据集概览 - **数据集规模**：600个视频样本 - **视频格式**：包含多种处理变体的MP4文件 - **标注信息**：每个视频对应的文本描述 - **处理变体**：模糊处理、Canny边缘检测、深度估计、SAM2分割 ## 文件组织结构 text physical-ai-bench-transfer/ ├── videos/ # 原始视频文件目录 ├── blur/ # 模糊处理视频目录 ├── canny/ # Canny边缘检测视频目录 ├── depth_vids/ # 深度估计视频目录 ├── depth_npzs/ # 深度估计NumPy数组目录 ├── sam2_vids/ # SAM2分割视频目录 ├── sam2_pkls/ # SAM2分割Pickle文件目录 └── captions/ # 视频描述JSON文件目录 ## 引用方式 若您在研究中使用物理AI基准测试数据集，请引用以下文献： bibtex @misc{zhou2025paibenchcomprehensivebenchmarkphysical, title={PAI-Bench: A Comprehensive Benchmark For Physical AI}, author={Fengzhe Zhou and Jiannan Huang and Jialuo Li and Deva Ramanan and Humphrey Shi}, year={2025}, eprint={2512.01989}, archivePrefix={arXiv}, primaryClass={cs.CV}, url={https://arxiv.org/abs/2512.01989}, }