xiaomoguhzz/R3-Bench-data

--- license: cc-by-4.0 pretty_name: R3-Bench Research Data size_categories: - n>1T tags: - benchmark - vision-language - video-understanding - evaluation --- # R3-Bench Research Data > **⚠️ This is not a HuggingFace `datasets` library dataset.** > This repo is a raw collection of project directories from R3-Bench research, > packaged as gzip'd tar files. Use the download + extraction instructions > below — do **not** use `datasets.load_dataset()` on this repo. ## What's inside Large binary outputs from the R3-Bench evaluation framework research — roughly **323 GB** across multiple tarballs. Each file corresponds one-to-one to a directory under the source working tree: | File | Source directory | Size | Shards | |---|---|---|---| | `alive.tar.gz` | `alive/` | 10.7 kB | 1 | | `backup_bin.tar.gz.part_{aa..ae}` | `backup_bin/` | ~193 GB | 5 | | `compare_methods_OmniVerifier_data.tar.gz` | `compare_methods/OmniVerifier/` | 2.09 GB | 1 | | `compare_methods_Step1X-Edit.tar.gz` | `compare_methods/Step1X-Edit/` | 6.23 GB | 1 | | `compare_methods_Step1X-Editv2.tar.gz` | `compare_methods/Step1X-Editv2/` | 129 MB | 1 | | `elo_human_eval.tar.gz` | `elo_human_eval/` | 146 MB | 1 | | `eval.tar.gz` | `eval/` | 3.02 GB | 1 | | `exps.tar.gz.part_{aa..ac}` | `exps/` | ~103 GB | 3 | | `images.tar.gz` | `images/` | 725 MB | 1 | | `iterative_ablation_output.tar.gz` | `iterative_ablation_output/` | 1.94 GB | 1 | | `logs.tar.gz` | `logs/` | 121 kB | 1 | | `output.tar.gz` | `output/` | 6.58 GB | 1 | | `paper_case.tar.gz` | `paper_case/` | 5.62 GB | 1 | Files larger than the HuggingFace single-file LFS limit were split into ~50 GB shards with `split -b 45G`. ## Download and extract ```bash # 1. Install the HF client pip install -U huggingface_hub # 2. Download everything to a local directory huggingface-cli download xiaomoguhzz/R3-Bench-data \ --repo-type dataset \ --local-dir R3-Bench-data/ cd R3-Bench-data/ # 3. Merge split shards back into monolithic tarballs cat backup_bin.tar.gz.part_* > backup_bin.tar.gz cat exps.tar.gz.part_* > exps.tar.gz rm backup_bin.tar.gz.part_* exps.tar.gz.part_* # 4. Extract each tarball for f in *.tar.gz; do echo "extracting $f ..." tar xzf "$f" done ``` After extraction the source working tree layout is reproduced: ``` alive/ backup_bin/ compare_methods/ elo_human_eval/ eval/ exps/ images/ iterative_ablation_output/ logs/ output/ paper_case/ ``` ## Why the Dataset Viewer shows an error The HuggingFace Dataset Viewer auto-detects tar files and tries to parse them as [WebDataset format](https://huggingface.co/docs/hub/datasets-webdataset). This repo is a raw tarball collection (not WebDataset), so the viewer reports a `SplitsNotFoundError`. **This is expected** and does not affect downloads — the error only concerns the preview panel, not the files themselves. ## Related code The research code that produces and consumes this data lives in the companion GitHub repository [`xiaomoguhz/R3-Bench`](https://github.com/xiaomoguhz/R3-Bench) (currently private; reach out for access if needed). ## License Released under **CC-BY-4.0**. You are free to share and adapt, provided you give appropriate credit.license: cc-by-4.0 pretty_name: R3-Bench 研究数据集 size_categories: - 数据量超过1TB tags: - 基准测试（benchmark） - 视觉语言（vision-language） - 视频理解（video-understanding） - 模型评估（evaluation） # R3-Bench 研究数据集 > **⚠️ 这并非 Hugging Face（HuggingFace）`datasets` 库兼容的标准数据集。** > 本仓库为 R3-Bench 研究项目的原始目录集合，以 gzip 压缩的 tar 包形式打包。请遵循下方的下载与解压说明操作——**切勿**在此仓库上调用 `datasets.load_dataset()`。 ## 数据集内容说明 本数据集包含 R3-Bench 评估框架研究产出的大型二进制输出文件，总容量约 **323 GB**，分布于多个 tar 包中。每个文件与源工作树下的目录一一对应： | 文件名 | 源目录 | 容量 | 分片数 | |---|---|---|---| | `alive.tar.gz` | `alive/` | 10.7 kB | 1 | | `backup_bin.tar.gz.part_{aa..ae}` | `backup_bin/` | ~193 GB | 5 | | `compare_methods_OmniVerifier_data.tar.gz` | `compare_methods/OmniVerifier/` | 2.09 GB | 1 | | `compare_methods_Step1X-Edit.tar.gz` | `compare_methods/Step1X-Edit/` | 6.23 GB | 1 | | `compare_methods_Step1X-Editv2.tar.gz` | `compare_methods/Step1X-Editv2/` | 129 MB | 1 | | `elo_human_eval.tar.gz` | `elo_human_eval/` | 146 MB | 1 | | `eval.tar.gz` | `eval/` | 3.02 GB | 1 | | `exps.tar.gz.part_{aa..ac}` | `exps/` | ~103 GB | 3 | | `images.tar.gz` | `images/` | 725 MB | 1 | | `iterative_ablation_output.tar.gz` | `iterative_ablation_output/` | 1.94 GB | 1 | | `logs.tar.gz` | `logs/` | 121 kB | 1 | | `output.tar.gz` | `output/` | 6.58 GB | 1 | | `paper_case.tar.gz` | `paper_case/` | 5.62 GB | 1 | 超过 Hugging Face 单文件 LFS 限制的文件已通过 `split -b 45G` 命令拆分为约 50 GB 的分片。 ## 下载与解压 bash # 1. 安装 Hugging Face 客户端 pip install -U huggingface_hub # 2. 将所有文件下载至本地目录 huggingface-cli download xiaomoguhzz/R3-Bench-data --repo-type dataset --local-dir R3-Bench-data/ cd R3-Bench-data/ # 3. 合并拆分的分片为完整 tar 包 cat backup_bin.tar.gz.part_* > backup_bin.tar.gz cat exps.tar.gz.part_* > exps.tar.gz rm backup_bin.tar.gz.part_* exps.tar.gz.part_* # 4. 解压所有 tar 包 for f in *.tar.gz; do echo "extracting $f ..." tar xzf "$f" done 解压完成后，将还原源工作树的目录结构： alive/ backup_bin/ compare_methods/ elo_human_eval/ eval/ exps/ images/ iterative_ablation_output/ logs/ output/ paper_case/ ## 数据集查看器报错说明 Hugging Face 数据集查看器会自动识别 tar 文件并尝试将其解析为 [WebDataset 格式](https://huggingface.co/docs/hub/datasets-webdataset)。本仓库仅为原始 tar 包集合（并非 WebDataset），因此查看器会抛出 `SplitsNotFoundError`。 **此现象属于预期行为**，不会影响文件下载——该报错仅影响预览面板，不影响文件本身的可用性。 ## 关联代码 用于生成和使用本数据集的研究代码存放在配套 GitHub 仓库 [`xiaomoguhz/R3-Bench`](https://github.com/xiaomoguhz/R3-Bench)（当前为私有仓库；如需访问可联系申请）。 ## 许可证 本数据集采用 **CC-BY-4.0** 许可证发布。您可自由分享或改编本数据集，但需提供恰当的署名。