RealVLG-11B

<p align="center"> <h1 align="center"> RealVLG-R1: A Large-Scale Real-World Visual-Language Grounding Benchmark for Robotic Perception and Manipulation <br> [CVPR 2026] </h1> <p align="center"> <a href="https://lif314.github.io/"><strong>Linfei Li</strong></a> · <a href="https://scholar.google.com/citations?user=8VOk_S4AAAAJ&hl=en"><strong>Lin Zhang*</strong></a> · <a href="https://scholar.google.com/citations?user=A0N_mS0AAAAJ&hl=en"><strong>Ying Shen</strong></a> </p> <h3 align="center"><a href="https://lif314.github.io/projects/realvlg_r1/">🌐Project page</a> | <a href="https://arxiv.org/abs/2603.14880">📝Paper(arXiv)</a> | <a href="https://github.com/lif314/RealVLG-R1">💻Code </a> </h3> <div align="center"></div> </p> ## Sample Each data sample is annotated as follows: ```json [ { "image_name": "", "image_path": "", "object_id": "", "mask_path": "", "description": "", "label": "", # short description "bbox": [x1, y1, x2, y2], "grasps": [ [x0,y0,x1,y1,x2,y2,x3,y3], ... ], "contact_points": [ [x1,y1, x2, y2], ... ] } ] ``` The definition diagrams of bbox and grasp are shown in the figure below:  ## Usage Download the dataset and extract `xxx_VLG.zip`. In each `xxx_VLG` folder, run `python metadata_viewer.py` to view the data formatting. The left/right keys switch between different objects in the same image, and the up/down keys switch between images. The visualization of different data subsets is shown below: | Subdata | Cornell_VLG | VMRD_VLG | OCID_VLG | GraspNet_VLG | Jacquard_VLG | |---------|-------------|----------|----------|--------------|---------------| | Demo |  |  |  |  |  | For more detailed data loading, please refer to `metadata_viewer.py`. > Note: ``Jacquard_VLG`` is a simulated dataset not discussed in the paper. Its language annotations are derived from ShapeNetSem category labels. ## License We thank all previous work. If you use this dataset, please cite the relevant work and comply with their licenses. - [Cornell](https://www.kaggle.com/datasets/oneoneliu/cornell-grasp) - [VMRD](https://opendatalab.com/OpenDataLab/VMRD) - [OCID-Grasp](https://github.com/stefan-ainetter/grasp_det_seg_cnn) - [GraspNet](https://graspnet.net/) - [Jacquard](https://jacquard.liris.cnrs.fr/)# RealVLG-R1：面向机器人感知与操作的大规模真实世界视觉语言接地（Visual-Language Grounding）基准数据集 [CVPR 2026] <p align="center"> <a href="https://lif314.github.io/"><strong>李林飞</strong></a> · <a href="https://scholar.google.com/citations?user=8VOk_S4AAAAJ&hl=en"><strong>张霖*</strong></a> · <a href="https://scholar.google.com/citations?user=A0N_mS0AAAAJ&hl=en"><strong>沈颖</strong></a> </p> <h3 align="center"><a href="https://lif314.github.io/projects/realvlg_r1/">🌐项目主页</a> | <a href="https://arxiv.org/abs/2603.14880">📝论文（arXiv）</a> | <a href="https://github.com/lif314/RealVLG-R1">💻代码</a></h3> <div align="center"></div> ## 样本格式 每个数据样本的标注格式如下： json [ { "图像名称": "", "图像路径": "", "物体ID": "", "掩码路径": "", "描述文本": "", "标签": "", # 简短描述 "边界框": [x1, y1, x2, y2], "抓取姿态": [ [x0,y0,x1,y1,x2,y2,x3,y3], ... ], "接触点": [ [x1,y1, x2, y2], ... ] } ] 边界框与抓取姿态的定义示意图如下所示：  ## 使用方法 下载数据集并解压`xxx_VLG.zip`。在每个`xxx_VLG`文件夹中，运行`python metadata_viewer.py`即可查看数据格式。使用左右方向键可切换同一张图像中的不同物体，上下方向键可切换图像。不同数据子集的可视化效果如下： | 子数据集 | Cornell_VLG | VMRD_VLG | OCID_VLG | GraspNet_VLG | Jacquard_VLG | |---------|-------------|----------|----------|--------------|---------------| | 示例 |  |  |  |  |  | 如需了解更详细的数据加载方法，请参考`metadata_viewer.py`文件。 > 注意：`Jacquard_VLG`为本文未涉及的仿真数据集，其语言标注源自ShapeNetSem的类别标签。 ## 许可协议 我们对所有先前的研究工作致以谢意。若您使用本数据集，请引用相关文献并遵守其许可协议。 - [Cornell](https://www.kaggle.com/datasets/oneoneliu/cornell-grasp) - [VMRD](https://opendatalab.com/OpenDataLab/VMRD) - [OCID-Grasp](https://github.com/stefan-ainetter/grasp_det_seg_cnn) - [GraspNet](https://graspnet.net/) - [Jacquard](https://jacquard.liris.cnrs.fr/)

搜集汇总

背景与挑战

背景概述

RealVLG-11B（全称RealVLG-R1）是一个大规模真实世界视觉-语言接地基准，专为机器人感知和操作设计，包含图像、对象标注、抓取数据等多模态信息。该数据集基于多个子集构建，采用Apache License 2.0许可证，更新于2026年5月，数据量约148.76GB。