VLMbench

VLMbench是由加州大学圣克鲁兹分校的研究人员创建的一个视觉与语言操作基准数据集，包含6555条记录。该数据集旨在通过各种语言指令来评估机器人在分类机器人操作任务中的表现，如导航和物体操作。数据集通过模块化规则基础的任务模板自动生成机器人演示，包含多样化的物体形状和外观、动作类型及运动约束。此外，数据集还开发了基于关键点的6D-CLIPort模型，以处理多视角观察和语言输入，输出一系列6自由度动作。VLMbench旨在推动未来在语言引导机器人操作领域的研究。VLMbench is a visual-language manipulation benchmark dataset developed by researchers at the University of California, Santa Cruz, comprising 6555 records. This dataset is designed to evaluate robot performance in categorical robotic manipulation tasks such as navigation and object manipulation using diverse natural language instructions. Robotic demonstrations are automatically generated via modular rule-based task templates within the dataset, which cover a wide range of object shapes and appearances, action types, and motion constraints. Furthermore, a keypoint-based 6D-CLIPort model has been developed for VLMbench to process multi-view observations and language inputs, outputting a sequence of 6-degree-of-freedom actions. VLMbench aims to advance future research in the field of language-guided robotic manipulation.

搜集汇总

背景与挑战

背景概述

VLMbench是一个由加州大学圣克鲁兹分校创建的视觉与语言操作基准数据集，包含6555条记录，旨在通过语言指令评估机器人在分类操作任务（如导航和物体操作）中的表现。该数据集通过模块化规则自动生成机器人演示，涵盖多样化的物体形状、外观和动作类型，并开发了基于关键点的6D-CLIPort模型以处理多视角观察和语言输入，输出6自由度动作序列，以推动语言引导机器人操作领域的研究。