Virtualhome

VirtualHome是由MIT、麦吉尔大学、卢布尔雅那大学和多伦多大学联合创建的家庭活动模拟数据集，旨在通过程序化方法模拟复杂家庭活动。该数据集包含2821个家庭活动程序，涵盖75种原子动作和308种对象，平均每个程序包含11.6个步骤。数据通过众包方式收集，利用类似Scratch的图形化编程界面，将自然语言描述转化为程序化指令。创建过程分为两阶段：首先收集活动描述，再将其转化为完整程序。VirtualHome可用于训练机器人执行复杂任务、视频理解模型开发以及自然语言指令解析等领域。VirtualHome is a household activity simulation dataset co-created by MIT, McGill University, University of Ljubljana, and University of Toronto, aiming to simulate complex household activities through procedural methods. This dataset contains 2821 household activity programs, covering 75 atomic actions and 308 types of objects, with an average of 11.6 steps per program. The data is collected via crowdsourcing, using a Scratch-like graphical programming interface to convert natural language descriptions into procedural instructions. Its creation process is divided into two stages: first collect activity descriptions, then convert them into complete programs. VirtualHome can be applied to fields such as training robots to perform complex tasks, developing video understanding models, and natural language instruction parsing.

搜集汇总

背景与挑战

背景概述

VirtualHome是一个多智能体家庭模拟平台，专注于模拟家庭环境中的活动，通过人形智能体执行高级指令来交互和完成任务。该数据集包含一个丰富的活动知识库，支持多模态记录和多智能体并发控制，适用于视频渲染和强化学习训练等研究应用。