LMAct Benchmark

该数据集是一个旨在评估大型基础模型在多种交互式决策任务中的多模态决策能力的基准测试。该基准测试涵盖了零样本、少样本和多样本情境，以评估模型在上下文学习环境中的表现。其规模可达到上百万个标记和最多512个完整演示集。任务包括玩井字游戏、国际象棋、Atari游戏、网格世界导航、解纵横字谜以及控制模拟猎豹等交互式决策任务。

This dataset is a benchmark developed to evaluate the multimodal decision-making abilities of large foundation models across various interactive decision-making tasks. This benchmark incorporates zero-shot, few-shot, and many-shot scenarios to assess model performance within in-context learning settings. It encompasses up to millions of tokens and a maximum of 512 complete demonstration sets. The covered tasks include interactive decision-making scenarios such as Tic-Tac-Toe, Chess, Atari games, grid world navigation, crossword puzzle solving, and controlling simulated cheetahs.