ck3-gameplay-mouse-keyboard-dataset
收藏资源简介：
CK3 Gameplay Mouse Keyboard Dataset 是一个专门记录人类玩策略游戏《十字军之王 III》（Crusader Kings III）时产生的屏幕视频与高精度输入事件的多模态数据集。其核心目的是为行为克隆、模仿学习以及视频到动作预测等研究提供宝贵的真实人机交互数据。数据集捕获了玩家在游戏过程中所有的鼠标移动、点击、滚动和键盘按键事件，并与30 fps的桌面屏幕录制视频进行帧级精确同步。数据集由多个独立的游戏会话记录构成。每个会话包含以下文件：`screen.mp4`（使用libx264编码的30 fps桌面视频，包含光标绘制）、`input_events.csv`（记录所有带纳秒级时间戳的离散输入事件）、`input_samples.csv`（将输入状态以60 Hz频率均匀采样后的规整数据）、`metadata.json`（包含会话元数据、同步信息和精确的ffmpeg命令）以及`ffmpeg.log`（编码日志）。`metadata.json`中提供的同步公式确保了视频帧时间与输入时间戳之间的亚帧对齐。数据规模方面，计划包含187个录制会话，总计约66 GB的游戏视频和约1.8 GB的密集输入遥测数据（分布在374个CSV文件中）。数据集当前已有56个会话上线，并处于持续增长状态。输入数据提供了两种形式：原始的、不规则时间间隔的事件流，以及规整的、60 Hz采样的光标坐标、鼠标按钮状态和按键状态。该数据集适用于多种任务，包括：训练能够模仿人类真实游戏行为（如悬停、误点击、纠正）的智能体；开发从屏幕状态预测下一步操作（鼠标/键盘动作）的模型；进行鼠标轨迹与意图建模研究；为强化学习策略提供基于人类演示的暖启动；以及用于人机交互（HCI）和用户体验（UX）研究，分析人类在时间压力下如何导航复杂的嵌套菜单系统。数据集使用自定义记录器采集，仅当《十字军之王 III》游戏窗口处于前台活动状态时捕获输入，确保了用户隐私。
CK3 Gameplay Mouse Keyboard Dataset is a multimodal dataset that specifically records screen videos and high-precision input events generated by humans while playing the strategy game Crusader Kings III. Its core objective is to provide valuable real-world human-computer interaction (HCI) data for research areas including behavior cloning, imitation learning, and video-to-action prediction. The dataset captures all mouse movements, clicks, scrolls, and keyboard press events during gameplay, and synchronizes these events frame-precisely with 30 fps desktop screen recording videos. The dataset comprises multiple independent gameplay sessions. Each session includes the following files: `screen.mp4` (a 30 fps desktop video encoded with libx264, with cursor rendering included), `input_events.csv` (records all discrete input events with nanosecond-level timestamps), `input_samples.csv` (standardized data obtained by uniformly sampling input states at 60 Hz), `metadata.json` (contains session metadata, synchronization information, and exact ffmpeg commands), and `ffmpeg.log` (encoding log). The synchronization formula provided in `metadata.json` ensures sub-frame alignment between video frame timestamps and input event timestamps. In terms of data scale, the dataset is planned to include 187 recording sessions, totaling approximately 66 GB of gameplay videos and about 1.8 GB of dense input telemetry data (distributed across 374 CSV files). Currently, 56 sessions have been released, and the dataset is in continuous growth. The input data is offered in two formats: the original, irregularly spaced event stream, and the standardized, 60 Hz sampled cursor coordinates, mouse button states, and keyboard key states. This dataset is applicable to a wide range of tasks, including: training agents that imitate real human gameplay behaviors such as hovering, misclicking, and corrective actions; developing models that predict subsequent mouse/keyboard operations from screen states; conducting research on mouse trajectory and intention modeling; providing human demonstration-based warm start for reinforcement learning policies; and being used for HCI and user experience (UX) research to analyze how humans navigate complex nested menu systems under time pressure. The dataset is collected using a custom recorder, which only captures input when the Crusader Kings III game window is in the foreground active state, thus ensuring user privacy.
数据集概述
CK3 Gameplay Mouse Keyboard Dataset 是一个用于行为克隆、模仿学习和人机交互研究的高精度数据集，记录了一位人类玩家在游玩《Crusader Kings III》时的完整屏幕画面、鼠标移动、点击、滚轮和键盘事件。
核心特性
- 游戏：Crusader Kings III（CK3）
- 模态：视频 + 输入事件
- 视频：30 FPS，桌面捕获（libx264，gdigrab）
- 输入采样：60 Hz 均匀采样，原始事件纳秒级时间戳
- 数据规模：计划 187 个录制会话，约 66 GB 视频 + 1.8 GB 输入 CSV 文件（当前已发布 56 个会话，持续增长）
- 同步精度：通过共享性能计数器实现子帧级视频与输入对齐
- 隐私保护：仅当 CK3 为活动前景窗口时记录输入，不记录 Discord 等应用的键盘操作
数据内容
每个录制会话是一个独立文件夹，结构如下：
screen.mp4：30 FPS 桌面视频捕获input_events.csv：原始离散输入事件（鼠标移动、点击、滚轮、按键按下/释放），含纳秒级时间戳input_samples.csv：60 Hz 均匀重采样后的鼠标位置、按钮状态、按键状态metadata.json：会话元数据、同步公式、ffmpeg 命令等ffmpeg.log：编码日志
输入事件字段
| 事件 | 字段 |
|---|---|
mouse_move |
x, y |
mouse_click |
x, y, button, pressed |
mouse_scroll |
x, y, dx, dy |
key_press / key_release |
key |
同步公式
video_time_sec ≈ input_t_sec − ffmpeg_popen_sec_from_t0
潜在应用
- 行为克隆 / 模仿学习：训练 AI 以人类方式操作 CK3
- 视频到动作模型：从屏幕状态预测下一步鼠标/键盘动作
- 鼠标轨迹与意图建模：利用 60 Hz 密集光标遥测数据
- 强化学习引导：用人类演示预热 RL 策略
- 人机交互 / 用户体验研究：分析用户在深层菜单系统下的操作模式
数据加载示例
python from huggingface_hub import snapshot_download
下载全部数据（约 66 GB）
local_dir = snapshot_download( repo_id="Ethosoft/ck3-gameplay-mouse-keyboard-dataset", repo_type="dataset", )
或仅下载单个会话的输入数据
from huggingface_hub import hf_hub_download import pandas as pd
path = hf_hub_download( repo_id="Ethosoft/ck3-gameplay-mouse-keyboard-dataset", repo_type="dataset", filename="ck3_20260602_210003/input_samples.csv", ) df = pd.read_csv(path)
引用格式
bibtex @dataset{ck3_gameplay_mouse_keyboard, title = {CK3 Gameplay Mouse Keyboard Dataset}, author = {Yağız Ekrem Dalar}, year = {2026}, doi = {10.57967/hf/9535}, url = {https://huggingface.co/datasets/Ethosoft/ck3-gameplay-mouse-keyboard-dataset} }
