a9f3c7e2/CaTSBench

CaTS-Bench数据集是一个综合性的基准，用于评估多模态模型在时间序列理解、字幕生成和推理任务上的表现，覆盖了多个不同的领域。数据集包括14种不同的任务类型，测试时间序列理解的不同方面。每种任务都支持多模态（文本+图像）和仅文本的评估模式。数据集分为训练数据和测试数据，包含地面真实字幕、统计元数据、图像和数值时间序列值。任务文件定义了不同的任务集合，包括完整基准和困难子集。数据集还包含了用于字幕生成的提示语。时间序列数据覆盖了空气质量监测、犯罪统计、边境过境数据、人口统计等多个现实世界领域。评估指标包括准确性、领域特定性能、任务类型分析和模态效应比较。

The CaTS-Bench dataset is a comprehensive benchmark designed for evaluating multi-modal models on time series understanding, captioning, and reasoning tasks across diverse domains. It includes 14 distinct task types that test different aspects of time series understanding, supporting both multimodal (text + image) and text-only evaluation modes. The dataset is divided into training and test data, containing ground-truth captions, statistical metadata, images, and numerical time series values. Task files define different task sets, including the complete benchmark and difficult subsets. The dataset also includes prompts for caption generation. Time series data span multiple real-world domains such as air quality monitoring, crime statistics, border crossing data, demographics, and more. Evaluation metrics include accuracy, domain-specific performance, task-type analysis, and a comparison of modality effects.