TSFM-Eval: Evaluating Accuracy, Calibration, and Efficiency in Zero-Shot Time Series Foundation Models

TSFM-Eval is a benchmark dataset and evaluation framework for zero-shot Time Series Foundation Models (TSFMs), accompanying the paper "Evaluating Accuracy, Calibration, and Efficiency in Zero-Shot Time Series Foundation Models". The benchmark contains more than 3.6 million forecasting records generated across 3 real-world datasets and 9 forecasting models under multiple experimental configurations involving varying context lengths and forecast horizons. The evaluated models include statistical baselines, supervised deep learning architectures, and modern TSFMs such as Chronos-2, TiRex, Moirai-2.0, Sundial, TimesFM, and Toto. TSFM-Eval supports research on point forecasting accuracy, probabilistic calibration, uncertainty quantification, inference latency, and deployment-oriented evaluation. The benchmark covers traffic forecasting, electricity load forecasting, and financial time series forecasting scenarios.