Metadata-informed transformers for informative time series forecasting

Time series forecasting is widely employed in real-world applications such as financial analysis and energy planning. Previous studies have mainly focused on the time series modality to capture the intricate variations and dependencies inherent in time series. Apart from numerical time series data, metadata (e.g., dataset and variate descriptions) convey valuable information, which is essential for forecasting; this information can be used to identify application scenarios and provide more interpretable knowledge than digit sequences. Based on this observation, we propose a metadata-informed time series Transformer (MetaTST), which incorporates multiple levels of context-specific metadata into Transformer forecasting models to enable informative time series forecasting. To tackle the unstructured nature of metadata, MetaTST formalizes them into natural languages bypre-designed templates and leverages large language models (LLMs) to encode these texts into metadata tokens as a supplement to classic series tokens, resulting in an informative embedding. Furthermore, a Transformer encoder is employed to communicate series and metadata tokens, extending series representations using metadata information to improve forecasting accuracy. Additionally, the proposed method allows the model to adaptively learn context-specific patterns across various scenarios; this is particularly effective in handling large-scale, diverse-scenario forecasting tasks. The results show that when applied to widely acknowledged short- and long-term forecasting benchmarks, MetaTST is superior to advanced time series models and LLM-based methods; additionally, it is suitable for both individual single- and joint-training multidataset settings.