MultiBench

MultiBench是一个系统化且统一的大规模基准，涵盖15个数据集、10种模态、20个预测任务和6个研究领域。它提供了一个自动化的端到端机器学习管道，简化和标准化了数据加载、实验设置和模型评估。MultiBench还提供了一种全面的方法来评估（1）泛化能力，（2）训练和推理的时间和空间复杂性，以及（3）对噪声和缺失模态的鲁棒性。

MultiBench is a systematic and unified large-scale benchmark encompassing 15 datasets, 10 modalities, 20 prediction tasks, and 6 research domains. It provides an automated end-to-end machine learning pipeline that simplifies and standardizes data loading, experimental setup, and model evaluation. MultiBench also offers a comprehensive approach to evaluate (1) generalization ability, (2) time and space complexity of training and inference, and (3) robustness to noise and missing modalities.