DataPerf

DataPerf是一个由社区驱动的基准套件，旨在评估机器学习数据集和以数据为中心的算法。该数据集通过竞争、可比性和可重复性来促进数据中心AI的创新。DataPerf允许机器学习社区迭代数据集，而不仅仅是架构，并提供一个开放的在线平台，支持这种迭代开发。首次迭代包含五个基准，涵盖视觉、语音、获取、调试和扩散提示中的数据中心技术、任务和模态的广泛范围，并支持社区贡献新的基准。这些基准、在线评估平台和基线实现都是开源的，MLCommons协会将维护DataPerf，以确保对学术界和工业界产生长期利益。

DataPerf is a community-driven benchmark suite designed to evaluate machine learning datasets and data-centric algorithms. It fosters innovation in data-centric AI through competition, comparability, and reproducibility. DataPerf enables the machine learning community to iterate on datasets, not just model architectures, and provides an open online platform to support such iterative development. The first iteration includes five benchmarks covering a broad spectrum of data-centric technologies, tasks, and modalities in vision, speech, data acquisition, debugging, and diffusion prompting, and supports community contributions of new benchmarks. All these benchmarks, the online evaluation platform, and baseline implementations are open-source, and the MLCommons Association will maintain DataPerf to deliver long-term benefits for both academia and industry.