PowerMath

This repository contains approximately 10 billion tokens of pretrain data generated using Qwen2.5-14B-Instruct.
The dataset utilizes a [MGA-style](https://arxiv.org/abs/2502.04235) methodology
to create diverse and comprehensive training data from the [MegaMath](https://huggingface.co/datasets/LLM360/MegaMath) dataset. The dataset is available under the Apache 2.0 license.

# Bias, Risks, and Limitations

- This dataset is mainly in English.
- The dataset inherits the biases, errors, and omissions known to exist in data used for seed sources and models used for data generation.
- The dataset is synthetically generated and hence may contain inaccuracies that do not accurately reflect real-world phenomena.
- The synthetic nature of this dataset may limit its ability to generalize to real-world cases.

本仓库包含约100亿个Token（Token）的预训练数据，这些数据由Qwen2.5-14B-Instruct生成。本数据集采用[MGA式](https://arxiv.org/abs/2502.04235)方法，基于[MegaMath](https://huggingface.co/datasets/LLM360/MegaMath)数据集构建多样化且全面的训练数据。本数据集遵循Apache 2.0开源协议发布。

# 偏见、风险与局限性

- 本数据集主要采用英语。
- 本数据集继承了种子数据源以及数据生成所用模型中已知存在的偏见、错误与疏漏。
- 本数据集为合成生成数据，因此可能包含无法准确反映现实世界现象的不准确之处。
- 本数据集的合成特性可能会限制其在现实场景中的泛化能力。