GPT-3 Training Traces
收藏arXiv2025-09-30 收录
下载链接：
https://github.com/NVIDIA/DeepLearningExamples
下载链接
链接失效反馈官方服务：
资源简介：
该数据集是在训练各种GPT-3模型变体时收集的追踪数据，旨在评估模型的表现和执行行为。这些追踪数据是使用PyTorch Kineto工具收集的，包含了在不同并行策略下的详细执行分解。该数据集涵盖了从150亿到1750亿参数规模的模型训练，其任务是对大规模语言模型训练的性能建模与估计。
This dataset consists of tracing data collected during the training of various GPT-3 model variants, designed to evaluate model performance and execution behavior. The tracing data was gathered using the PyTorch Kineto tool and includes detailed execution breakdowns under different parallelization strategies. This dataset covers model training with parameter sizes ranging from 15 billion to 175 billion parameters, and its core task is performance modeling and estimation for large-scale language model training.
提供机构：
NVIDIA
