ibm-research/LLM_Fine-Tuning_Performance

该数据集包含大语言模型（LLM）微调在各种硬件和软件配置下的性能基准测试，覆盖959个有效配置的吞吐量测量（每秒处理的令牌数），数据收集耗时超过1000 GPU小时，在一个Kubernetes集群上完成。数据集专为预测性能建模研究设计，特别是用于评估处理分类配置空间扩展（CCSE）的方法，即当分类变量引入新值（如新LLM模型、GPU类型、微调方法或软件版本）时配置空间的扩展。数据集包括7个变量（4个分类变量、3个数值变量）：分类变量有LLM模型（如llama2-7b、granite-13b-v2等）、微调方法（全微调、LoRA）、GPU类型（NVIDIA A100-80GB、L40S-48GB）和软件版本（v2.0.0、v2.1.0）；数值变量包括GPU数量、批大小和每个样本的令牌数。数据通过accelerated discovery orchestrator (ado)平台收集，使用SFTTrainer执行器，测量单周期内合成数据集的吞吐量。数据集还支持18个预测任务，用于评估在CCSE下的模型构建方法，任务按扩展变量（LLM、GPU、方法、版本）和泛化水平（广义、狭义）分类。数据集主要用于研究目的，如迁移学习、性能预测和样本高效建模策略。

This dataset contains performance benchmarks for Large Language Model (LLM) fine-tuning across various hardware and software configurations. It includes throughput measurements (tokens per second) for 959 valid configurations, collected over 1000 GPU hours on a Kubernetes cluster. The dataset is designed for research on predictive performance modeling, specifically for evaluating methods that handle Categorical Configuration Space Expansion (CCSE) which occur when new values are introduced for categorical variables (e.g., new LLM models, GPU types, fine-tuning methods, or software versions). It features 7 variables (4 categorical, 3 numerical): categorical variables include LLM (e.g., llama2-7b, granite-13b-v2), Method (full fine-tuning, LoRA), GPU (NVIDIA A100-80GB, L40S-48GB), and Version (v2.0.0, v2.1.0); numerical variables are #GPUs, batch size, and tokens per sample. Data was collected using the accelerated discovery orchestrator (ado) platform with the SFTTrainer actuator, measuring throughput over a single epoch on a synthetic dataset. The dataset supports 18 distinct prediction tasks for evaluating model building methods under CCSE, categorized by expansion variable (LLM, GPU, Method, Version) and generalization level (generalized, specialized). It is intended for research purposes such as transfer learning, performance prediction, and sample-efficient modeling strategies.