Task-Conditioned Performance Variance Decomposition of Large Language Model Design

Repository Overview

This repository contains raw data and experimental results. At the top level, you will find three folders:

data/ – Contains the core datasets used in the project, including both raw and cleaned versions.

results/ – Contains experimental results and their visualization.

data/

df_long_accuracy.csvContains structured meta-features and evaluation results for large language models across multiple benchmarks and evaluation settings. Each row corresponds to a model–dataset–evaluation setting combination, linking scale, architectural, and training meta-features with an observed performance value.

df_long_accuracy_final.parquetThe cleaned and preprocessed version of the dataset. The notebooks used to generate this file can be found in the notebooks/ directory on the software link below. This file is used in downstream experiments.

Other filesThe remaining files in this directory consist of visualizations derived from the above data and generated using the notebooks.

results/

importance_accuracy.csvContains meta-feature importance results generated using fANOVA. Each row represents the importance of a single meta-feature or pair of meta-features for a given task (evaluation setting–dataset combination).

marginals/Contains marginal plots for each of the meta-features and individual tasks.

labels_6-cosine-complete.csvContains clustering results of tasks based on fANOVA embeddings.

Other filesThe remaining files in this directory consist of visualizations derived from the above data and generated using the notebooks.Software link: https://anonymous.4open.science/r/llm-fanova-analysis-BC18

仓库概览
本仓库包含原始数据与实验结果。顶层目录共包含三个文件夹：

data/ — 存放项目所用的核心数据集，涵盖原始版本与清洗后版本。

df_long_accuracy.csv：收录了大语言模型（Large Language Model, LLM）在多基准测试与多种评估设置下的结构化元特征与评估结果。每一行对应一组模型-数据集-评估设置组合，将模型规模、架构及训练相关元特征与观测得到的性能指标进行关联。

df_long_accuracy_final.parquet：该数据集的清洗与预处理版本。用于生成此文件的Jupyter笔记本可在下方软件链接对应的notebooks/目录中获取。该文件将用于下游实验。

该目录下其余文件均为基于上述数据生成的可视化产物，由对应笔记本脚本生成。

results/ — 存放实验结果及其可视化内容。

importance_accuracy.csv：包含使用fANOVA生成的元特征重要性计算结果。每一行代表针对某一特定任务（评估设置-数据集组合）的单个元特征或元特征对的重要性值。

marginals/ — 存放各元特征与单个任务的边缘分布可视化绘图结果。

labels_6-cosine-complete.csv：包含基于fANOVA嵌入得到的任务聚类结果。

该目录下其余文件均为基于上述数据生成的可视化产物，由对应笔记本脚本生成。

软件链接：https://anonymous.4open.science/r/llm-fanova-analysis-BC18