Task-Conditioned Performance Variance Decomposition of Large Language Model Design

Repository Overview

This repository contains raw data and experimental results. At the top level, you will find three folders:

data/ – Contains the core datasets used in the project, including both raw and cleaned versions.

results/ – Contains experimental results and their visualization.

data/

df_long_accuracy.csvContains structured meta-features and evaluation results for large language models across multiple benchmarks and evaluation settings. Each row corresponds to a model–dataset–evaluation setting combination, linking scale, architectural, and training meta-features with an observed performance value.

df_long_accuracy_final.parquetThe cleaned and preprocessed version of the dataset. The notebooks used to generate this file can be found in the notebooks/ directory on the software link below. This file is used in downstream experiments.

Other filesThe remaining files in this directory consist of visualizations derived from the above data and generated using the notebooks.

results/

importance_accuracy.csvContains meta-feature importance results generated using fANOVA. Each row represents the importance of a single meta-feature or pair of meta-features for a given task (evaluation setting–dataset combination).

marginals/Contains marginal plots for each of the meta-features and individual tasks.

labels_6-cosine-complete.csvContains clustering results of tasks based on fANOVA embeddings.

Other filesThe remaining files in this directory consist of visualizations derived from the above data and generated using the notebooks.Software link: https://anonymous.4open.science/r/llm-fanova-analysis-BC18