Human-aligned Large Language Models Evaluation for Network Automation

This dataset accompanies the study in which the HALLE (Human-aligned Auto LLM evaluation) framework is applied to a use case involving explainable AI (XAI) outputs for optical network quality-of-transmission (QoT) estimation. It includes the prompt sets used to query the candidate large language models (LLMs), the explanations generated by each model, and the evaluation scores assigned by human experts and by the LLM judges. Human evaluators annotated 40 instances, and these annotations were used to compare all candidate LLM judges. The selected judge then evaluated a set of 100 instances to produce the final scoring used in the paper. The dataset also contains the trained XGBoost QoT prediction model and the SHAP data used to produce the XAI feature attributions. All files are provided to enable exact reproduction of the experiments reported in the associated publication.