Enhanced QORA-OA: Raw results and datasets

Introduction This repository contains the experimental datasets, golden (reference) answers, raw model outputs, LLM-as-a-Judge evaluations, and fault-reproduction scripts for QORA-OA (Query-Optimized Retrieval-Augmented ObservAbility), covering: 1. the conference version of the work (IEEE LCN 2026), and2. its extended version. Summary File Contents ieee-lcn-2026-reference-results.zip Reference results of the conference version: experiments E1–E6, generator models GPT-4o and GPT-5.1, judge panel J1–J4, and the independent human expert evaluation (JH). extension-2026-new-results.zip Full extended evaluation: experiments E1–E6 on the baseline benchmark, AE1/AE2 on the advanced benchmark, and the component ablation study. free5gc-troubleshooting-scenarios.zip Scripts to reproduce every fault scenario (S1–S15) on a live free5gc v4.1.0 deployment on Kubernetes, including the collected telemetry evidence. QORA-OA-datasets.zip Datasets were prepared and utilized to evaluate the QORA-OA system. Experiments (extension-2026-new-results.zip) Experiment Event type Scenarios Queries Repetitions E1 troubleshooting S1–S9 9 5 E2 deep_troubleshooting S1–S9 9 5 E3 telco_observer S10 6 5 E4 static_retrieval S10 6 5 E5 generic S10 5 5 E6 generic S10 5 5 AE1 troubleshooting S11–S15 5 10 AE2 deep_troubleshooting S11–S15 5 10 AB deep_troubleshooting S11–S15 5 5 The structure of the results: <EXP>/├── <EXP>_dataset.json # scenarios + testing queries├── <EXP>_golden-answers.json # reference answers the judges score against├── openai_gpt-5.1/ # generator model 1│ ├── results/ # raw QORA-OA responses, one file per repetition:│ │ # <EXP>_R<r>_generator-<model>.json│ └── evaluations/ # judge scores, one sub-directory per judge:│ # <EXP>_J<j>_judge-<judge>/│ # <EXP>_R<r>_J<j>_reasoning_engine-<model>_judge-<judge>.json└── llm_Qwen3.6-27B/ # generator model 2 (locally hosted open-weight model) ├── results/ └── evaluations/ The ablation (AB) results live under AE2/openai_gpt-5.1/ablations/<configuration>/ for the four configurations: no_decomposition, no_expansion, no_kb, and no_telemetry, each with its results and evaluations and a self-describing ablation_config snapshot inside every result file. Judge panel Judge Model J1, J2 OpenAI GPT-5.1 J3, J4 Google Gemini 2.5 Pro J5, J6 Qwen3.6-27B (locally hosted) JH Human domain experts Attribution The Helm charts in free5gc-troubleshooting-scenarios/charts are modified versions of the towards5gs-helm charts, Copyright (c) 2021 Orange, distributed under the Apache License 2.0; the original copyright and SPDX notices are retained in the chart files. The charts deploy the publicly available free5gc (Apache-2.0) and UERANSIM (AGPL-3.0) container images; no free5gc or UERANSIM source code or binaries are redistributed in this repository.