遇见数据集

AERO-RAG replication package: a volume-matched comparison of parent-document and flat retrieval over aeronautical safety documentation

收藏
Zenodo2026-09-30 更新2026-10-01 收录
官方服务:

资源简介:

Replication package for a volume-matched comparison of parent-document and flat retrieval in retrieval-augmented generation over aeronautical safety and support documentation.Contents: the evaluation set of 292 queries in 8 behavioural categories; the per-query results of every run of every arm (8 arms x 3 runs x 292 queries, collected in a single session on 2026-09-06, with untruncated answers); the LLM-judge rubric scores; and the analysis and figure-generation scripts, which regenerate the figures, tables and numerical macros of the paper from the raw results without network access or API credentials.The package has two layers. The analysis layer runs offline and was verified by extracting the archive into an empty directory and running it there. The collection layer is included so that the design is auditable, but re-running it requires an OpenAI API key and the retrieval index, which are not redistributed.Start with README.md. Requires Python 3.11+ and the packages listed in requirements.txt. v1.0.2 (2026-09-20): figure generator (aerorag/evaluation/gerar_figuras.py) updated; figures redrawn for legibility and colour-vision accessibility. All numerical results, tables and macros are unchanged. v1.0.3 (2026-09-29): the rubric Wilcoxon tests now rank paired differences of integer per-query sums across runs; the previous implementation used floating-point means, which broke ties (total rubric p = 0.415, previously reported as 0.398; conclusions unchanged). README updated with the current manuscript title; results/generalizacao.json refreshed (auxiliary fields only).

提供机构:
Zenodo
创建时间:
2026-09-30
二维码
社区交流群
二维码
科研交流群
商业服务