遇见数据集

Reproduction Artifact - RAG-RETEST: Dependency-Scoped Test Selection and Paired Regression Validation

收藏
Zenodo2026-10-30 更新2026-11-05 收录
官方服务:

资源简介:

This package supports reproducibility of RAG-RETEST: Dependency-Scoped Test Selection and Paired Regression Validation, a study of regression test selection, prioritization, and paired validation for retrieval-augmented generation applications. It includes frozen source code, original input catalogs, raw generated answers, anonymized accepted review fields, saved analyses, a matching manuscript and supplement, and reproduction and verification scripts. The evidence covers: Experiment A: 1,296 policy executions. A stratified answer audit covering 1,200 outputs. Experiment B: 981 generated outputs and 51 selected reviewed cases. Experiment C: separate Haystack/Controlled evaluations on Google Colab and Kaggle, each containing 144 timing-qualified policy executions. Saved-output analyses can be reproduced offline on an ordinary CPU without a GPU, model download, API account, or paid service. Human judgments are preserved observations; reproduction does not generate new human assessments. Optional fresh inference requires additional dependencies and model downloads, and may produce different answers or timings. Cloud platforms are analyzed separately. Three additional cohorts per platform remain unrun. Timing drift, uncertain judgments, and unfavorable policy outcomes are retained explicitly. The evidence does not establish universal policy superiority, industrial effectiveness, or energy or monetary savings. Artifact-owned software is licensed under MIT. Released datasets, including SQuAD adaptations, use CC BY-SA 4.0. The reference manuscript and external components retain their respective rights, as documented in the package.

提供机构:
Zenodo
创建时间:
2026-10-30
二维码
社区交流群
二维码
科研交流群
商业服务