遇见数据集

Amplitude Gating: code and experimental data

收藏
Zenodo2026-10-26 更新2026-10-29 收录
官方服务:

资源简介:

This project is the reproducibility package of the Amplitude Gating paper (arXiv Link). It contains the algorithm, scoring, and analysis code together with the experimental data, preserving the required Python module hierarchy. The main experiments in this release cover six tasks: ConvFinQA dev, Hermes function call, Hermes function call single-turn, Hermes JSON mode single-turn, ToolBench in-domain, and ToolBench out-domain. The September main matrix spans 18 model/task scopes across three models, with 522 configurations and 773,082 evaluation records. GSM8K and choice diagnostics are reported under their respective protocols. RF was retrained on 8,886 samples from the six tasks and evaluated with five-fold cross-validation. The ConvFinQA selectors were retrained and evaluated using 957 training samples and 533 test samples. Paired intervals are conditional on the fixed splits and fitted policies. The complete code, parameters, and per-sample records are in analyses/six_task_refit_20261022/. The numerical definition and saved values of the general confidence feature G_relative are unchanged. Directory layout Location Contents orp_core.py, orp/, adapters/, orp_model_*.py AG/ORP intervention implementations and model interfaces text_gating_v3/, diagnostics/ Inference, task-data preparation, scoring, and diagnostic code mining/, analyze_*.py, future_work/ Selector, feature, gate, and related analysis code results/ Configurations, inputs, reference answers, predictions, scores, and summaries from historical and September experiments reproducibility_addenda/ Paired records, compressed predictions, three-model feature views, decode-only controls, and annotations model_assets/ Configurations, tokenizers, vocabularies, weight indexes, and upstream documentation and licenses for three models datasets/ Public benchmark data and sources of experimental inputs analyses/, data/, numeric/ PFHE/RF data, saved five-fold results, numerical inputs, and aggregation tools environment/, metadata/, provenance/ Environment locks, dependency URLs and hashes, model identities, protocols, and provenance records paper/ The revised paper, source files, and 11 figures RESULT_MAP.json Repository-relative indexes for 30 tables and 11 figures MANIFEST.json, verify_package.py File inventory and SHA-256 verification Per-material licensing This repository is not licensed as a single whole. Use of any folder is governed by the license applicable to that folder. Sources, actual material locations, and original licenses for the seven datasets are listed in the root License.txt and LICENSE_SOURCES.json. Each dataset directory under datasets/ provides License.txt and SOURCE.md. Result data generated for the paper in results/, reproducibility_addenda/, analyses/, analysis_p1_n1319/, data/, and numeric/data/ are released under CC BY-NC-SA 4.0. Each directory includes the full license text and LICENSE_SCOPE.md. Quoted third-party source materials retain their original licenses. Code, model assets, and the paper are governed by their respective applicable terms. Distribution contents and privacy Release archives contain the manifest-listed payload and its manifest, without Git metadata, private backups, or local audit files. Infrastructure connection addresses in provenance metadata are represented by placeholders. Authentication query parameters and credential values in archived ToolBench tool responses, and signed URL queries in archival target text, are removed for distribution. Prepared evaluation inputs, gold action names, saved predictions, scores, and numerical features are preserved. Public benchmark examples and published author contacts retain their source context.

提供机构:
Zenodo
创建时间:
2026-10-26
二维码
社区交流群
二维码
科研交流群
商业服务