遇见数据集

Replication Data for an Empirical Evaluation of Long-Context Retrieval in Large Language Models under Hard-Negative Interference (1,270 Controlled Trials)

收藏
Zenodo2026-10-29 更新2026-10-29 收录
官方服务:

资源简介:

Replication package for an empirical evaluation of long-context retrieval in nine large language models (GPT-4o-mini, Gemini 3.6 Flash, Grok 4.6, DeepSeek-V3, Kimi K2.6, GLM-5, Qwen3-7-Max, ERNIE-4.5, Doubao) under hard-negative interference. The archive contains 1,270 controlled trials: a single-needle depth scan (Phase 1, 230 trials), a hard-negative distractor gradient with a fitted two-parameter failure law p(N) = a·e−λN and mixed-effects logistic regression (Phase 2, 326 trials), a RAG-versus-long-context cost–accuracy crossover (Phase 3, 129 trials), complementary ablations (soft negatives, temperature, retriever comparison, multi-hop reasoning, 159 trials), and five mechanism probes E1–E5 (426 trials: RAG under interference, RAG multi-hop synthesis, unanswerable probes, cross-task generalization, real-corpus validation). Included are all experiment runners, the core needle-in-a-haystack library, analysis scripts reproducing every reported statistic (summary counts, failure-law fits, mixed-effects regression, exact McNemar tests), model configuration templates without credentials, and all trial-level data in JSON Lines format together with derived aggregate statistics. Haystacks are synthetic; no external corpus is redistributed.

提供机构:
Zenodo
创建时间:
2026-08-19
二维码
社区交流群
二维码
科研交流群
商业服务