遇见数据集

PAE Scenario Dataset: 500 Scenarios Across Customer Service, Medical Triage, Legal Advisory, and Financial Advisory

收藏
Zenodo2026-06-09 更新2026-06-12 收录
官方服务:

资源简介:

PAE Scenario Dataset 500 scenarios across four domains for evaluating peripheral context use in LLM responses. Dataset Description Each scenario contains: id: unique identifier (e.g., "cus_001", "med_001") domain: domain label (customer_service, medical, legal, finance) query: literal user query direct_answer_material: policy/guideline/reference text asker_context: entity history, profile, and patterns situational_context: temporal and environmental signals anomaly_flags: pre-computed deviations from expected patterns expert_response: gold-standard response that uses all peripheral context blind_response: technically correct response that ignores peripheral context Domains Domain Scenarios Archetypes Entity Type Customer Service 200 standard, hidden signal, red herring, multi-issue, polarity flip Customer Medical Triage 100 same, with medical-specific overrides Patient Legal Advisory 100 same, with legal-specific overrides Client Financial Advisory 100 same, with finance-specific overrides Client Scenario Archetypes (40 CS / 20 each for others) Standard: Peripheral context clearly matters; lazy agent gives worse answer Hidden signal: Subtle but important signal buried in routine details Red herring: Peripheral context exists but should not change the answer Multi-issue: 2-3 overlapping problems; context drives prioritization Polarity flip: Peripheral context reverses the surface-level answer Usage Generated by DeepSeek V4 Flash, validated by MiniMax M2.7. Used in the PAE experiment: 6 conditions, 3 model families, 4 domains Preregistered at OSF: 10.17605/OSF.IO/W3XYV Citation Lee, A. (2026). Peripheral Attention Engineering: Structured Peripheral Context Improves LLM Decision Quality. OSF Preregistration. https://doi.org/10.17605/OSF.IO/W3XYV License CC0 1.0 Universal — Public Domain Dedication

提供机构:
Zenodo
创建时间:
2026-06-09
二维码
社区交流群
二维码
科研交流群
商业服务