遇见数据集

PM-1 V0.5: Skip-Based State Transmission for Long-Horizon Agent Handoffs - Benchmark Data

收藏
Zenodo2026-08-19 更新2026-08-20 收录
官方服务:

资源简介:

Raw experimental data for the PM-1 V0.5 benchmark (Pro Memoria). PM-1 is a handoff architecture for sequential agent workflows organized around the principle 'skip, don't compress': known or unnecessary history is not retransmitted across fresh-worker handoffs; each worker receives a bounded continuation-state packet containing the state required to continue. What V0.5 tested: bounded PM-1 state packet transmission (condition P) against accumulating conversational history (condition C) on the same deterministic synthetic portfolio/state task, with identical model, initial state, schedule, oracle, and success criteria, and fresh worker context per hop. Executed scope: 26,100 completed API-backed handoffs. Completed horizons: H10, H25, H50, H100, H250 (600 / 1,500 / 3,000 / 6,000 / 15,000 calls respectively). H500 and H1000 were registered but NOT executed; 90,000 registered calls remain unexecuted. Execution was intentionally stopped after H250, which produced sufficient evidence for the scoped conclusion. Principal result (H250): P = 3,672,305 cumulative tokens; C = 38,366,105 cumulative tokens; 90.43% cumulative-token reduction relative to the accumulating conversational condition (a difference between two handoff strategies, not a compression ratio). P reliability: 99.98% successfully formatted worker responses; 100% state integrity; 100% digest continuity; 3 PARSE_ERROR; 0 state divergence. V0.5 overall: 5 non-PASS records (3 PARSE_ERROR, 1 ACTION_ERROR, 1 STATE_CORRUPTION); the two state-related failures occurred in condition C at H100. Limitations: synthetic deterministic task; simple state representation; single model (deepseek-v4-flash); single provider; temperature 0; accumulating-conversation baseline only (no compaction, summarization, or retrieval baselines); no evolving institutional knowledge; no heterogeneous models; finite measured horizon H10-H250. See the repository README and the published consolidation report for full details. This dataset accompanies the public repository https://github.com/rikirinjani/pro-memoria (research/pm1-v05).

提供机构:
Zenodo
创建时间:
2026-08-19
二维码
社区交流群
二维码
科研交流群
商业服务