遇见数据集

Positive-Control-Validated Closed-Loop Evaluation of Vision-Language-Action Models: A Null Result for Parameter-Efficient Fine-Tuning Under Domain Shift

收藏
Zenodo2026-07-11 更新2026-08-01 收录
官方服务:

资源简介:

Data and code accompanying the paper "Positive-Control-Validated Closed-Loop Evaluation of Vision-Language-Action Models: A Null Result for Parameter-Efficient Fine-Tuning Under Domain Shift." OpenVLA-7B was adapted with low-rank adaptation (LoRA) on a pick-and-place task (PickCube-v1, ManiSkill3) and evaluated closed-loop under three conditions (standard, lighting shift, texture shift) across 200/500/1000 demonstrations. Before any policy was judged, the evaluation harness was certified by a scripted positive control that achieved 100% success, so the null result is attributable to the policy rather than a non-functional evaluator. Offline metrics improved and then saturated, yet closed-loop success remained near zero. The null is robust across three adaptation recipes (attention-only LoRA, all-linear LoRA, and a scoped full fine-tune): across three recipes × three scales × three conditions, only one of 27 cells is non-zero (1/25) and every domain-shift cell is 0/25. Contents: the clean demonstration datasets, exact training and evaluation configurations, the seed registry, all training/evaluation code (including the positive-control oracle and the adaptation-recipe robustness experiment), and result summaries.

提供机构:
Zenodo
创建时间:
2026-07-11
二维码
社区交流群
二维码
科研交流群
商业服务