遇见数据集

Physics as the label for measuring and correcting materials reasoning in multimodal models

收藏
Zenodo2026-08-14 更新2026-08-20 收录
官方服务:

资源简介:

Vision-language and language models increasingly interpret materials data, yet benchmarks report that they hallucinate invalid properties and violate physical law. Evaluation matches final answers to scarce human labels, while discovery agents verify final proposals or density functional theory (DFT) execution. Neither measures the physical consistency of a model's reasoning chain. Materials data carries its own physics, making a large class of materials reasoning verifiable without annotation. We introduce MatPCR, a label-free benchmark whose programmatic oracles check diffraction geometry through Bragg's law, scale bars, spectral peaks, and DFT-grounded stability, metallicity and property magnitude. We define the Physical-Consistency Rate over image and structure inputs; introduce Constraint-Grounded Self-Verification, an agentic loop whose gain survives self-refinement and equal-compute re-prompting controls; release an open verifier useful in distribution but near chance on all six held-out constraint types; and derive an exact identity for how oracle error displaces the reported rate.

提供机构:
Zenodo
创建时间:
2026-08-06
二维码
社区交流群
二维码
科研交流群
商业服务