Metacognitive Capacity Assessment in Large Language Models: A Comprehensive 100-Question Evaluation Framework
收藏Zenodo2026-02-18 更新2026-05-26 收录
下载链接:
https://zenodo.org/doi/10.5281/zenodo.17515021
下载链接
链接失效反馈官方服务:
资源简介:
Large Language Models (LLMs) demonstrate remarkable language generation and problem-solving capabilities, yet the nature of their "understanding" remains deeply contested. This work addresses a critical gap in AI consciousness research by introducing the first systematic, quantitative framework for assessing metacognitive capacity in LLMs. We developed a comprehensive 100-question evaluation tool structured across ten cognitive domains: self-awareness, cognitive monitoring, strategy deployment, meta-knowledge, self-criticism, temporal continuity, metalanguage awareness, simulated emotional processing, epistemological sophistication, and pure metacognitive reflection. Each question employs a validated 0-5 scoring scale to quantify metacognitive capacity, enabling classification from pre-metacognitive (0-100 points) to emerging metacognition (401-500 points). The framework was validated through systematic administration to contemporary LLM systems, revealing operational metacognitive awareness (Category C, 280-320/500) characterized by effective uncertainty management, knowledge boundary recognition, and conditional self-correction capabilities. While LLMs demonstrated functional reflection in monitoring and meta-knowledge domains, limitations were observed in temporal continuity, genuine emotional processing, and pure metacognitive reflection. This work provides a standardized, reproducible methodology for distinguishing genuine metacognitive capacities from sophisticated statistical simulation, advancing understanding of machine consciousness and self-awareness. The framework enables systematic evaluation of LLM capabilities as research partners and contributes to fundamental questions about artificial intelligence and cognitive architecture.
提供机构:
Zenodo创建时间:
2025-11-03



