LPFQA

LPFQA is an authentic, structured, and interdisciplinary dataset with long-tail knowledge for evaluating LLMs’ ability in complex reasoning, providing a robust benchmark for assessing and advancing LLM performance in real-world professional contexts. The main innovations of LPFQA and contributions of this work can be summarized as follows. Innovated evaluation dimension design. We design a set of fine-grained evaluation dimensions, including knowledge depth, reasoning ability, terminology comprehension, and contextual analysis, ensuring LPFQA ’s comprehensiveness in evaluating LLMs’ capabilities in handling complex tasks. Hierarchical difficulty design with guaranteed uniqueness. We employ a tiered difficulty structure to match varying capabilities of different LLMs, while ensuring semantic clarity and answer uniqueness for each task, enhancing the reliability, fairness, and discriminative power of LPFQA. Authentic professional scenario modeling. We ground questions in authentic use cases by constructing detailed user personas and realistic contextual scenarios, enhancing the ability of LPFQA to validate the performance of LLMs in real-world professional environments. Interdisciplinary knowledge integration. We integrate long-tail knowledge from diverse fields, improving the LPFQA’s effectiveness in evaluating LLMs’ integrative capabilities of judgment and reasoning in complex scenarios.