ExpliCA Dataset

The ExpliCA dataset comprises 100 causal natural language explanations (NLEs), each meticulously paired with a set of causal triples. This dataset was developed to advance research in explainable artificial intelligence, with a focus on understanding and modeling causal relationships in text. The dataset has been structured to enable comprehensive analysis of both original, human-curated explanations and AI-generated explanations, allowing researchers to make direct comparisons between the two. This setup supports a deeper investigation into how causal reasoning is represented in AI-generated content versus human explanations. Original Explanations and Causal Triples Each of the 100 curated explanations is linked with a corresponding set of causal triples, designed to capture key components of the causal relationship: T1: The subject or initiator of the causal relationship. T2: The causal verb or predicate describing the cause-effect connection. T3: The object or effect, representing the outcome of the causal relationship. Generated Explanations In addition to original explanations, the dataset includes several types of generated explanations: Explanations Generated from Triples: Explanations generated directly from the causal triples to assess the potential of automated explanation generation. Explanations Generated from Triples with Reference: Explanations generated from triples that also reference the original explanations, providing additional context and coherence. Human, Automated and LLMa Evaluation Using the REFLEX Framework To evaluate the quality and reliability of explanations, both human and automated evaluations were conducted, as well as evaluation using LLMs as evaluators. Human Evaluation of Original Explanations: Human evaluators assessed the original explanations to establish baseline quality metrics. Human Evaluation of Generated Explanations: Human evaluators reviewed the generated explanations (both from triples alone and with reference to original explanations) for clarity, accuracy, and consistency with causal relationships. The REFLEX framework, as presented in the PhD thesis of Miruna Clinciu, was applied to evaluate both original and generated explanations.ExpliCA数据集包含100条因果型自然语言解释（Natural Language Explanations，NLEs），每条均与一组因果三元组精心配对。本数据集旨在推动可解释人工智能领域的研究，聚焦于理解并建模文本中的因果关系。 该数据集的架构设计支持对人工甄选的原始解释与人工智能生成解释开展全面分析，便于研究人员直接对比两类解释。此设计有助于更深入地探究因果推理在人工智能生成内容与人类解释中的表征差异。 原始解释与因果三元组 100条经人工甄选的解释均配有一组对应的因果三元组，旨在捕捉因果关系的核心构成要素： T1：因果关系的主体或引发者 T2：描述因果关联的谓词或因果动词 T3：代表因果关系结果的客体或效应 生成式解释 除原始解释外，本数据集还包含多类生成式解释： 基于三元组生成的解释：直接从因果三元组生成的解释，用于评估自动解释生成的应用潜力。 带参考的三元组生成解释：在生成时参考原始解释的三元组衍生解释，可提供更丰富的上下文信息与逻辑连贯性。 基于REFLEX框架的人类、自动化及大语言模型评估 为评估解释的质量与可靠性，本数据集开展了人类评估、自动化评估以及以大语言模型（Large Language Model，LLM）作为评估者的评估。 原始解释的人类评估：人类评估者对原始解释进行评审，以确立质量基准指标。 生成式解释的人类评估：人类评估者针对两类生成式解释（仅基于三元组生成的解释，以及参考原始解释生成的解释）的清晰度、准确性及与因果关系的一致性进行审核。 本研究采用了Miruna Clinciu博士论文中提出的REFLEX框架，用于评估原始解释与生成式解释的质量。