Intelligent Causal Reasoning and Summarization of Incident Reports with Large Language Models

# LLM-based Structured Summarization and Causal Analysis Framework ## 1. Overview This artifact provides the implementation of a structured framework for automatic summarization and causal analysis of complex textual artifacts using large language models (LLMs). The framework aims to transform long, heterogeneous, and unstructured documents into concise, structured, and interpretable representations. Specifically, the system generates:- Structured summaries following a predefined schema- Explicit causal chains representing multi-step cause–effect relationships The artifact supports a comprehensive experimental evaluation covering summarization quality, causal extraction accuracy, interpretability, and component-level analysis. --- ## 2. Methodology Summary The framework consists of three main stages: ### (1) Input SpecificationRaw textual documents are preprocessed and transformed into structured inputs. This includes:- Text normalization and noise removal- Metadata integration (e.g., date, location, type)- Structured prompt construction ### (2) LLM-based ReasoningA large language model (DeepSeek-V3.2) performs:- Structured summarization based on a predefined schema- Causal chain extraction, including direct and indirect relations- Optional iterative refinement to improve consistency and completeness ### (3) Output and EvaluationThe system produces JSON-formatted outputs and evaluates them using:- ROUGE metrics for summarization quality- F1-score for causal extraction accuracy- Interpretability measures based on structural validity and entity consistency --- ## 3. Artifact Contents The artifact includes the following components: - `main.py`: Main script for running all experiments- `dataset.json`: Input dataset for evaluation- `README.md`: Documentation (this file) --- ## 4. Dataset Description The dataset used in this artifact consists of two parts: ### Dataset 1: Real-world Reports- Source: Public industrial and safety-related reports \cite{Zhang2024MiningReports,ApplicationMiningNLP2025}- Size: 1,200 samples- Split: 1,000 for training/validation, 200 for testing- Content: - Narrative descriptions - Event processes - Causal factors - Recovery procedures ### Dataset 2: Scenario-based Corpus- Source: Publicly available structured text corpora \cite{CrashNarratives2024}- Size: 800 samples- Split: 640 training/validation, 160 testing- Content: - Structured event descriptions - Human-written summaries Each sample includes:- Raw text- Metadata (date, location, type)- Reference summary- Ground-truth causal relations- Extracted entities --- ## 5. Data Format Each data instance follows the structure: ```json{ "text": "preprocessed report text", "raw_text": "original report text", "metadata": { "date": "YYYY-MM-DD", "location": "Location", "type": "Incident type" }, "reference_summary": "ground truth summary", "gold_causal": [ {"cause": "...", "effect": "..."} ], "entities": ["entity1", "entity2"]}