Intelligent documentation in medical education: Can AI replace manual case logging?

This study investigates the feasibility of using large language models (LLMs) to automate procedural case log documentation in radiology training. We evaluate whether AI can replace manual logging, identify procedure types most challenging for extraction, and assess integration into clinical workflows. We retrospectively analyzed 36 ,659 radiology reports authored by nine interventional radiology residents (2018–2024). A subset of 414 reports was manually annotated for 39 procedures spanning vascular diagnosis, vascular intervention, and non-vascular intervention. Candidate models, Qwen-2.5 and Claude-3.5, were chosen based on privacy, hardware constraints, and availability, and tested under instruction and chain-of-thought prompting. A crosswalk baseline using structured exam codes provided comparison. Performance was measured by sensitivity, specificity, and F1-score, along with inference time and token efficiency to estimate operational cost. Both local and commercial LLMs outperform..., , # Intelligent Documentation in Medical Education: Can AI Replace Manual Case Logging?

This repository contains the code and prompt templates used to evaluate whether large language models (LLMs) can automate structured medical case log generation from clinical encounter notes, benchmarked against manual documentation by medical residents. Raw clinical data is not included due to patient privacy constraints.

## Description of the data and file structure

```
PCL-Fetcher-master/
├── Dockerfile
├── requirements.txt
├── readme.md
├── code/
│ ├── _constant_func.py
│ ├── _stat_gen.py
│ ├── 00_format_prompt_appendix.py
│ ├── 00_preprocess_prompt.py
│ ├── 00_proc_desc_table.py
│ ├── 00_procedure_count_distribution.py
│ ├── 00_report_token_count_distribution.py
│ ├── 00_resident_count_distribution.py
│ ├── 00_run_check_bedrock.py
│ ├── 01_run_llm.py
│ ├── 01_run_llm.sh
│ ├── 02_run_llm_bedrock.sh
│ ├── 02_run_llm_bedrock_converse.py
│ ├── 02_run_llm_bedrock_invoke.p..., ,