CT-Agent: a multimodal-LLM agent for 3D CT radiology question answering

Computed tomography (CT) scans can produce 3D volumetric medical data, which is viewed as hundreds of cross-sectional images (slices) and provides detailed anatomical information for diagnosis.Creating CT radiology reports is time-consuming and error-prone for radiologists.A visual question answering (VQA) system is needed to answer radiologists' anatomical questions about CT scans and to automatically generate radiology reports.However, existing VQA systems cannot adequately handle the CT radiology question answering (CTQA) task due to anatomic complexity, which makes CT images difficult to understand, and spatial relationships across hundreds of slices, which are difficult to capture.To address these challenges, this study proposes CT-Agent, a multimodal agentic framework for CTQA. CT-Agent uses anatomically independent tools to break down anatomic complexity and captures across-slice spatial relationships via global-local token compression. Experimental results on the CT-RATE and RadGenome-Chest CT datasets verify its superior performance.