Dataset - Multimodal MRI–HSI synthetic brain tissue dataset based on agar phantoms

The Multimodal Agar Brain Phantom Dataset provides a collection of co-registered imaging data acquired from brain-mimicking phantoms designed for research in multimodal medical image registration, tissue characterization, and surgical guidance. The dataset includes two phantom generations (Phantom v1 and Phantom v2), comprising 13 phantoms in total with different tissue configurations and acquisition workflows. Each phantom reproduces four brain tissue types: white matter, gray matter, tumor, and blood vessels, which are constructed using controlled agar concentrations and colorants. The phantoms incorporate fiducial markers, ArUco tags (in v2), and characteristic container geometries enabling precise multimodal alignment. The dataset contains HSI (hyperspectral imaging), RGB, depth, and MRI data, together with the 3D models of the phantom containers, calibration files, and ground-truth labels for both MRI and HSI modalities. For Phantom v2, an additional set of tracked points on the phantom surface enables direct MRI-to-tracking registration. Contents 1. Hyperspectral Imaging (HSI) Raw captures acquired at 8 exposure times (20–200 ms). 24 spectral bands between 665 - 950 nm, White-balance images and processed reflectance hypercubes. Ground-truth tissue labels. Intrinsic camera parameters and tracking calibration. 2. RGB-D Cameras Raw color, depth, IR, and stereo captures (depending on the device). Metadata files describing dimensions and data formats. Intrinsic and extrinsic calibrations for each sensor. Calibration matrices relating each camera to the tracking system. 3. Magnetic Resonance Imaging T1-weighted, T2-weighted, and Proton Density sequences. Axial and coronal slices. Tissue labels for all phantoms. 4. 3D Models and Registration Data Full 3D models of the phantom containers. Fiducial positions and tracked probe measurements. ArUco marker coordinates (Phantom v2). Calibration matrices for multimodal registration pipelines. Phantom Versions Phantom v1 5 phantoms. Registration based on fiducial markers tracked with an optical tracking system. MRI-to-camera alignment performed through the container’s fiducial geometry. Phantom v2 8 phantoms (3 normal, 5 inverted layers). ArUco markers printed on the container for direct camera-to-phantom registration. Direct MRI-to-tracking alignment through localized surface points on the phantom. Simplified acquisition workflow to reduce drying effects in agar tissues. Applications This dataset is designed to support research in: Multimodal image registration (MRI ↔ HSI ↔ RGB-D ↔ 3D models). Tissue classification and hyperspectral tissue characterization. Validation of surgical navigation pipelines. Development and benchmarking of medical imaging algorithms. Synthetic data generation and domain adaptation studies. Code Included in the Repository Includes a set of scripts and tools that facilitate working with the dataset: • MRI Labeling Tool (Dockerized) A ready-to-use labeling tool as a Docker image.It allows interactive MRI annotation and export of final label masks in the dataset format. • dataclass.py A structured Python dataclass defining the dataset hierarchy, phantom metadata, and consistent access to multimodal components (HSI, MRI, depth, 3D models, calibration data). • loaders.py Modular data loaders for: MRI slices HSI captures RGB-D data 3D modelsThese loaders ensure unified and reproducible dataset access. • utils.py Helper functions for: Loading .bin files Handling calibration matrices Parsing metadata These scripts are provided to simplify dataset usage and accelerate the development of multimodal pipelines.