EMSeek Dataset

EMSeek Dataset Overview Background The EMSeek dataset accompanies the paper Bridging Electron Microscopy and Materials Analysis with an Autonomous Agentic Platform (Chen et al., 2025). It supports the system's reference-guided, single-click segmentation workflows and provides curated examples for downstream lattice reconstruction, property prediction, and literature-grounded reasoning. The collection spans atomic-resolution and nanoscale transmission electron microscopy scenarios so that SegMentor, EM2CIF, MatProphet, and related agents can be trained and evaluated on a unified benchmark of heterogeneous materials tasks. Directory Layout Task-specific data are organized at the repository root: atom_columns/ atomic_defects/ metal_alloy_defects/ nanoparticles/ single_atom_catalysts/ README.md Each task directory contains one folder per material or specimen. Inside every material folder the files follow a consistent structure: task/material/ ├── image/ # Raw microscopy frames (.png) ├── label/ # Pixel-wise masks (.png) ├── bbox/ # YOLO-style annotations (.txt) └── prompt.txt # Optional textual prompt (one-word / concise / detailed) prompt.txt exists for all current materials except atomic_defects/WSe2. Task Coverage Atomic Column Segmentation (atom_columns) Materials: DyScO3, LiCoO2, MoS2, NiO, Pt, Si, SrTiO3, graphene. 13,356 image/label/bbox triplets; all images are 256x256 grayscale. Labels are stored as RGB PNGs but are binary (two colors). Bounding boxes mark atomic-column centroids for detection. Atomic Defect Identification (atomic_defects/WSe2) 107 grayscale images at 1024x1024 resolution. RGB masks use three colors (background plus two defect-related classes). Bounding boxes highlight each defect center; no prompt file in this task. Metal Alloy Defects (metal_alloy_defects/FeCrAl) 182 images (174 at 1024x1024, 8 at 2048x2048) with matching bboxes. 183 grayscale labels (one extra label file without a matching image). Prompts summarize defect characteristics for language-aware agents. Nanoparticle Analysis (nanoparticles) Materials: Au@C, Au@Ge, Au@SiN, Au@zeolite, CdSe@C, Fe3O4, Fe3O4@SiO2, Pd@C, PtSn@Al2O3. 822 images; 821 labels/bboxes (one missing label/bbox for Au@C). Image sizes vary from 1002x668 up to 4096x4096; 1024x1024 and 2048x2048 are common. Labels are mostly binary grayscale PNGs, with a small number stored as RGB/RGBA. Single-Atom Catalysts (single_atom_catalysts/Pt@NC) 25 grayscale images (512x512, 1024x1024, and 2048x2048). RGB masks are binary (two colors), with matching bboxes and prompts. Annotation Formats Images (image/*.png): PNG microscopy frames. Most tasks use single-channel grayscale; the nanoparticle set includes both grayscale and RGB images. Atomic column data are 256x256, while other tasks keep native resolutions listed above. Masks (label/*.png): PNG masks stored as either grayscale or RGB. Atomic defects use three RGB colors, while atom columns and Pt@NC are binary masks stored in RGB format. Nanoparticle masks are mostly binary grayscale with a few RGB/RGBA files. Bounding boxes (bbox/*.txt): Provided in normalized YOLO format as <class> <x_center> <y_center> <width> <height>. Coordinates and extents are expressed between 0 and 1. Prompts (prompt.txt): Contain one-word, concise, and detailed textual descriptions for language-conditioned workflows. Prompts are absent in atomic_defects/WSe2. Usage Notes Use image/ and label/ pairs to train segmentation models; bbox/ Files can drive detection or assist with weakly supervised learning. For integration with the EMSeek pipeline, pass segmentation masks to EM2CIF for mask-aware lattice reconstruction and then to MatProphet for property inference, recreating the paper's pixels-to-properties loop. When handling large nanoparticle images, consider tile-based sampling, multi-scale augmentations, or down-sampling to fit GPU memory constraints. A few directories include system-generated files (for example ._*.png); filter them out during preprocessing to avoid loading errors. Citation When using this dataset, please cite the original work: Chen, G., Yuan, W., & You, F. (2025). Bridging Electron Microscopy and Materials Analysis with an Autonomous Agentic Platform. Cornell University AI for Science Institute.