MatSeg DataSet and Benchmark For Zero-Shot Material States Segmentation From images

This is an old version for the new version see https://zenodo.org/records/11331618

A Dataset and Benchmark for zero-shot segmentation of materials states described in: “Learning Zero-Shot Material States Segmentation, by Implanting Natural Image Patterns in Synthetic Data” Described in https://arxiv.org/pdf/2403.03309.pdf 

See ReadMe in the zip file for technical details.

MatSeg Benchmark 

A benchmark for zero-shot material state segmentation. The benchmark contains 820 real-world images with a wide range of material states and settings. For example: food states (cooked/burned..), plants (infected/dry.), to rocks/soil (minerals/sediment),  construction/metals (rusted, worn),  liquids  (foam/sediment), and many other states in a class-agnostic manner.  The goal is to evaluate the segmentation of material materials without knowledge or pretraining on the material or setting. The focus is on materials with complex scattered boundaries, and gradual transition  (like the level of wetness of the surface). The annotation of the benchmark is point-based and similarity-based. Hence, for each image, we select several points and regions (Figure 4). We group the points of the same materials into the same label, we also define a group of points that have partial similarity. For example points in group A are more similar to points in group B than to points in group C (In case materials A and B are similar to each other but not identical). This approach allows us to capture the complexity of gradual transition and partial similarities in the world. While also enabling dealing with complex scattered and blurry shapes without needing to annotate the full shape which in many cases is unclear or very hard.

Files MatSegBenchmark*.zip

MatSeg synthetic Dataset Samples 

Synthethic dataset of images of materials spread on object surfaces and their segmentation map.

The synthetic dataset is a very big, sample of the dataset as been uploaded.

Files:       MatSegSynthehticDataSample*.zip

The full dataset can be found in this URLS:

https://e.pcloud.link/publink/show?code=kZHCcnZOfzqInb3anSl7xzFBoqCDmkr2JKV

https://icedrive.net/s/SBb3g9WzQ5wZuxX9892Z3R4bW8jw

Generation Script for the synthetic data:

https://github.com/sagieppel/MatSeg-Synthethic-Dataset-Generation-Script

https://zenodo.org/records/10822596