A benchmark for identifying chemical structures from images and generating correct IUPAC names with explicit evaluation rules.

The IUPAC Multimodal Naming Benchmark is a chemistry-focused benchmark for evaluating whether models can infer the correct chemical name directly from structure images.The task requires multimodal understanding of molecular diagrams and application of chemical naming rules to produce the correct IUPAC name. This benchmark is designed for structured evaluation and includes explicit handling rules for stereochemistry, synonyms, aromatic compounds, tautomeric forms, and ionic compounds.

Task

Given an image of a chemical structure, the goal is to predict the correct chemical name.This benchmark is intended for:

multimodal LLM evaluation

chemistry reasoning benchmarks

image-to-text scientific understanding

exact-match and structured evaluation pipelines

Evaluation Policy

Each example contains:

a canonical expected name

optional accepted synonyms

a stereochemistry requirement flag

metadata notes for special cases

Naming rules

Stereochemistry is required when explicitly shown in the image.

Common names are not accepted unless explicitly listed in the accepted synonym list.

Tautomeric alternatives are not accepted unless they match the depicted structure or are explicitly allowed.

Salts and ionic compounds must be named as depicted, including relevant counterions.

Aromatic compounds follow a predefined canonical naming policy, with optional curated aliases.

Dataset Fields

Typical fields include:

id

image_path

expected_name

accepted_names

stereochemistry_required

common_names_accepted

notes

Example Use Cases

evaluate multimodal chemistry models

compare exact-match naming performance

test structured reasoning pipelines

benchmark scientific image understanding systems

No