GenSIE Benchmark v1.0: Schema-Guided Information Extraction with Small Language Models in Spanish
收藏资源简介:
A Spanish-language benchmark for schema-guided information extraction with small language models (3B–14B parameters). Given a passage, a natural-language instruction, and a target JSON Schema, a system must emit a JSON object that validates against the schema and faithfully reflects the text — including returning null for fields the text does not support (a null-as-ground-truth contract that penalises parametric hallucination). The release contains 334 self-contained instances across three splits: a 40-instance starter kit, a 149-instance development set, and a 145-instance held-out gold test set, spanning nine domains and complexity-tagged schemas. Each instance carries its own text, instruction, target schema, gold answer, and metadata. Per-leaf grounding annotations (evidence spans and rationale) are included for all 149 development instances and 69 of the test instances. Every gold output validates against its target schema. The benchmark accompanies the GenSIE @ IberLEF 2026 shared-task overview paper. Documentation (format specification, schema catalogue, source provenance) is bundled; code (task harness, reference baseline, evaluator, schema definitions) is at gia-uh/gensie-internal.



