遇见数据集

GenSIE Benchmark v1.0: Schema-Guided Information Extraction with Small Language Models in Spanish

收藏
Zenodo2026-07-10 更新2026-08-02 收录
官方服务:

资源简介:

A Spanish-language benchmark for schema-guided information extraction with small language models (3B–14B parameters). Given a passage, a natural-language instruction, and a target JSON Schema, a system must emit a JSON object that validates against the schema and faithfully reflects the text — including returning null for fields the text does not support (a null-as-ground-truth contract that penalises parametric hallucination). The release contains 334 self-contained instances across three splits: a 40-instance starter kit, a 149-instance development set, and a 145-instance held-out gold test set, spanning nine domains and complexity-tagged schemas. Each instance carries its own text, instruction, target schema, gold answer, and metadata. Per-leaf grounding annotations (evidence spans and rationale) are included for all 149 development instances and 69 of the test instances. Every gold output validates against its target schema. The benchmark accompanies the GenSIE @ IberLEF 2026 shared-task overview paper. Documentation (format specification, schema catalogue, source provenance) is bundled; code (task harness, reference baseline, evaluator, schema definitions) is at gia-uh/gensie-internal.

提供机构:
Zenodo
创建时间:
2026-07-10
二维码
社区交流群
二维码
科研交流群
商业服务