telepix/STELLA

STELLA是一个航空航天领域的信息检索（IR）基准数据集，基于NASA技术报告服务器（NTRS）的文档构建。它旨在评估检索器的词汇匹配能力（是否受益于精确的技术术语）和语义匹配能力（即使未明确使用技术术语，是否能匹配概念）。数据集包含约240万英文段落和1000个评估查询（500个词汇匹配查询和500个语义匹配查询），并支持多种语言查询（英语、韩语、印尼语、泰语、法语、中文和日语）。数据来源经过筛选，仅包括2000年及以后发布的英文PDF文档，排除了非文本中心类型（如视频/海报）的材料，并选择了不受版权保护的内容。数据处理包括文档布局检测（排除图表）和使用递归令牌分块器（RT100-20）将文档分块为段落（块大小为100个令牌，重叠20个）。

STELLA is an aerospace-domain Information Retrieval (IR) benchmark constructed from NASA Technical Reports Server (NTRS) documents. It is designed to evaluate both lexical matching ability (does the retriever benefit from exact technical terms?) and semantic matching ability (can the retriever match concepts even when technical terms are not explicitly used?). The dataset contains approximately 2.4 million English passages and 1,000 evaluation queries (500 for lexical matching and 500 for semantic matching), with support for multiple query languages (English, Korean, Indonesian, Thai, French, Chinese, and Japanese). The data source is filtered to include only English PDF documents published in or after 2000, excluding non-text-centric types (e.g., video/poster), and selecting materials that are not under copyright. Data processing involves document layout detection (excluding figures/tables) and chunking documents into passages using a Recursive-Token-Chunker with RT100-20 (chunk size 100 tokens, overlap 20).