Chrisyichuan/text-qa-pair

--- license: mit task_categories: - question-answering - text-retrieval language: - en pretty_name: Text QA Pairs With Filtered Hard Negatives size_categories: - 10K<n<100K --- # Text QA Pair Dataset With Filtered Hard Negatives This dataset contains English Wikipedia text QA pairs together with retrieval candidates and LLM-filtered hard negatives. The source pairs were first mined with text retrieval (`top20` candidates per query), then filtered with an LLM: 1. Check whether the positive passage can answer the query. 2. Scan non-positive retrieval candidates in rank order. 3. If a candidate passage is judged `CORRECT`, it is treated as a false negative and skipped. 4. If a candidate passage is judged `WRONG` or `CANNOT_ANSWER`, it is kept as a hard negative. 5. Keep examples only when at least 7 filtered hard negatives are found. ## Summary - Total chunks: `10` - Input examples: `18,572` - Kept examples: `14,952` - Skipped examples: `3,620` - API errors during filtering: `0` - Skipped because not enough hard negatives: `3,504` - Skipped because the positive passage was not judged correct: `116` ## Directory Layout The dataset is sharded by row range: ```text chunk_000000_001999/ chunk_002000_003999/ ... chunk_018000_018571/ ``` Each chunk contains: - `filtered_hn.jsonl`: final kept examples for training - `candidate_reviews.jsonl`: per-candidate LLM review logs - `summary.json`: chunk-level aggregate statistics ## `filtered_hn.jsonl` Format Each line is one JSON object. Main fields: - `query`: natural-language question - `answer`: short answer from the source generation step - `passage`: positive passage text - `article_id`: positive article ID - `chunk_index`: positive chunk index within the article - `source_sentence`: supporting sentence used during pair generation - `source_type`: source type metadata - `positive_rank`: rank of the positive chunk in retrieval top-20 - `positive_score`: retrieval score of the positive chunk - `retrieve_top20`: raw top-20 retrieval candidates - `neg_hits`: filtered hard-negative candidates that survived LLM filtering - `neg_passages`: text-only view of `neg_hits` - `source_positive_rank`: copied positive rank metadata from the mining stage - `source_positive_score`: copied positive score metadata from the mining stage `neg_hits` entries contain: - `rank` - `score` - `article_id` - `chunk_index` - `char_offset` - `n_tokens` - `title` - `url` - `text` ## `candidate_reviews.jsonl` Format This file stores the LLM filtering trace for auditing: - one row for the positive passage review - one row for each inspected candidate passage Common fields include: - `example_index` - `query` - `candidate_rank` - `candidate_article_id` - `candidate_chunk_index` - `candidate_score` - `candidate_title` - `candidate_url` - `answer` - `verdict` - `path_role` Possible verdicts: - `CORRECT` - `WRONG` - `CANNOT_ANSWER` - `API_ERROR` ## Example Example `filtered_hn.jsonl` record shape: ```json { "query": "Who was the first Hong Kong athlete to win medals in two different Olympic Games?", "answer": "Lee Wai-sze", "article_id": 4353883, "chunk_index": 1, "positive_rank": 14, "positive_score": 0.6786706447601318, "neg_hits": [ { "rank": 1, "score": 0.7558059096336365, "article_id": 3427201, "chunk_index": 0, "title": "Hong Kong at the Olympics", "url": "https://en.wikipedia.org/wiki/Hong_Kong_at_the_Olympics", "text": "..." } ], "neg_passages": [ "Hong Kong at the Olympics ..." ] } ``` ## Notes - This release is stored in chunked form to make long-running filtering and uploads resumable. - `filtered_hn.jsonl` is the training-ready output. - `candidate_reviews.jsonl` is mainly for inspection, auditing, and debugging false-negative filtering behavior.--- 许可证：MIT协议 任务类别： - 问答（question-answering） - 文本检索（text-retrieval） 语言：英语（en） 数据集名称：带过滤硬负样本的文本问答对数据集（Text QA Pairs With Filtered Hard Negatives） 样本规模类别：10K < n < 100K --- # 带过滤硬负样本的文本问答对数据集（Text QA Pair Dataset With Filtered Hard Negatives） 本数据集包含英语维基百科文本问答对，附带检索候选集与经大语言模型（Large Language Model，LLM）过滤的硬负样本。 该数据集的源问答对首先通过文本检索进行挖掘（每个查询对应top20个候选样本），随后经大语言模型过滤： 1. 验证正样本段落是否能够回答对应查询。 2. 按排序顺序遍历非正样本检索候选集。 3. 若候选段落被判定为`CORRECT`，则视为假负样本并跳过。 4. 若候选段落被判定为`WRONG`或`CANNOT_ANSWER`，则将其作为硬负样本保留。 5. 仅当至少筛选出7个硬负样本时，才保留该示例。 ## 统计摘要 - 总块数：`10` - 输入示例总数：`18,572` - 保留示例数：`14,952` - 剔除示例数：`3,620` - 过滤过程中API错误数：`0` - 因硬负样本数量不足被剔除：`3,504` - 因正样本段落未被判定为正确被剔除：`116` ## 目录结构 本数据集按行范围进行分片存储： text chunk_000000_001999/ chunk_002000_003999/ ... chunk_018000_018571/ 每个分片包含以下文件： - `filtered_hn.jsonl`：用于训练的最终保留示例集 - `candidate_reviews.jsonl`：单候选样本的大语言模型审核日志 - `summary.json`：分片级聚合统计信息 ## `filtered_hn.jsonl` 格式说明 每行对应一个JSON对象，主要字段如下： - `query`：自然语言查询问题 - `answer`：源生成步骤产出的简短答案 - `passage`：正样本段落文本 - `article_id`：正样本所属文章ID - `chunk_index`：正样本段落在所属文章中的分片索引 - `source_sentence`：问答对生成时使用的支撑句 - `source_type`：源类型元数据 - `positive_rank`：正样本段落在top20检索结果中的排名 - `positive_score`：正样本段落的检索得分 - `retrieve_top20`：原始top20检索候选集 - `neg_hits`：经大语言模型过滤后保留的硬负样本候选集 - `neg_passages`：`neg_hits`的纯文本形式 - `source_positive_rank`：挖掘阶段保留的正样本排名元数据 - `source_positive_score`：挖掘阶段保留的正样本得分元数据 `neg_hits`条目包含以下字段： - `rank`：排名 - `score`：得分 - `article_id`：文章ID - `chunk_index`：分片索引 - `char_offset`：字符偏移量 - `n_tokens`：Token数（Token） - `title`：标题 - `url`：链接 - `text`：文本内容 ## `candidate_reviews.jsonl` 格式说明 该文件用于存储大语言模型过滤的审核轨迹： - 1行对应正样本段落的审核记录 - 每个被审核的候选样本对应1行记录 通用字段包括： - `example_index`：示例索引 - `query`：查询问题 - `candidate_rank`：候选样本排名 - `candidate_article_id`：候选样本所属文章ID - `candidate_chunk_index`：候选样本分片索引 - `candidate_score`：候选样本检索得分 - `candidate_title`：候选样本标题 - `candidate_url`：候选样本链接 - `answer`：答案 - `verdict`：审核结论 - `path_role`：路径角色元数据 可选审核结论包括： - `CORRECT`（正确） - `WRONG`（错误） - `CANNOT_ANSWER`（无法回答） - `API_ERROR`（API错误） ## 示例 `filtered_hn.jsonl` 的示例记录格式如下： json { "query": "Who was the first Hong Kong athlete to win medals in two different Olympic Games?", "answer": "Lee Wai-sze", "article_id": 4353883, "chunk_index": 1, "positive_rank": 14, "positive_score": 0.6786706447601318, "neg_hits": [ { "rank": 1, "score": 0.7558059096336365, "article_id": 3427201, "chunk_index": 0, "title": "Hong Kong at the Olympics", "url": "https://en.wikipedia.org/wiki/Hong_Kong_at_the_Olympics", "text": "..." } ], "neg_passages": [ "Hong Kong at the Olympics ..." ] } ## 说明 - 本数据集以分片形式存储，以便长时间运行的过滤与上传任务可断点续传。 - `filtered_hn.jsonl` 为可直接用于训练的输出文件。 - `candidate_reviews.jsonl` 主要用于审核、检查以及调试假负样本过滤逻辑。