rlhn/remove-250K

RLHN是一个级联LLM框架，旨在准确重标记现有IR/RAG训练数据集中的难负例，如MS MARCO和HotpotQA。这个Tevatron数据集（250K训练对）包含原始查询、正例和经过删除每个训练对中的单个错误负例后的难负例。该存储库包含可以用来微调嵌入、ColBERT或多元向量以及重排模型的训练对。原始数据集（质量差；包含错误负例）可以在rlhn/default-250K找到。注意：RLHN数据集不是新的训练数据集，而是带有清理后的难负例的现有BGE集合训练数据集。

RLHN is a cascading LLM framework designed to accurately relabel hard negatives in existing IR/RAG training datasets, such as MS MARCO and HotpotQA. This Tevatron dataset (250K training pairs) contains the original queries, positives, and hard negatives after dropping each training pair with a single false negative. This repository contains the training pairs that can be used to fine-tune embedding, ColBERT, or multi-vector, and reranker models. The original dataset (bad quality; containing false negatives) can be found at rlhn/default-250K. Note: RLHN datasets are not new training datasets, but rather existing BGE collection training datasets with hard negatives cleaned!