dureader-context-relevance-with-think

# DuReader Context Relevance Dataset with Think Process This dataset is used for training the `zilliz/semantic-highlight-bilingual-v1`(https://huggingface.co/zilliz/semantic-highlight-bilingual-v1) model for semantic highlighting in RAG (Retrieval-Augmented Generation) systems. ## Dataset Description This dataset contains query-context pairs with relevance annotations for context spans. The annotations help identify which parts of a document are semantically relevant to a query, even when they don't contain exact keyword matches. ### Key Features - **`context_spans`**: Indicates the positions of segmented spans within the context text - **`context_spans_relevance`**: Binary labels indicating whether each span should be highlighted (i.e., provides key information for answering the query) - **`think_process`**: Contains the reasoning process used during annotation to ensure more accurate context span relevance labels and improve observability and interpretability The dataset was annotated using Qwen3-8B, with the complete thinking process preserved in the `think_process` field. ## Data Source This dataset is derived from `sentence-transformers/dureader`. ## License This dataset follows the license of the original dataset.