问题思维链相似度数据集

# QTH-Embedding Training Dataset Pre-compressed embedding training dataset for reasoning-aware retrieval models. Each record contains structured summaries (anchor/positive/negative) produced by a condenser model, along with the original raw texts. ## Dataset Statistics | Source | Rows | Description | |--------|------|-------------| | hard | 100,521 | Method-level hard negatives (abstract problem type + methodology) | | think | 100,000 | Thinking traces (query + long CoT reasoning) | | index | 100,000 | Reasoning solutions for RAG index construction | | **Total** | **300,521** | | ## Schema ```json { "anchor_text": "## Summary\nTopic: ...\nProblem: ...\nSkill: ...\nKnowledge: ...\n\n## More\n- ...", "positive_text": "## Summary\nTopic: ...\nProblem: ...\nSkill: ...\nKnowledge: ...\n\n## More\n- ...", "negative_texts": ["## Summary\n..."], "source": "hard | think | index", "query_raw": "original query text before compression", "cot_raw": "original CoT / positive passage before compression", "negs_raw": ["original negative passage(s)"] } ``` | Field | Type | Description | |-------|------|-------------| | `anchor_text` | string | Compressed query — structured summary for embedding | | `positive_text` | string | Compressed positive (CoT/passage) — structured summary | | `negative_texts` | list[string] | Compressed hard negative(s); empty list if none | | `source` | string | Data source category: `hard`, `think`, or `index` | | `query_raw` | string | Original uncompressed query | | `cot_raw` | string | Original uncompressed CoT / positive passage | | `negs_raw` | list[string] | Original uncompressed negative passage(s) | ## Condenser All texts are compressed using **Qwen3.5-4B-CM-v2** (`twinkle-kit/Qwen3.5-4B-CM-v2`), a fine-tuned condenser model based on Qwen3.5-4B. Inference is performed via vLLM (8×GPU, tensor parallel) with API fallback to `qwen3.7-max` for validation failures. **Compression parameters:** - Temperature: 0.2 - Top-p: 0.5 - Max output tokens: 8192 ### Compression Strategy The condenser applies different prompts depending on source type: - **think / index**: Extract a retrieval-friendly structured summary preserving topic, problem, skill, and knowledge. - **hard**: Extract the **abstract problem type** (ignoring specific numbers/values) and the **abstract methodology** (general technique and reasoning steps). This enables method-level clustering in embedding space. ### Output Format All compressed texts follow the same skeleton: ``` ## Summary Topic: <subject — scope> Problem: <what needs solving> Skill: <method/technique required> Knowledge: <theoretical concepts invoked> ## More - <topic-key>: <expansion hint> ``` ## Source Datasets ### Think (100,000 rows) Sampled from long-form reasoning traces: - Modotte/CodeX-2M-Thinking - open-thoughts/OpenThoughts3-1.2M - GAIR/LIMO-v2 - AI-ModelScope/Chinese-DeepSeek-R1-Distill-data-110k - nohurry/Opus-4.6-Reasoning-3000x-filtered - Roman1111111/claude-opus-4.6-10000x - angrygiraffe/claude-opus-4.6-4.7-reasoning-8.7k ### Index (100,000 rows) Sampled from diverse reasoning solutions: - AI-ModelScope/OpenMathReasoning - open-r1/OpenR1-Math-220k - simplescaling/s1K-1.1 - nv-community/OpenCodeReasoning-2 - open-r1/codeforces-cots - bespokelabs/Bespoke-Stratos-17k - AI-ModelScope/dolphin-r1 - glaiveai/reasoning-v1-20m - facebook/natural_reasoning - HuggingFaceTB/cosmopedia - AI-ModelScope/Omni-MATH ### Hard (100,521 rows) Method-level hard negatives constructed from: - AI-ModelScope/reasonir-data (hq subset) — provides query + positive + hard negative triplets - xlangai/BRIGHT (documents corpus) — supplies the positive/negative passage texts Each hard sample contains a query, a positive passage, and one hard negative passage. The three texts are independently compressed into abstract method descriptions, enabling the embedding model to learn fine-grained methodology discrimination. ## Intended Use This dataset is designed for training embedding models with InfoNCE contrastive loss, where: - `anchor_text` serves as the query embedding - `positive_text` serves as the positive match - `negative_texts` provide in-batch or explicit hard negatives The `*_raw` fields are preserved for analysis, condenser retraining, and quality auditing. ## Production Pipeline Built with `make_embedding_dataset.py` from the [Twinkle](https://github.com/tastelikefeet/twinkle) framework: 1. Load and cap source datasets (think ≤100k, index ≤100k, hard = all) 2. Compress via vLLM condenser (8 GPU, batch size 128) 3. Validate structure (`## Summary` + `## More` sections, 4-line template) 4. API fallback for validation failures 5. Incremental flush to JSONL with atomic progress tracking