MichaelR207/rephraser_kimi_v1_0403_output

--- dataset_info: features: - name: output dtype: string - name: spec dtype: string - name: spec_id dtype: string - name: model dtype: string - name: warc_file dtype: string - name: doc_id dtype: string splits: - name: train num_examples: 28566 - name: validation num_examples: 100 license: cc-by-4.0 --- # rephraser_kimi_v1_0403_output Clean extracted text from web pages, produced by **Kimi-K2.5**. This dataset contains only the model-generated extraction output (no prompts, no HTML, no reasoning traces). Suitable for text quality analysis, downstream NLP tasks, and training data. ## Source Derived from [MichaelR207/rephraser_kimi_v1_0403](https://huggingface.co/datasets/MichaelR207/rephraser_kimi_v1_0403) by extracting the assistant response, stripping `<think>` reasoning and DSPy field markers, and dropping rows containing `[NO_USEFUL_CONTENT]`. ## Processing - Reasoning (`<think>...</think>`) stripped - DSPy markers (`[[ ## text ## ]]`, `[[ ## completed ## ]]`) stripped - Rows with `[NO_USEFUL_CONTENT]` dropped (49,125 rows removed) ## Schema | Column | Description | |--------|-------------| | `output` | Clean extracted text | | `spec` | Extraction specification used | | `spec_id` | Specification identifier (0-999) | | `model` | Model that generated the extraction | | `warc_file` | Source Common Crawl WARC file | | `doc_id` | Document identifier within WARC | ## License This dataset is licensed under [CC-BY-4.0](https://creativecommons.org/licenses/by/4.0/). The output text is model-generated extraction from public web content. ## Stats | Split | Rows | |-------|-----:| | Train | 28,566 | | Validation | 100 | | Total | 28,666 | | Dropped (no useful content) | 49,125 |