wrmedford/Qwen3.5-9B-SSD

# SSD Dataset Replication (Qwen3.5-9B)

This dataset is a replication of the **"Embarrassingly Simple Self-Distillation Improves Code Generation" (SSD)** paper ([arXiv:2604.01193](https://arxiv.org/pdf/2604.01193)).

## Overview
The dataset contains coding problems and their corresponding solutions generated by **Qwen3.5-9B** using high-temperature sampling (T=1.1) to explore the model's latent capabilities. This approach, known as SSD, focuses on "self-distillation" where a model's own correct but non-greedy outputs are used for fine-tuning to improve its standard (greedy) performance.

## Dataset Construction
- **Seed Prompts:** Samples from `deepmind/code_contests`
- **Generation:** High-temperature sampling (T=1.1, Top-K=20, Top-P=0.95) with **Qwen3.5-9B**.
- **Reasoning:** Reasoning chains (thinking) were enabled during generation to capture the model's logical process.

## Contents
- `ssd_dataset.jsonl`: The generated samples in OpenAI-style message format.
- `generate_dataset.py`: The replication script used to synthesize the data (vLLM-based, data-parallel).

## Usage
To replicate the generation process:
```bash
python generate_dataset.py --model Qwen/Qwen3.5-9B --dp <NUM_GPUS>
```

## Citation
```bibtex
@misc{zhang2026embarrassinglysimpleselfdistillationimproves,
title={Embarrassingly Simple Self-Distillation Improves Code Generation},
author={Ruixiang Zhang and Richard He Bai and Huangjie Zheng and Navdeep Jaitly and Ronan Collobert and Yizhe Zhang},
year={2026},
eprint={2604.01193},
archivePrefix={arXiv},
primaryClass={cs.CL},
url={https://arxiv.org/abs/2604.01193},
}
```