DeepSeek-R1-Distill-Qwen-1.5B-pts

# PTS Pivotal Tokens Dataset A dataset of pivotal tokens discovered using the Pivotal Token Search (PTS) technique. ## Details - **Source:** Generated using the [PTS](https://github.com/codelion/pts) tool - **Model:** deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B ## Format Each token in the dataset includes: - `query`: The original query that was processed - `pivot_context`: The context leading up to the pivotal token - `pivot_token`: The actual token that impacts success probability - `prob_delta`: The change in success probability caused by the token - Other metadata about the token ## Usage These pivotal tokens can be used for creating DPO datasets or extracting steering vectors: ```bash # Export to DPO format pts export --input-path="pivotal_tokens.jsonl" --format="dpo" --output-path="dpo_dataset.jsonl" --model="deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B" --find-rejected-tokens # Extract steering vectors pts export --input-path="pivotal_tokens.jsonl" --format="steering" --output-path="steering_vectors.jsonl" --model="deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B" ```