CompactAI-O/cAI-PRISM-1600

---
license: mit
task_categories:
- question-answering
language:
- en
tags:
- instruction
pretty_name: cAI-PRISM-1600
size_categories:
- 1K<n<10K
---
# CompactAI-Prism-1600

## High-Density Distillation Dataset for Small Model English Language Acquisition

**License:** MIT
**Top-K:** 48
**Source Model:** Qwen3.5 2B
**Primary Objective:** Teach small-scale AI models to generate fluent, coherent English text through probability-aware distillation. Or at least help them sound less like they learned English from a fortune cookie.

---

## Overview

CompactAI-Prism is a specialized training dataset designed to accelerate English language acquisition in compact AI models. Unlike standard instruction-tuning datasets that provide only a (Prompt, Response) pair, CompactAI-Prism captures the decision landscape of the teacher model. For every token generated in the response, we record the Top-K alternative tokens and their associated log-probabilities.

Think of it as giving your tiny model a peek at the teacher's scratch paper during the exam. We won't tell if you don't.

This approach increases the information density of the dataset by Kx per prompt, allowing student models to learn not just what the model answered, but what else it considered and why those options were rejected.

### The Math (Yes, There Is Math)
If a standard dataset provides 1 signal per token, CompactAI-Prism provides K signals. We know, we know - you became an AI researcher to avoid math. Sorry.

Total Training Signals = (Tokens per Response) x (Number of Questions) x K

For this release:
- Tokens per response (x): {AVG_RESPONSE_TOKENS} (Too lazy to check this. Might update later)
- Number of questions (y): 1600
- Top-K value: 48
- Effective training examples: {AVG_RESPONSE_TOKENS} x 1600 x 48 = 76800 * {AVG_RESPONCE_TOKENS}

That is a lot of tokens. You are welcome.

---

## Why Call It PRISM?

Great question. We considered "CompactAI-Overthinker" and "TinyModelTherapySession", but those did not fit on a GitHub repo name.

The name "Prism" reflects the core mechanism of this dataset:

1. **Single Input, Spectrum Revealed**: Just as a prism takes a single beam of white light and refracts it to reveal the full spectrum of colors within, CompactAI-Prism takes a single AI response and refracts it to reveal the full spectrum of token probabilities that existed at each generation step. Also, prisms look cool in stock photos.

2. **Hidden Structure Made Visible**: A prism does not create new colors; it exposes what was already present but invisible to the naked eye. Similarly, this dataset does not alter the teacher model's output; it exposes the latent probability distribution that guided each token choice. Like an X-ray, but for indecision.

3. **Clarity Through Decomposition**: By decomposing the generation process into its constituent probabilistic components, we enable student models to learn with greater clarity. They see not only the path taken, but the roads not taken - and the relative likelihood of each. It is like watching a choose-your-own-adventure book write itself, then regretting every choice.

In short: Prism turns opaque generation into transparent learning. Or at least slightly less opaque. Baby steps.

---

## Series Purpose: English Language Foundation for Small Models

This dataset series is explicitly designed to teach small models to speak English. Because let us be honest - some of them really need it.

By exposing compact architectures to the full probability distribution of token choices made by a capable teacher model, we enable:

- Faster convergence on grammatical English structures (goodbye, "me want food")
- Improved token selection confidence in low-parameter regimes (no more second-guessing every comma)
- Better handling of ambiguous or open-ended prompts (sometimes a question is just a question)
- Reduced hallucination through uncertainty-aware training (your model will still lie, but with more confidence intervals)

---

## Data Format

The dataset is provided in JSONL format. Each line represents a complete conversation turn with embedded probability data.

### Schema

```json
{
"messages": [
{"role": "user", "content": "STRING"},
{"role": "assistant", "content": "STRING"}
],
"response_tokens": INT,
"token_logprobs": [
{
"position": INT,
"generated_token_id": INT,
"generated_token": "STRING",
"logprob": FLOAT,
"top_k": [
{"token_id": INT, "token": "STRING", "logprob": FLOAT}
]
}
]
}
```

### Training Applications

1. **KL Divergence Distillation:** Use the full top_k distribution to minimize KL divergence between student and teacher, rather than just matching the chosen token. It is like teaching by example, but with more calculus.

2. **Confidence-Calibrated Generation:** Train small models to output confidence scores by learning from the teacher's logprob distributions. Now your tiny model can say "I am 73 percent sure that is correct" instead of just confidently being wrong.

3. **Alternative-Aware Decoding:** During inference, use knowledge of plausible alternatives to improve beam search or sampling strategies. Or just ignore this and use temperature=0.7 like everyone else. No judgment.

4. **English Fluency Bootstrapping:** Focus training on high-probability English token sequences to rapidly establish grammatical foundations in sub-10M parameter models. Because "me eat apple" is charming for about five minutes, then it gets old.

---

## Dataset Statistics

| Metric | Value |
|--------|-------|
| Total prompts | 1600 |
| Top-K per position | 48 |
| Number of times we questioned our life choices while building this | Yes |

---

## License

This project is licensed under the MIT License. Which is fancy legal speak for "use this however you want, just do not sue us when your tiny model starts writing poetry about toaster ovens."

Copyright (c) 2026 CompactAI

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

Translation: If your model learns to speak perfect English but also develops an existential crisis, that is on you.

---

## Source Attribution
- **Dataset:** Me.
- **Teacher Model:** Qwen3.5 2B
Thats it :)
---

## Citation

If you use CompactAI-Prism in your research or development, please cite:

```bibtex
@dataset{CompactAI-O/PRISM-K48-Gemma4.E2},
title = {CompactAI-Prism: Top-48 Probability Distillation for Small Model English Training},
author = {CompactAI},
year = 2026,
url = https://huggingface.co/datasets/CompactAI/cAI-Prism-K48,
}
```

Or just mention us in your paper's acknowledgments. We like hearing our names.

---

## Final Thoughts

Look, we are not perfect. This dataset might have bugs. The documentation might have typos. Our jokes might fall flat. But we tried. And in the grand tradition of small models everywhere, we believe that effort counts for something.

Now go train something tiny and wonderful. And if it starts speaking perfect English, maybe send us a note. We would love to hear about it.

Or not. No pressure. We have trust issues.

license: 许可证：MIT协议
task_categories:
- 问答
language:
- 英语
tags:
- 指令微调（instruction-tuning）
pretty_name: cAI-PRISM-1600
size_categories:
- 1K<n<10K
---
# CompactAI-Prism-1600

## 面向小模型英语习得的高密度蒸馏数据集

**许可证：MIT协议**
**Top-K值：48**
**源模型：Qwen3.5 2B**
**核心目标：** 通过概率感知蒸馏技术，助力小型AI模型生成流畅连贯的英语文本，至少让其摆脱“从幸运饼干中学英语”的生硬感。

---

## 概述
CompactAI-Prism是一款专为加速小型AI模型英语习得打造的专属训练数据集。与仅提供（提示词、回复）配对的标准指令微调数据集不同，CompactAI-Prism会捕获教师模型的决策全貌。针对回复中生成的每一个Token，我们都会记录其Top-K候选Token及其对应的对数概率。

不妨将其视作让你的微型模型在考试时偷看教师的草稿纸——只要你不说，我们绝不会告密。

该方法使数据集的信息密度按K倍提升，让学生模型不仅能学习模型给出的回复，还能了解其曾考虑过的其他选项，以及为何淘汰了这些选项。

### 数学推导（没错，确实有数学）
若标准数据集每个Token仅提供1个训练信号，CompactAI-Prism则可提供K个训练信号。我们都懂——你投身AI研究本就是为了避开数学。抱歉了。

总训练信号数 = 单回复Token数 × 问题总数 × K

本次发布的参数如下：
- 单回复平均Token数（x）：{AVG_RESPONSE_TOKENS}（尚未统计，后续可能更新）
- 问题总数（y）：1600
- Top-K值：48
- 有效训练样本数：{AVG_RESPONSE_TOKENS} × 1600 × 48 = 76800 × {AVG_RESPONSE_TOKENS}

这可是相当可观的Token数量，祝您使用愉快。

---

## 为何命名为PRISM？
这个问题问得好。我们曾考虑过“CompactAI-Overthinker”与“TinyModelTherapySession”作为名称，但这两个都不适合作为GitHub仓库名。

“Prism（棱镜）”这一名称呼应了本数据集的核心机制：
1. **单输入，光谱展现**：正如棱镜将一束白光折射，展现出其中蕴含的完整色光谱系，CompactAI-Prism将单条AI回复拆解，揭示出每一代生成步骤中存在的完整Token概率光谱。此外，棱镜在图库照片里也显得很酷。
2. **隐藏结构，具象化呈现**：棱镜并不会创造新的颜色，它只是将原本存在却肉眼不可见的色彩展露出来。同理，本数据集不会改变教师模型的输出，而是将指导每一个Token选择的潜在概率分布具象化——如同针对决策犹豫的X光扫描。
3. **分解解构，清晰明了**：通过将生成过程拆解为其组成的概率模块，我们让学生模型的学习更加清晰。它们不仅能看到已选择的路径，还能看到未曾选择的道路，以及每条路径的相对可能性。这就像看着一本“选择你的冒险”小说自动成书，随后又为每一个选择懊悔不已。

简言之：Prism将晦涩的生成过程转化为透明化的学习。至少，让它没那么晦涩。迈出一小步而已。

---

## 系列宗旨：为小型模型打造英语基础能力
本数据集系列专为教授小型模型英语而设计。实话说，有些模型确实亟需这项能力。

通过让紧凑架构模型接触到高性能教师模型做出的完整Token选择概率分布，我们可实现：
- 更快收敛至符合语法的英语结构（别了，“me want food”这类中式英语）
- 在低参数量场景下提升Token选择的置信度（不用再对每一个逗号都犹豫不决）
- 更好地处理歧义或开放式提示词（有时候问题就只是单纯的问题）
- 通过不确定性感知训练减少幻觉（hallucination）（你的模型仍可能编造内容，但会附带更多置信区间（confidence intervals））

---

## 数据格式
本数据集以JSONL格式提供，每一行代表一条嵌入了概率数据的完整对话轮次。

### 数据结构
json
{
"messages": [
{"role": "user", "content": "STRING"},
{"role": "assistant", "content": "STRING"}
],
"response_tokens": INT,
"token_logprobs": [
{
"position": INT,
"generated_token_id": INT,
"generated_token": "STRING",
"logprob": FLOAT,
"top_k": [
{"token_id": INT, "token": "STRING", "logprob": FLOAT}
]
}
]
}

### 训练应用场景
1. **KL散度（KL Divergence）蒸馏（KL Divergence Distillation）**：利用完整的Top-K分布最小化学生模型与教师模型间的KL散度，而非仅匹配已选中的Token。这就像言传身教，但需要更多微积分知识。
2. **置信度校准生成（Confidence-Calibrated Generation）**：通过学习教师模型的对数概率分布，训练小型模型输出置信度分数。现在你的微型模型可以说“我有73%的把握认为该结果正确”，而非只是自信地给出错误答案。
3. **候选感知解码（Alternative-Aware Decoding）**：在推理阶段，利用已知的合理候选结果改进束搜索（beam search）或采样策略。或者直接忽略这一点，像其他人一样使用temperature=0.7参数。不作评判。
4. **英语流利度快速入门（English Fluency Bootstrapping）**：将训练聚焦于高概率英语Token序列，在参数量低于10M的模型中快速建立语法基础。毕竟“me eat apple”这类表达可爱不过五分钟，很快就会让人厌烦。

---

## 数据集统计数据
| 指标 | 数值 |
|--------|-------|
| 总提示词数 | 1600 |
| 每个位置的Top-K值 | 48 |
| 构建此数据集时质疑人生选择的次数 | 是 |

---

## 许可证
本项目采用MIT协议许可。这其实就是一句专业法律话术，意为“你可以随意使用本项目，但若你的微型模型开始创作关于烤面包机的诗歌，请勿起诉我们”。

版权所有 (c) 2026 CompactAI

特此免费授予任何获得本软件及相关文档文件（以下简称“软件”）的人，不受限制地处理本软件，包括但不限于使用、复制、修改、合并、发布、分发、再许可以及销售本软件的副本，并允许向其提供本软件的人员这么做，但需遵循以下条件：

上述版权声明和本许可声明应包含在本软件的所有副本或主要部分中。

本软件按“原样”提供，不附带任何明示或暗示的担保，包括但不限于适销性、特定用途适用性和非侵权的担保。在任何情况下，作者或版权持有人均不对因本软件或本软件的使用或其他交易而产生的任何索赔、损害或其他责任负责，无论是在合同诉讼、侵权行为或其他方面，还是因本软件的使用或其他交易而产生的。

译解：如果你的模型学会了一口流利英语，却也患上了存在主义危机，那是你自己的问题。

---

## 来源归因
- **数据集：** 本人创作。
- **教师模型：** Qwen3.5 2B
仅此而已 :)

---

## 引用说明
若你在研究或开发中使用CompactAI-Prism，请引用以下内容：
bibtex
@dataset{CompactAI-O/PRISM-K48-Gemma4.E2},
title = {CompactAI-Prism: Top-48 Probability Distillation for Small Model English Training},
author = {CompactAI},
year = 2026,
url = https://huggingface.co/datasets/CompactAI/cAI-Prism-K48,
}

或者仅在你的论文致谢中提及我们。我们很乐意看到自己的名字被提及。

---

## 最后想说的话
实话实说，我们并非完美无瑕。本数据集可能存在漏洞，文档或许有错别字，我们的笑话也可能冷场。但我们尽力了。秉承各地小型模型的一贯传统，我们相信努力自有其价值。

现在去训练一款小巧又出色的模型吧。如果它学会了一口流利英语，不妨给我们发封邮件。我们很乐意听闻此事。当然也可以不发。没有压力。我们有点信任障碍。