locailabs/nemotron_terminal_filtered

--- license: cc-by-4.0 task_categories: - question-answering language: - en tags: - code - terminal - uncertainty-sampling size_categories: - 10K<n<100K --- # Nemotron Terminal Filtered An uncertainty-curated subset of NVIDIA's [Nemotron-Terminal-Corpus](https://huggingface.co/datasets/nvidia/Nemotron-Terminal-Corpus) (`dataset_adapters` split), selected for high-formation density for post-training [NVIDIA-Nemotron-3-Super-120B-A12B-BF16](https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16). ## Motivation The full `dataset_adapters` split contains ~226k terminal execution trajectories. To curate a compact, high-value subset for post-training we score each sample by how hard the model finds it, using entropy as a proxy for uncertainty. The resulting **30,000 samples** represent the tasks where the model is most uncertain, and therefore stands to learn the most. All original columns from the NVIDIA dataset are preserved, with `conversations` renamed to `messages` for OpenAI chat format compatibility. ## Method 1. For each sample, we extract the system message and first user message as a prompt. 2. The model generates 32 tokens at temperature 0 (greedy decoding, reasoning enabled) and we collect the top-20 logprobs per token. The 32-token window captures the model's initial reasoning about the task. 3. **Entropy** is computed per sample: the mean Shannon entropy of the renormalised top-k distribution across the 32-token window. High entropy means the model spreads probability across many alternatives — it is genuinely uncertain about what to produce. 4. Samples are ranked by entropy and the top 30,000 are selected. ## Columns | Column | Description | |---|---| | `messages` | Multi-turn chat messages (renamed from `conversations`) | | `agent` | Agent identifier | | `model` | Model used for trajectory generation | | `model_provider` | Provider of the model | | `date` | Trajectory generation date | | `task` | Task description | | `episode` | Episode identifier | | `run_id` | Run identifier | | `trial_name` | Trial name | | `enable_thinking` | Whether thinking/reasoning was enabled during trajectory generation | | `source` | Source dataset the trajectory was adapted from (null for some subsets) | ## Usage ```python from datasets import load_dataset ds = load_dataset("locailabs/nemotron_terminal_filtered", split="train") ``` ## Source This dataset is derived from: > **Terminal-Corpus: Large-Scale SFT Dataset for Terminal Agents** > NVIDIA — [nvidia/Nemotron-Terminal-Corpus](https://huggingface.co/datasets/nvidia/Nemotron-Terminal-Corpus) Uncertainty scoring was performed against: > [nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16](https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16) ```bibtex @misc{pi2026dataengineeringscalingllm, title={On Data Engineering for Scaling LLM Terminal Capabilities}, author={Renjie Pi and Grace Lam and Mohammad Shoeybi and Pooya Jannaty and Bryan Catanzaro and Wei Ping}, year={2026}, eprint={2602.21193}, archivePrefix={arXiv}, primaryClass={cs.CL}, url={https://arxiv.org/abs/2602.21193}, } ```license: CC BY 4.0 任务类别： - 问答 语言： - 英语 标签： - 代码 - 终端 - 不确定性采样 样本量范围： - 10K < n < 100K # 经筛选的Nemotron终端数据集 本数据集是NVIDIA的[Nemotron-Terminal-Corpus (Nemotron终端语料库)](https://huggingface.co/datasets/nvidia/Nemotron-Terminal-Corpus)的`dataset_adapters`拆分子集，经过不确定性筛选，专为针对[NVIDIA-Nemotron-3-Super-120B-A12B-BF16](https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16)的后训练任务筛选出高信息密度样本。 ## 研究动机 完整的`dataset_adapters`拆分包含约22.6万条终端执行轨迹。为了筛选出紧凑且高价值的后训练子集，我们以熵(entropy)作为不确定性的代理指标，对每个样本按照模型处理难度进行评分。最终选取的**30000个样本**对应模型最不确定、因此具备最大学习潜力的任务。 所有原始NVIDIA数据集的列均被保留，仅将`conversations`重命名为`messages`以兼容OpenAI对话格式。 ## 筛选方法 1. 针对每个样本，提取系统提示语与第一条用户消息作为提示(prompt)。 2. 模型在温度为0的设置下（贪心解码(greedy decoding)，启用推理功能）生成32个Token，收集每个Token的前20个对数概率(logprobs)。这32个Token的窗口能够捕捉模型对任务的初始推理过程。 3. 为每个样本计算**熵(entropy)**：即32个Token窗口内经重归一化的前k分布的平均香农熵(Shannon entropy)。熵值越高，代表模型在多个备选方案间分配概率，即对输出内容存在真实的不确定性。 4. 按熵值对样本进行排序，选取排名前30000的样本。 ## 数据集列说明 | 列名 | 描述 | |---|---| | `messages` | 多轮对话消息（原列名为`conversations`） | | `agent` | 智能体标识符 | | `model` | 用于生成轨迹的模型 | | `model_provider` | 模型提供商 | | `date` | 轨迹生成日期 | | `task` | 任务描述 | | `episode` | 回合标识符 | | `run_id` | 运行标识符 | | `trial_name` | 试验名称 | | `enable_thinking` | 轨迹生成过程中是否启用思考/推理功能 | | `source` | 轨迹所适配的源数据集（部分子集无此项） | ## 使用示例 python from datasets import load_dataset ds = load_dataset("locailabs/nemotron_terminal_filtered", split="train") ## 数据集来源 本数据集衍生自以下工作： > **《终端语料库：面向终端智能体的大规模监督微调(Supervised Fine-Tuning, SFT)数据集》** > NVIDIA — [nvidia/Nemotron-Terminal-Corpus](https://huggingface.co/datasets/nvidia/Nemotron-Terminal-Corpus) 不确定性评分基于以下模型完成： > [nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16](https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16) bibtex @misc{pi2026dataengineeringscalingllm, title={On Data Engineering for Scaling LLM Terminal Capabilities}, author={Renjie Pi and Grace Lam and Mohammad Shoeybi and Pooya Jannaty and Bryan Catanzaro and Wei Ping}, year={2026}, eprint={2602.21193}, archivePrefix={arXiv}, primaryClass={cs.CL}, url={https://arxiv.org/abs/2602.21193}, }