mateowilliam/kimi-k2.6-reap-observations-v1
收藏官方服务:
资源简介:
该数据集包含在Kimi-K2.6模型上完整REAP校准过程的观察者输出。它不是一个剪枝后的模型。每条记录描述了基础模型每个MoE层的每个令牌路由决策、专家激活范数以及REAP显著性成分。下游用户可以将这些观察结果反馈到`reap.prune`(或任何其他基于专家显著性的压缩器)中,以生成任意压缩比的剪枝检查点,而无需重新运行(昂贵的)前向传递校准。
This dataset contains the **observer output** of a full REAP calibration pass on Kimi-K2.6. It is *not* a pruned model. Each record describes per-token routing decisions, expert activation norms, and the REAP saliency ingredients for every MoE layer of the base model. Downstream consumers can feed these observations back into `reap.prune` (or any other expert-saliency-based compressor) to produce pruned checkpoints at arbitrary compression ratios without re-running the (expensive) forward-pass calibration.
提供机构:
mateowilliam


