J-RUM/exemplar-partitioning

该数据集提供了针对Gemma-2-2B和Gemma-2-2B-it模型的预训练示例分区（EP）字典，覆盖多个网络层和分辨率百分位数。每个字典都是残差流激活的居中单位球面的Voronoi分区，基于从构建流中提取的观察到的激活方向（称为“示例”）进行锚定。EP字典通过流式处理Pile激活数据，使用领导者聚类方法和单个校准余弦距离阈值θ_p构建，构建过程在批次内无新区域产生时终止（达到饱和）。数据集包含不同层和百分位数设置的多个文件，每个文件包含一个序列化的Dictionary对象和元数据，用于机器可解释性研究，如特征字典和稀疏自编码器应用。Pretrained Exemplar Partitioning (EP) dictionaries for Gemma-2-2B and Gemma-2-2B-it across multiple layers and resolution percentiles. Each dictionary is a Voronoi partition of the centered unit sphere of residual-stream activations, anchored on observed activation directions (exemplars) drawn from the construction stream. EP dictionaries are built by streaming Pile activations through leader clustering with a single calibrated cosine-distance threshold θ_p, and construction terminates when no new regions are produced for one batch (saturation). The dataset includes files for various layers and percentile settings, each containing a serialized Dictionary object and metadata, intended for interpretability research such as feature dictionaries and sparse autoencoder applications.