open-concept-steering/OLMo-2_Residual_Streams

该数据集包含了大约6亿个从FineWeb数据集的子集运行OLMo-2-1124-7B-Instruct模型得到的残差流。这些残差流使用uint16格式保存，但在使用前需要转换为bfloat16格式。数据集来源于FineWeb数据集的train分区的sample-10BT子集，使用的是AllenAI的OLMo-2-1124-7B-Instruct模型。数据以.parquet文件格式存储，未经归一化处理。This dataset contains approximately 600 million residual streams derived from running the OLMo-2-1124-7B-Instruct model on a subset of the FineWeb dataset. These residual streams are saved in uint16 format but need to be cast to bfloat16 format before use. The dataset is sourced from the sample-10BT subset of the train split of the FineWeb dataset, using the AllenAI OLMo-2-1124-7B-Instruct model. The data is stored in .parquet file format and has not been normalized.