dpohl/smollm-cosmosample-tokens-1.0B

该数据集包含了文本相关的特征，如提示(prompt)、文本内容(text)、token长度(token_length)、受众(audience)、格式(format)和种子数据(seed_data)。数据集分为训练集(train)，共有约1390895个示例，总大小为7.55GB。数据集来源于smollm-corpus和cosmopedia-v2，采用流式采样方式，并在达到约10亿个token时提前停止，总共包含大约10亿个token。The dataset includes text-related features such as prompt, text content, token length, audience, format, and seed data. The dataset is split into a training set (train) with approximately 1,390,895 examples and a total size of 7.55GB. The dataset is sourced from smollm-corpus and cosmopedia-v2, using a streaming sampling method with buffered shuffle and stops early at around 1 billion tokens, containing a total of approximately 1 billion tokens.