KantaHayashiAI/test2

--- pretty_name: test strict shuffled task_categories: - text-classification tags: - parquet - shuffled - exact-shuffle --- # Strict shuffled copy of `KantaHayashiAI/test` This dataset was produced by assigning each source row a deterministic pseudo-random sort key derived from: - source parquet path - row index inside that parquet - shuffle seed `2026-04-01-strict-shuffle-v1` Rows were first partitioned into `512` buckets by the high bits of the key, then each bucket was fully sorted by `(__key_hi, __key_lo, __file_id, __row_idx)`. This yields a deterministic global shuffled order without requiring the full dataset to be materialized twice on local disk. Expected train shard count: `512`.pretty_name: 严格打乱测试集 task_categories: - 文本分类 tags: - Parquet（帕克尔列式存储格式） - 已打乱 - 精确打乱 # 数据集 `KantaHayashiAI/test` 的严格打乱副本 本数据集通过为每条源数据行分配由以下要素生成的确定性伪随机排序键而构建： - 源Parquet文件路径 - 该Parquet文件内的行索引 - 打乱种子 `2026-04-01-strict-shuffle-v1` 数据行首先依据排序键的高位被划分为`512`个分桶，随后每个分桶均按照`(__key_hi, __key_lo, __file_id, __row_idx)`完成全量排序。该方案可生成确定性的全局打乱顺序，且无需将完整数据集在本地磁盘上实例化两次。 预期训练分片数：`512`。