Name: self-long/RULER-llama3-1M
Creator: self-long
Published: 2025-03-17 06:20:50
License: 暂无描述

下载链接：

https://hf-mirror.com/datasets/self-long/RULER-llama3-1M

下载链接

链接失效反馈

官方服务：

资源简介：

--- dataset_info: - config_name: cwe_128k features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 34715963 num_examples: 100 download_size: 24390261 dataset_size: 34715963 - config_name: cwe_16k features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 22452603 num_examples: 500 download_size: 13858692 dataset_size: 22452603 - config_name: cwe_1M features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 36610903 num_examples: 100 download_size: 25734435 dataset_size: 36610903 - config_name: cwe_256k features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 36610514 num_examples: 100 download_size: 25734267 dataset_size: 36610514 - config_name: cwe_32k features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 43587393 num_examples: 500 download_size: 29410509 dataset_size: 43587393 - config_name: cwe_4k features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 6236910 num_examples: 500 download_size: 2847578 dataset_size: 6236910 - config_name: cwe_512k features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 36611213 num_examples: 100 download_size: 25734491 dataset_size: 36611213 - config_name: cwe_64k features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 87117921 num_examples: 500 download_size: 60838325 dataset_size: 87117921 - config_name: cwe_8k features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 11604588 num_examples: 500 download_size: 6254506 dataset_size: 11604588 - config_name: fwe_128k features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 31427116 num_examples: 100 download_size: 8175372 dataset_size: 31427116 - config_name: fwe_16k features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 20328568 num_examples: 500 download_size: 5333658 dataset_size: 20328568 - config_name: fwe_1M features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 272481773 num_examples: 100 download_size: 71161110 dataset_size: 272481773 - config_name: fwe_256k features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 67529274 num_examples: 100 download_size: 17621748 dataset_size: 67529274 - config_name: fwe_32k features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 42239151 num_examples: 500 download_size: 11037910 dataset_size: 42239151 - config_name: fwe_4k features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 5648420 num_examples: 500 download_size: 1476616 dataset_size: 5648420 - config_name: fwe_512k features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 132946403 num_examples: 100 download_size: 34712894 dataset_size: 132946403 - config_name: fwe_64k features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 84049910 num_examples: 500 download_size: 21861044 dataset_size: 84049910 - config_name: fwe_8k features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 10651775 num_examples: 500 download_size: 2810914 dataset_size: 10651775 - config_name: niah_multikey_1_128k features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 59824107 num_examples: 100 download_size: 34955994 dataset_size: 59824107 - config_name: niah_multikey_1_16k features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 38997638 num_examples: 500 download_size: 22736891 dataset_size: 38997638 - config_name: niah_multikey_1_1M features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 300428089 num_examples: 100 download_size: 174484749 dataset_size: 300428089 - config_name: niah_multikey_1_256k features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 122709682 num_examples: 100 download_size: 71083810 dataset_size: 122709682 - config_name: niah_multikey_1_32k features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 77719421 num_examples: 500 download_size: 45886323 dataset_size: 77719421 - config_name: niah_multikey_1_4k features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 9344094 num_examples: 500 download_size: 2058827 dataset_size: 9344094 - config_name: niah_multikey_1_512k features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 244982823 num_examples: 100 download_size: 142249373 dataset_size: 244982823 - config_name: niah_multikey_1_64k features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 154622206 num_examples: 500 download_size: 90211659 dataset_size: 154622206 - config_name: niah_multikey_1_8k features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 18115924 num_examples: 500 download_size: 6428656 dataset_size: 18115924 - config_name: niah_multikey_2_128k features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 47391886 num_examples: 100 download_size: 15796360 dataset_size: 47391886 - config_name: niah_multikey_2_16k features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 30110317 num_examples: 500 download_size: 10106092 dataset_size: 30110317 - config_name: niah_multikey_2_1M features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 389636388 num_examples: 100 download_size: 129699742 dataset_size: 389636388 - config_name: niah_multikey_2_256k features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 97336323 num_examples: 100 download_size: 32416678 dataset_size: 97336323 - config_name: niah_multikey_2_32k features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 60518140 num_examples: 500 download_size: 20233362 dataset_size: 60518140 - config_name: niah_multikey_2_4k features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 6944494 num_examples: 500 download_size: 2300452 dataset_size: 6944494 - config_name: niah_multikey_2_512k features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 194684057 num_examples: 100 download_size: 64819711 dataset_size: 194684057 - config_name: niah_multikey_2_64k features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 121133718 num_examples: 500 download_size: 40411210 dataset_size: 121133718 - config_name: niah_multikey_2_8k features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 14390682 num_examples: 500 download_size: 4811008 dataset_size: 14390682 - config_name: niah_multikey_3_128k features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 25165413 num_examples: 100 download_size: 16324473 dataset_size: 25165413 - config_name: niah_multikey_3_16k features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 16101300 num_examples: 500 download_size: 10380093 dataset_size: 16101300 - config_name: niah_multikey_3_1M features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 206448878 num_examples: 100 download_size: 133854963 dataset_size: 206448878 - config_name: niah_multikey_3_256k features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 51388406 num_examples: 100 download_size: 33326772 dataset_size: 51388406 - config_name: niah_multikey_3_32k features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 31778232 num_examples: 500 download_size: 20612281 dataset_size: 31778232 - config_name: niah_multikey_3_4k features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 3275106 num_examples: 500 download_size: 2006688 dataset_size: 3275106 - config_name: niah_multikey_3_512k features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 103204174 num_examples: 100 download_size: 66925675 dataset_size: 103204174 - config_name: niah_multikey_3_64k features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 63130382 num_examples: 500 download_size: 40947031 dataset_size: 63130382 - config_name: niah_multikey_3_8k features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 7549912 num_examples: 500 download_size: 4797994 dataset_size: 7549912 - config_name: niah_multiquery_128k features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 59879290 num_examples: 100 download_size: 34984014 dataset_size: 59879290 - config_name: niah_multiquery_16k features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 39263533 num_examples: 500 download_size: 22877361 dataset_size: 39263533 - config_name: niah_multiquery_1M features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 300491487 num_examples: 100 download_size: 174502937 dataset_size: 300491487 - config_name: niah_multiquery_256k features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 122768797 num_examples: 100 download_size: 71136819 dataset_size: 122768797 - config_name: niah_multiquery_32k features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 76539753 num_examples: 500 download_size: 45155047 dataset_size: 76539753 - config_name: niah_multiquery_4k features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 9584220 num_examples: 500 download_size: 2185251 dataset_size: 9584220 - config_name: niah_multiquery_512k features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 245051799 num_examples: 100 download_size: 142294465 dataset_size: 245051799 - config_name: niah_multiquery_64k features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 153398595 num_examples: 500 download_size: 89601118 dataset_size: 153398595 - config_name: niah_multiquery_8k features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 18343680 num_examples: 500 download_size: 6564788 dataset_size: 18343680 - config_name: niah_multivalue_128k features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 59888819 num_examples: 100 download_size: 34979069 dataset_size: 59888819 - config_name: niah_multivalue_16k features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 39241843 num_examples: 500 download_size: 22809731 dataset_size: 39241843 - config_name: niah_multivalue_1M features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 300485898 num_examples: 100 download_size: 174509547 dataset_size: 300485898 - config_name: niah_multivalue_256k features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 122763229 num_examples: 100 download_size: 71106116 dataset_size: 122763229 - config_name: niah_multivalue_32k features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 78005605 num_examples: 500 download_size: 45980103 dataset_size: 78005605 - config_name: niah_multivalue_4k features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 9527643 num_examples: 500 download_size: 2119838 dataset_size: 9527643 - config_name: niah_multivalue_512k features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 245046992 num_examples: 100 download_size: 142280478 dataset_size: 245046992 - config_name: niah_multivalue_64k features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 154894913 num_examples: 500 download_size: 90327506 dataset_size: 154894913 - config_name: niah_multivalue_8k features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 18320958 num_examples: 500 download_size: 6493426 dataset_size: 18320958 - config_name: niah_single_1_128k features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 47765131 num_examples: 100 download_size: 2342872 dataset_size: 47765131 - config_name: niah_single_1_16k features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 29575630 num_examples: 500 download_size: 1602061 dataset_size: 29575630 - config_name: niah_single_1_1M features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 393194289 num_examples: 100 download_size: 18964884 dataset_size: 393194289 - config_name: niah_single_1_256k features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 98165142 num_examples: 100 download_size: 4767829 dataset_size: 98165142 - config_name: niah_single_1_32k features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 61075357 num_examples: 500 download_size: 3139493 dataset_size: 61075357 - config_name: niah_single_1_4k features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 7075574 num_examples: 500 download_size: 416867 dataset_size: 7075574 - config_name: niah_single_1_512k features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 196490311 num_examples: 100 download_size: 9498359 dataset_size: 196490311 - config_name: niah_single_1_64k features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 121825590 num_examples: 500 download_size: 6065942 dataset_size: 121825590 - config_name: niah_single_1_8k features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 14950694 num_examples: 500 download_size: 832340 dataset_size: 14950694 - config_name: niah_single_2_128k features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 59811112 num_examples: 100 download_size: 34942504 dataset_size: 59811112 - config_name: niah_single_2_16k features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 38898773 num_examples: 500 download_size: 22672485 dataset_size: 38898773 - config_name: niah_single_2_1M features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 300420099 num_examples: 100 download_size: 174479186 dataset_size: 300420099 - config_name: niah_single_2_256k features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 122701413 num_examples: 100 download_size: 71083099 dataset_size: 122701413 - config_name: niah_single_2_32k features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 77623389 num_examples: 500 download_size: 45822549 dataset_size: 77623389 - config_name: niah_single_2_4k features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 9251315 num_examples: 500 download_size: 1988139 dataset_size: 9251315 - config_name: niah_single_2_512k features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 244981098 num_examples: 100 download_size: 142254345 dataset_size: 244981098 - config_name: niah_single_2_64k features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 154523169 num_examples: 500 download_size: 90130522 dataset_size: 154523169 - config_name: niah_single_2_8k features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 18016765 num_examples: 500 download_size: 6329974 dataset_size: 18016765 - config_name: niah_single_3_128k features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 59824514 num_examples: 100 download_size: 34958376 dataset_size: 59824514 - config_name: niah_single_3_16k features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 38969669 num_examples: 500 download_size: 22752171 dataset_size: 38969669 - config_name: niah_single_3_1M features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 300422102 num_examples: 100 download_size: 174483566 dataset_size: 300422102 - config_name: niah_single_3_256k features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 122710335 num_examples: 100 download_size: 71093432 dataset_size: 122710335 - config_name: niah_single_3_32k features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 77691041 num_examples: 500 download_size: 45899994 dataset_size: 77691041 - config_name: niah_single_3_4k features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 9314979 num_examples: 500 download_size: 2058451 dataset_size: 9314979 - config_name: niah_single_3_512k features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 244982061 num_examples: 100 download_size: 142274363 dataset_size: 244982061 - config_name: niah_single_3_64k features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 154595247 num_examples: 500 download_size: 90211128 dataset_size: 154595247 - config_name: niah_single_3_8k features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 18085940 num_examples: 500 download_size: 6401683 dataset_size: 18085940 - config_name: qa_1_128k features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 59692943 num_examples: 100 download_size: 36858575 dataset_size: 59692943 - config_name: qa_1_16k features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 32398879 num_examples: 500 download_size: 19060014 dataset_size: 32398879 - config_name: qa_1_1M features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 500615723 num_examples: 100 download_size: 301798055 dataset_size: 500615723 - config_name: qa_1_256k features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 122950365 num_examples: 100 download_size: 74914321 dataset_size: 122950365 - config_name: qa_1_32k features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 76190123 num_examples: 500 download_size: 46513533 dataset_size: 76190123 - config_name: qa_1_4k features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 6202187 num_examples: 500 download_size: 1585206 dataset_size: 6202187 - config_name: qa_1_512k features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 245938273 num_examples: 100 download_size: 148769779 dataset_size: 245938273 - config_name: qa_1_64k features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 154797878 num_examples: 500 download_size: 95198173 dataset_size: 154797878 - config_name: qa_1_8k features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 15756125 num_examples: 500 download_size: 5803640 dataset_size: 15756125 - config_name: qa_2_128k features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 54533191 num_examples: 100 download_size: 34328757 dataset_size: 54533191 - config_name: qa_2_16k features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 33331258 num_examples: 500 download_size: 20870561 dataset_size: 33331258 - config_name: qa_2_1M features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 447563949 num_examples: 100 download_size: 281209412 dataset_size: 447563949 - config_name: qa_2_256k features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 111648627 num_examples: 100 download_size: 70193564 dataset_size: 111648627 - config_name: qa_2_32k features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 65719450 num_examples: 500 download_size: 41281832 dataset_size: 65719450 - config_name: qa_2_4k features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 7302728 num_examples: 500 download_size: 4343081 dataset_size: 7302728 - config_name: qa_2_512k features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 223700315 num_examples: 100 download_size: 140582154 dataset_size: 223700315 - config_name: qa_2_64k features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 138463519 num_examples: 500 download_size: 87081598 dataset_size: 138463519 - config_name: qa_2_8k features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 15985571 num_examples: 500 download_size: 9876685 dataset_size: 15985571 - config_name: vt_128k features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 48443146 num_examples: 100 download_size: 2402921 dataset_size: 48443146 - config_name: vt_16k features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 30631834 num_examples: 500 download_size: 1785737 dataset_size: 30631834 - config_name: vt_1M features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 396981759 num_examples: 100 download_size: 19167034 dataset_size: 396981759 - config_name: vt_256k features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 99312584 num_examples: 100 download_size: 4849483 dataset_size: 99312584 - config_name: vt_32k features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 62028234 num_examples: 500 download_size: 3317272 dataset_size: 62028234 - config_name: vt_4k features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 7423855 num_examples: 500 download_size: 516551 dataset_size: 7423855 - config_name: vt_512k features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 198773989 num_examples: 100 download_size: 9636166 dataset_size: 198773989 - config_name: vt_64k features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 123909872 num_examples: 500 download_size: 6296516 dataset_size: 123909872 - config_name: vt_8k features: - name: index dtype: int64 - name: input dtype: string - name: answers sequence: string - name: length dtype: int64 - name: predictions struct: - name: self-long/SelfLong-Llama3.1-8B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-1B-Instruct dtype: string - name: self-long/SelfLong-Llama3.2-3B-Instruct dtype: string splits: - name: validation num_bytes: 15161459 num_examples: 500 download_size: 942804 dataset_size: 15161459 configs: - config_name: cwe_128k data_files: - split: validation path: cwe_128k/validation-* - config_name: cwe_16k data_files: - split: validation path: cwe_16k/validation-* - config_name: cwe_1M data_files: - split: validation path: cwe_1M/validation-* - config_name: cwe_256k data_files: - split: validation path: cwe_256k/validation-* - config_name: cwe_32k data_files: - split: validation path: cwe_32k/validation-* - config_name: cwe_4k data_files: - split: validation path: cwe_4k/validation-* - config_name: cwe_512k data_files: - split: validation path: cwe_512k/validation-* - config_name: cwe_64k data_files: - split: validation path: cwe_64k/validation-* - config_name: cwe_8k data_files: - split: validation path: cwe_8k/validation-* - config_name: fwe_128k data_files: - split: validation path: fwe_128k/validation-* - config_name: fwe_16k data_files: - split: validation path: fwe_16k/validation-* - config_name: fwe_1M data_files: - split: validation path: fwe_1M/validation-* - config_name: fwe_256k data_files: - split: validation path: fwe_256k/validation-* - config_name: fwe_32k data_files: - split: validation path: fwe_32k/validation-* - config_name: fwe_4k data_files: - split: validation path: fwe_4k/validation-* - config_name: fwe_512k data_files: - split: validation path: fwe_512k/validation-* - config_name: fwe_64k data_files: - split: validation path: fwe_64k/validation-* - config_name: fwe_8k data_files: - split: validation path: fwe_8k/validation-* - config_name: niah_multikey_1_128k data_files: - split: validation path: niah_multikey_1_128k/validation-* - config_name: niah_multikey_1_16k data_files: - split: validation path: niah_multikey_1_16k/validation-* - config_name: niah_multikey_1_1M data_files: - split: validation path: niah_multikey_1_1M/validation-* - config_name: niah_multikey_1_256k data_files: - split: validation path: niah_multikey_1_256k/validation-* - config_name: niah_multikey_1_32k data_files: - split: validation path: niah_multikey_1_32k/validation-* - config_name: niah_multikey_1_4k data_files: - split: validation path: niah_multikey_1_4k/validation-* - config_name: niah_multikey_1_512k data_files: - split: validation path: niah_multikey_1_512k/validation-* - config_name: niah_multikey_1_64k data_files: - split: validation path: niah_multikey_1_64k/validation-* - config_name: niah_multikey_1_8k data_files: - split: validation path: niah_multikey_1_8k/validation-* - config_name: niah_multikey_2_128k data_files: - split: validation path: niah_multikey_2_128k/validation-* - config_name: niah_multikey_2_16k data_files: - split: validation path: niah_multikey_2_16k/validation-* - config_name: niah_multikey_2_1M data_files: - split: validation path: niah_multikey_2_1M/validation-* - config_name: niah_multikey_2_256k data_files: - split: validation path: niah_multikey_2_256k/validation-* - config_name: niah_multikey_2_32k data_files: - split: validation path: niah_multikey_2_32k/validation-* - config_name: niah_multikey_2_4k data_files: - split: validation path: niah_multikey_2_4k/validation-* - config_name: niah_multikey_2_512k data_files: - split: validation path: niah_multikey_2_512k/validation-* - config_name: niah_multikey_2_64k data_files: - split: validation path: niah_multikey_2_64k/validation-* - config_name: niah_multikey_2_8k data_files: - split: validation path: niah_multikey_2_8k/validation-* - config_name: niah_multikey_3_128k data_files: - split: validation path: niah_multikey_3_128k/validation-* - config_name: niah_multikey_3_16k data_files: - split: validation path: niah_multikey_3_16k/validation-* - config_name: niah_multikey_3_1M data_files: - split: validation path: niah_multikey_3_1M/validation-* - config_name: niah_multikey_3_256k data_files: - split: validation path: niah_multikey_3_256k/validation-* - config_name: niah_multikey_3_32k data_files: - split: validation path: niah_multikey_3_32k/validation-* - config_name: niah_multikey_3_4k data_files: - split: validation path: niah_multikey_3_4k/validation-* - config_name: niah_multikey_3_512k data_files: - split: validation path: niah_multikey_3_512k/validation-* - config_name: niah_multikey_3_64k data_files: - split: validation path: niah_multikey_3_64k/validation-* - config_name: niah_multikey_3_8k data_files: - split: validation path: niah_multikey_3_8k/validation-* - config_name: niah_multiquery_128k data_files: - split: validation path: niah_multiquery_128k/validation-* - config_name: niah_multiquery_16k data_files: - split: validation path: niah_multiquery_16k/validation-* - config_name: niah_multiquery_1M data_files: - split: validation path: niah_multiquery_1M/validation-* - config_name: niah_multiquery_256k data_files: - split: validation path: niah_multiquery_256k/validation-* - config_name: niah_multiquery_32k data_files: - split: validation path: niah_multiquery_32k/validation-* - config_name: niah_multiquery_4k data_files: - split: validation path: niah_multiquery_4k/validation-* - config_name: niah_multiquery_512k data_files: - split: validation path: niah_multiquery_512k/validation-* - config_name: niah_multiquery_64k data_files: - split: validation path: niah_multiquery_64k/validation-* - config_name: niah_multiquery_8k data_files: - split: validation path: niah_multiquery_8k/validation-* - config_name: niah_multivalue_128k data_files: - split: validation path: niah_multivalue_128k/validation-* - config_name: niah_multivalue_16k data_files: - split: validation path: niah_multivalue_16k/validation-* - config_name: niah_multivalue_1M data_files: - split: validation path: niah_multivalue_1M/validation-* - config_name: niah_multivalue_256k data_files: - split: validation path: niah_multivalue_256k/validation-* - config_name: niah_multivalue_32k data_files: - split: validation path: niah_multivalue_32k/validation-* - config_name: niah_multivalue_4k data_files: - split: validation path: niah_multivalue_4k/validation-* - config_name: niah_multivalue_512k data_files: - split: validation path: niah_multivalue_512k/validation-* - config_name: niah_multivalue_64k data_files: - split: validation path: niah_multivalue_64k/validation-* - config_name: niah_multivalue_8k data_files: - split: validation path: niah_multivalue_8k/validation-* - config_name: niah_single_1_128k data_files: - split: validation path: niah_single_1_128k/validation-* - config_name: niah_single_1_16k data_files: - split: validation path: niah_single_1_16k/validation-* - config_name: niah_single_1_1M data_files: - split: validation path: niah_single_1_1M/validation-* - config_name: niah_single_1_256k data_files: - split: validation path: niah_single_1_256k/validation-* - config_name: niah_single_1_32k data_files: - split: validation path: niah_single_1_32k/validation-* - config_name: niah_single_1_4k data_files: - split: validation path: niah_single_1_4k/validation-* - config_name: niah_single_1_512k data_files: - split: validation path: niah_single_1_512k/validation-* - config_name: niah_single_1_64k data_files: - split: validation path: niah_single_1_64k/validation-* - config_name: niah_single_1_8k data_files: - split: validation path: niah_single_1_8k/validation-* - config_name: niah_single_2_128k data_files: - split: validation path: niah_single_2_128k/validation-* - config_name: niah_single_2_16k data_files: - split: validation path: niah_single_2_16k/validation-* - config_name: niah_single_2_1M data_files: - split: validation path: niah_single_2_1M/validation-* - config_name: niah_single_2_256k data_files: - split: validation path: niah_single_2_256k/validation-* - config_name: niah_single_2_32k data_files: - split: validation path: niah_single_2_32k/validation-* - config_name: niah_single_2_4k data_files: - split: validation path: niah_single_2_4k/validation-* - config_name: niah_single_2_512k data_files: - split: validation path: niah_single_2_512k/validation-* - config_name: niah_single_2_64k data_files: - split: validation path: niah_single_2_64k/validation-* - config_name: niah_single_2_8k data_files: - split: validation path: niah_single_2_8k/validation-* - config_name: niah_single_3_128k data_files: - split: validation path: niah_single_3_128k/validation-* - config_name: niah_single_3_16k data_files: - split: validation path: niah_single_3_16k/validation-* - config_name: niah_single_3_1M data_files: - split: validation path: niah_single_3_1M/validation-* - config_name: niah_single_3_256k data_files: - split: validation path: niah_single_3_256k/validation-* - config_name: niah_single_3_32k data_files: - split: validation path: niah_single_3_32k/validation-* - config_name: niah_single_3_4k data_files: - split: validation path: niah_single_3_4k/validation-* - config_name: niah_single_3_512k data_files: - split: validation path: niah_single_3_512k/validation-* - config_name: niah_single_3_64k data_files: - split: validation path: niah_single_3_64k/validation-* - config_name: niah_single_3_8k data_files: - split: validation path: niah_single_3_8k/validation-* - config_name: qa_1_128k data_files: - split: validation path: qa_1_128k/validation-* - config_name: qa_1_16k data_files: - split: validation path: qa_1_16k/validation-* - config_name: qa_1_1M data_files: - split: validation path: qa_1_1M/validation-* - config_name: qa_1_256k data_files: - split: validation path: qa_1_256k/validation-* - config_name: qa_1_32k data_files: - split: validation path: qa_1_32k/validation-* - config_name: qa_1_4k data_files: - split: validation path: qa_1_4k/validation-* - config_name: qa_1_512k data_files: - split: validation path: qa_1_512k/validation-* - config_name: qa_1_64k data_files: - split: validation path: qa_1_64k/validation-* - config_name: qa_1_8k data_files: - split: validation path: qa_1_8k/validation-* - config_name: qa_2_128k data_files: - split: validation path: qa_2_128k/validation-* - config_name: qa_2_16k data_files: - split: validation path: qa_2_16k/validation-* - config_name: qa_2_1M data_files: - split: validation path: qa_2_1M/validation-* - config_name: qa_2_256k data_files: - split: validation path: qa_2_256k/validation-* - config_name: qa_2_32k data_files: - split: validation path: qa_2_32k/validation-* - config_name: qa_2_4k data_files: - split: validation path: qa_2_4k/validation-* - config_name: qa_2_512k data_files: - split: validation path: qa_2_512k/validation-* - config_name: qa_2_64k data_files: - split: validation path: qa_2_64k/validation-* - config_name: qa_2_8k data_files: - split: validation path: qa_2_8k/validation-* - config_name: vt_128k data_files: - split: validation path: vt_128k/validation-* - config_name: vt_16k data_files: - split: validation path: vt_16k/validation-* - config_name: vt_1M data_files: - split: validation path: vt_1M/validation-* - config_name: vt_256k data_files: - split: validation path: vt_256k/validation-* - config_name: vt_32k data_files: - split: validation path: vt_32k/validation-* - config_name: vt_4k data_files: - split: validation path: vt_4k/validation-* - config_name: vt_512k data_files: - split: validation path: vt_512k/validation-* - config_name: vt_64k data_files: - split: validation path: vt_64k/validation-* - config_name: vt_8k data_files: - split: validation path: vt_8k/validation-* license: mit --- # RULER-Llama3-1M A 1M token version of the [RULER dataset](https://arxiv.org/pdf/2404.06654) based on the Llama-3 chat template. It is automatically generated based on the scripts available in the RULER repository: [https://github.com/NVIDIA/RULER](https://github.com/NVIDIA/RULER). It is designed for evaluating the performance of Long Language Models (LLMs) on various tasks with varying sequence lengths. ## How to Use ```python from datasets import load_dataset LENGTH_IN_STRING = ['4k', '8k', '16k', '32k', '64k', '128k', '256k', '512k', '1M'] TASKS = ['niah_single_1', 'niah_single_2', 'niah_single_3', 'niah_multiquery', 'niah_multivalue', 'niah_multikey_1', 'niah_multikey_2', 'niah_multikey_3', 'fwe', 'cwe', 'vt', 'qa_1', 'qa_2'] for length in LENGTH_IN_STRING: for task in TASKS: ds = load_dataset('self-long/RULER-llama3-1M', f'{task}_{length}', split='validation') print(ds[0]) ``` ## Dataset Contents The dataset comprises **13 distinct tasks** with sequence lengths ranging from **4k to 1M** tokens. Each example in the dataset contains the following fields: * **"input"**: This field holds the prompt intended for a Language Model. The prompts are designed for the **Completion** API style, not the Chat API. * **"answers"**: This field contains the expected answers for the corresponding "input" prompt. * **"predictions"**: This field contains the generated completions from three different [SelfLong models](https://arxiv.org/pdf/2412.18860) for the corresponding "input" prompt. ## Evaluation For evaluating the performance of your models on this dataset, please refer to the evaluation script provided in the RULER repository: [https://github.com/NVIDIA/RULER/blob/main/scripts/eval/evaluate.py](https://github.com/NVIDIA/RULER/blob/main/scripts/eval/evaluate.py). ## Caveats 1. **Token Length Definition:** To maintain consistency with the OpenAI API, `128k` refers to **128,000** tokens (instead of the traditional 128 * 1024 = 131,072). For other lengths, we use `1k = 1024` tokens. 2. For inference efficiency, the tasks with lengths **128k, 256k, 512k, and 1M** each contain only **100 examples**. The tasks with shorter lengths have 500 examples. ## References ``` @article{hsieh2024ruler, title={RULER: What's the Real Context Size of Your Long-Context Language Models?}, author={Hsieh, Cheng-Ping and Sun, Simeng and Kriman, Samuel and Acharya, Shantanu and Rekesh, Dima and Jia, Fei and Zhang, Yang and Ginsburg, Boris}, journal={arXiv preprint arXiv:2404.06654}, year={2024} } @article{wang2024bootstrap, title={Bootstrap Your Own Context Length}, author={Wang, Liang and Yang, Nan and Zhang, Xingxing and Huang, Xiaolong and Wei, Furu}, journal={arXiv preprint arXiv:2412.18860}, year={2024} } ```

应用场景：