Qwen2.5-7B-Instruct_qwq_mix_r1_science_eval_8179

# mlfoundations-dev/Qwen2.5-7B-Instruct_qwq_mix_r1_science_eval_8179 Precomputed model outputs for evaluation. ## Evaluation Results ### Summary | Metric | AIME24 | AMC23 | MATH500 | JEEBench | GPQADiamond | LiveCodeBench | CodeElo | CodeForces | AIME25 | HLE | LiveCodeBenchv5 | HMMT | |--------|------|-----|-------|--------|-----------|-------------|-------|----------|------|---|---------------|----| | Accuracy | 60.7 | 90.8 | 89.4 | 63.2 | 52.4 | 48.5 | 27.4 | 26.2 | 48.3 | 12.0 | 34.3 | 34.7 | ### AIME24 - **Average Accuracy**: 60.67% ± 2.25% - **Number of Runs**: 10 | Run | Accuracy | Questions Solved | Total Questions | |-----|----------|-----------------|----------------| | 1 | 63.33% | 19 | 30 | | 2 | 66.67% | 20 | 30 | | 3 | 63.33% | 19 | 30 | | 4 | 50.00% | 15 | 30 | | 5 | 53.33% | 16 | 30 | | 6 | 53.33% | 16 | 30 | | 7 | 70.00% | 21 | 30 | | 8 | 53.33% | 16 | 30 | | 9 | 63.33% | 19 | 30 | | 10 | 70.00% | 21 | 30 | ### AMC23 - **Average Accuracy**: 90.75% ± 1.00% - **Number of Runs**: 10 | Run | Accuracy | Questions Solved | Total Questions | |-----|----------|-----------------|----------------| | 1 | 85.00% | 34 | 40 | | 2 | 90.00% | 36 | 40 | | 3 | 90.00% | 36 | 40 | | 4 | 92.50% | 37 | 40 | | 5 | 87.50% | 35 | 40 | | 6 | 92.50% | 37 | 40 | | 7 | 97.50% | 39 | 40 | | 8 | 92.50% | 37 | 40 | | 9 | 90.00% | 36 | 40 | | 10 | 90.00% | 36 | 40 | ### MATH500 - **Accuracy**: 89.40% | Accuracy | Questions Solved | Total Questions | |----------|-----------------|----------------| | 89.40% | 447 | 500 | ### JEEBench - **Average Accuracy**: 63.24% ± 0.75% - **Number of Runs**: 3 | Run | Accuracy | Questions Solved | Total Questions | |-----|----------|-----------------|----------------| | 1 | 64.22% | 330.75 | 515 | | 2 | 61.41% | 316.25 | 515 | | 3 | 64.08% | 330.0 | 515 | ### GPQADiamond - **Average Accuracy**: 52.36% ± 1.17% - **Number of Runs**: 3 | Run | Accuracy | Questions Solved | Total Questions | |-----|----------|-----------------|----------------| | 1 | 54.04% | 107 | 198 | | 2 | 49.49% | 98 | 198 | | 3 | 53.54% | 106 | 198 | ### LiveCodeBench - **Average Accuracy**: 48.53% ± 0.96% - **Number of Runs**: 6 | Run | Accuracy | Questions Solved | Total Questions | |-----|----------|-----------------|----------------| | 1 | 49.12% | 251 | 511 | | 2 | 50.10% | 256 | 511 | | 3 | 50.88% | 260 | 511 | | 4 | 45.99% | 235 | 511 | | 5 | 45.21% | 231 | 511 | | 6 | 49.90% | 255 | 511 | ### CodeElo - **Average Accuracy**: 27.37% ± 0.44% - **Number of Runs**: 3 | Run | Accuracy | Questions Solved | Total Questions | |-----|----------|-----------------|----------------| | 1 | 28.13% | 110 | 391 | | 2 | 27.37% | 107 | 391 | | 3 | 26.60% | 104 | 391 | ### CodeForces - **Average Accuracy**: 26.20% ± 1.15% - **Number of Runs**: 3 | Run | Accuracy | Questions Solved | Total Questions | |-----|----------|-----------------|----------------| | 1 | 26.05% | 118 | 453 | | 2 | 28.26% | 128 | 453 | | 3 | 24.28% | 110 | 453 | ### AIME25 - **Average Accuracy**: 48.33% ± 1.35% - **Number of Runs**: 10 | Run | Accuracy | Questions Solved | Total Questions | |-----|----------|-----------------|----------------| | 1 | 56.67% | 17 | 30 | | 2 | 50.00% | 15 | 30 | | 3 | 53.33% | 16 | 30 | | 4 | 46.67% | 14 | 30 | | 5 | 43.33% | 13 | 30 | | 6 | 43.33% | 13 | 30 | | 7 | 50.00% | 15 | 30 | | 8 | 50.00% | 15 | 30 | | 9 | 46.67% | 14 | 30 | | 10 | 43.33% | 13 | 30 | ### HLE - **Average Accuracy**: 11.96% ± 0.21% - **Number of Runs**: 3 | Run | Accuracy | Questions Solved | Total Questions | |-----|----------|-----------------|----------------| | 1 | 11.70% | 60 | 513 | | 2 | 11.70% | 60 | 513 | | 3 | 12.48% | 64 | 513 | ### LiveCodeBenchv5 - **Average Accuracy**: 34.33% ± 1.89% - **Number of Runs**: 3 | Run | Accuracy | Questions Solved | Total Questions | |-----|----------|-----------------|----------------| | 1 | 37.40% | 138 | 369 | | 2 | 30.89% | 114 | 369 | | 3 | 34.69% | 128 | 369 | ### HMMT - **Average Accuracy**: 34.67% ± 1.35% - **Number of Runs**: 10 | Run | Accuracy | Questions Solved | Total Questions | |-----|----------|-----------------|----------------| | 1 | 40.00% | 12 | 30 | | 2 | 33.33% | 10 | 30 | | 3 | 36.67% | 11 | 30 | | 4 | 33.33% | 10 | 30 | | 5 | 33.33% | 10 | 30 | | 6 | 36.67% | 11 | 30 | | 7 | 43.33% | 13 | 30 | | 8 | 30.00% | 9 | 30 | | 9 | 30.00% | 9 | 30 | | 10 | 30.00% | 9 | 30 |