Eurus-2-7B-SFT_eval_2e29

# mlfoundations-dev/Eurus-2-7B-SFT_eval_2e29 Precomputed model outputs for evaluation. ## Evaluation Results ### Summary | Metric | AIME24 | AMC23 | MATH500 | MMLUPro | JEEBench | GPQADiamond | LiveCodeBench | CodeElo | CodeForces | AIME25 | HLE | LiveCodeBenchv5 | |--------|------|-----|-------|-------|--------|-----------|-------------|-------|----------|------|---|---------------| | Accuracy | 2.3 | 21.0 | 30.6 | 11.0 | 11.4 | 10.4 | 6.8 | 1.5 | 2.1 | 1.3 | 4.1 | 4.4 | ### AIME24 - **Average Accuracy**: 2.33% ± 0.67% - **Number of Runs**: 10 | Run | Accuracy | Questions Solved | Total Questions | |-----|----------|-----------------|----------------| | 1 | 0.00% | 0 | 30 | | 2 | 3.33% | 1 | 30 | | 3 | 6.67% | 2 | 30 | | 4 | 3.33% | 1 | 30 | | 5 | 3.33% | 1 | 30 | | 6 | 3.33% | 1 | 30 | | 7 | 0.00% | 0 | 30 | | 8 | 3.33% | 1 | 30 | | 9 | 0.00% | 0 | 30 | | 10 | 0.00% | 0 | 30 | ### AMC23 - **Average Accuracy**: 21.00% ± 1.42% - **Number of Runs**: 10 | Run | Accuracy | Questions Solved | Total Questions | |-----|----------|-----------------|----------------| | 1 | 22.50% | 9 | 40 | | 2 | 20.00% | 8 | 40 | | 3 | 22.50% | 9 | 40 | | 4 | 27.50% | 11 | 40 | | 5 | 22.50% | 9 | 40 | | 6 | 20.00% | 8 | 40 | | 7 | 17.50% | 7 | 40 | | 8 | 10.00% | 4 | 40 | | 9 | 22.50% | 9 | 40 | | 10 | 25.00% | 10 | 40 | ### MATH500 - **Accuracy**: 30.60% | Accuracy | Questions Solved | Total Questions | |----------|-----------------|----------------| | 30.60% | 153 | 500 | ### MMLUPro - **Average Accuracy**: 11.00% ± 0.00% - **Number of Runs**: 1 | Run | Accuracy | Questions Solved | Total Questions | |-----|----------|-----------------|----------------| | 1 | 11.00% | 55 | 500 | ### JEEBench - **Average Accuracy**: 11.36% ± 0.67% - **Number of Runs**: 3 | Run | Accuracy | Questions Solved | Total Questions | |-----|----------|-----------------|----------------| | 1 | 12.18% | 62.75 | 515 | | 2 | 12.18% | 62.75 | 515 | | 3 | 9.71% | 50.0 | 515 | ### GPQADiamond - **Average Accuracy**: 10.44% ± 0.50% - **Number of Runs**: 3 | Run | Accuracy | Questions Solved | Total Questions | |-----|----------|-----------------|----------------| | 1 | 10.10% | 20 | 198 | | 2 | 11.62% | 23 | 198 | | 3 | 9.60% | 19 | 198 | ### LiveCodeBench - **Average Accuracy**: 6.78% ± 0.79% - **Number of Runs**: 3 | Run | Accuracy | Questions Solved | Total Questions | |-----|----------|-----------------|----------------| | 1 | 5.48% | 28 | 511 | | 2 | 6.65% | 34 | 511 | | 3 | 8.22% | 42 | 511 | ### CodeElo - **Average Accuracy**: 1.53% ± 0.15% - **Number of Runs**: 3 | Run | Accuracy | Questions Solved | Total Questions | |-----|----------|-----------------|----------------| | 1 | 1.28% | 5 | 391 | | 2 | 1.79% | 7 | 391 | | 3 | 1.53% | 6 | 391 | ### CodeForces - **Average Accuracy**: 2.06% ± 0.19% - **Number of Runs**: 3 | Run | Accuracy | Questions Solved | Total Questions | |-----|----------|-----------------|----------------| | 1 | 1.77% | 8 | 453 | | 2 | 2.43% | 11 | 453 | | 3 | 1.99% | 9 | 453 | ### AIME25 - **Average Accuracy**: 1.33% ± 0.52% - **Number of Runs**: 10 | Run | Accuracy | Questions Solved | Total Questions | |-----|----------|-----------------|----------------| | 1 | 0.00% | 0 | 30 | | 2 | 0.00% | 0 | 30 | | 3 | 0.00% | 0 | 30 | | 4 | 3.33% | 1 | 30 | | 5 | 0.00% | 0 | 30 | | 6 | 3.33% | 1 | 30 | | 7 | 0.00% | 0 | 30 | | 8 | 3.33% | 1 | 30 | | 9 | 3.33% | 1 | 30 | | 10 | 0.00% | 0 | 30 | ### HLE - **Average Accuracy**: 4.10% ± 0.18% - **Number of Runs**: 3 | Run | Accuracy | Questions Solved | Total Questions | |-----|----------|-----------------|----------------| | 1 | 3.71% | 19 | 512 | | 2 | 4.10% | 21 | 512 | | 3 | 4.49% | 23 | 512 | ### LiveCodeBenchv5 - **Average Accuracy**: 4.43% ± 0.09% - **Number of Runs**: 3 | Run | Accuracy | Questions Solved | Total Questions | |-----|----------|-----------------|----------------| | 1 | 4.34% | 16 | 369 | | 2 | 4.34% | 16 | 369 | | 3 | 4.61% | 17 | 369 |