II-Search-Benchmark-Details

# Inspect-Search-Models-Benchmarking-Result

Overall result

| | Qwen 4B | Jan 4B | WebSailor-3B | II-Search-4B | II-Search-CIR-4B|
| --- | --- | --- | --- | --- |--- |
| OpenAI/SimpleQA | 76.8 | 80.1 | 81.8 | 91.8 | 91.8 |
| Google/Frames | 30.7 | 24.8 | 34.0 | 67.5 | 72.2 |
| Seal_0 | 6.31 | 2.7 | 1.8 | 22.5 | 26.4|

Simple QA (SerpDev)

| | Qwen 4B | Jan 4B | WebSailor-3B | II-Search-4B |II-Search-CIR-4B|
| --- | --- | --- | --- | --- |--- |
| Pass rate % | 76.8 | 80.1 | 81.8 | 91.8 |91.8 |
| # Search | 1.0 | 0.9 | 2.1 | 2.2 | 2.5|
| # Visit | 0.1 | 1.9 | 6.4 | 3.5 | 5.3|
| # Tool used | 1.1 | 2.8 | 8.5 | 5.7 | 7.8|

Frames (SerpDev)

| | Qwen 4B | Jan 4B | WebSailor-3B | II-Search-4B |II-Search-CIR-4B|
| --- | --- | --- | --- | --- |--- |
| Pass rate % | 30.7 | 24.8 | 34.0 | 67.5 |72.2 |
| # Search | 1.1 | 1.0 | 7.4 | 4.2 |6.1 |
| # Visit | 0.1 | 3.7 | 7.2 | 3.2 |5.0 |
| # Tool used | 1.2 | 4.7 | 14.6 | 7.4 |11.1 |

Seal_0 (SerpDev)

| | Qwen 4B | Jan 4B | WebSailor-3B | II-Search-4B | II-Search-CIR-4B|
| --- | --- | --- | --- | --- |--- |
| Pass rate % | 6.31 | 2.7 | 1.8 | 22.5 | 26.4|
| # Search | 0.9 | 0.9 | 6.6 | 4.3 | 5.9|
| # Visit | 0.1 | 5.2 | 10.0 | 5.7 | 7.7|
| # Tool used | 1.0 | 6.1 | 16.6 | 10.0 | 13.5|

# 检索模型基准测试结果集（Inspect-Search-Models-Benchmarking-Result）

整体基准测试结果
| | Qwen 4B | Jan 4B | WebSailor-3B | II-Search-4B | II-Search-CIR-4B |
| --- | --- | --- | --- | --- | --- |
| OpenAI/SimpleQA数据集 | 76.8 | 80.1 | 81.8 | 91.8 | 91.8 |
| Google/Frames数据集 | 30.7 | 24.8 | 34.0 | 67.5 | 72.2 |
| Seal_0数据集 | 6.31 | 2.7 | 1.8 | 22.5 | 26.4 |

简单问答任务（Simple QA, 搜索开发基准（SerpDev））
| | Qwen 4B | Jan 4B | WebSailor-3B | II-Search-4B | II-Search-CIR-4B |
| --- | --- | --- | --- | --- | --- |
| 通过率（%） | 76.8 | 80.1 | 81.8 | 91.8 | 91.8 |
| 检索次数 | 1.0 | 0.9 | 2.1 | 2.2 | 2.5 |
| 访问次数 | 0.1 | 1.9 | 6.4 | 3.5 | 5.3 |
| 工具使用次数 | 1.1 | 2.8 | 8.5 | 5.7 | 7.8 |

框架任务（Frames, 搜索开发基准（SerpDev））
| | Qwen 4B | Jan 4B | WebSailor-3B | II-Search-4B | II-Search-CIR-4B |
| --- | --- | --- | --- | --- | --- |
| 通过率（%） | 30.7 | 24.8 | 34.0 | 67.5 | 72.2 |
| 检索次数 | 1.1 | 1.0 | 7.4 | 4.2 | 6.1 |
| 访问次数 | 0.1 | 3.7 | 7.2 | 3.2 | 5.0 |
| 工具使用次数 | 1.2 | 4.7 | 14.6 | 7.4 | 11.1 |

Seal_0任务（搜索开发基准（SerpDev））
| | Qwen 4B | Jan 4B | WebSailor-3B | II-Search-4B | II-Search-CIR-4B |
| --- | --- | --- | --- | --- | --- |
| 通过率（%） | 6.31 | 2.7 | 1.8 | 22.5 | 26.4 |
| 检索次数 | 0.9 | 0.9 | 6.6 | 4.3 | 5.9 |
| 访问次数 | 0.1 | 5.2 | 10.0 | 5.7 | 7.7 |
| 工具使用次数 | 1.0 | 6.1 | 16.6 | 10.0 | 13.5 |

搜集汇总

背景与挑战

背景概述

该数据集由Intelligent-Internet发布，基于Apache 2.0许可证，提供了搜索模型的基准测试详细结果，包括Qwen 4B、II-Search-4B等多个模型在SimpleQA、Frames和Seal_0任务上的性能比较，如通过率、搜索和工具使用次数。数据集于2025年8月更新，旨在评估和对比不同搜索模型的效能。