IBRAHIM1990/search-arena-24k

---
size_categories:
- 10K<n<100K
configs:
- config_name: default
data_files:
- split: test
path: data/search-arena-*
---
## Overview

This dataset contains **ALL** in-the-wild conversation crowdsourced from [Search Arena](https://lmarena.ai/?search) between March 18, 2025 and May 8, 2025. It includes **24,069** multi-turn conversations with search-LLMs across diverse intents, languages, and topics—alongside **12,652** human preference votes. The dataset spans approximately 11,000 users across 136 countries, 13 publicly released models, around 90 languages (including 11\% multilingual prompts), and over 5,000 multi-turn sessions.

While user interaction patterns with general-purpose LLMs and traditional search engines are increasingly well-understood, we believe that search-LLMs represent a new and understudied interface—blending open-ended generation with real-time retrieval. This hybrid interaction mode introduces new user behaviors: how questions are posed, how retrieved information is interpreted, and what types of responses are preferred. We release this dataset to support analysis of this emerging paradigm in human–AI interaction, grounded in large-scale, real-world usage.

All users consented the terms of use to share their conversational data with us for research purposes, and all entries have been redacted using Google’s Data Loss Prevention (DLP) API to remove PII and sensitive information.

Each data point includes:
- Two model standardized responses (`messages_a` and `messages_b`)
- The human vote result (half of them have this feature)
- Timestamp
- Full system metadata, LLM + web search trace
- Post-processed annotations such as language and user intent

## Transparency Statement
We clarify the following for transparency:
(1) No pre-release models were deployed to Search Arena during data collection.
(2) All model responses are anonymized to ensure that user preferences reflect conversational quality rather than brand identity.
(3) All anonymized data is publicly released; no model provider received privileged access.

For more details, see our [blog post](https://lmarena.ai/blog/search-arena) and our [paper (ICLR 2026)](https://arxiv.org/abs/2506.05334).

## License
User prompts are licensed under CC-BY-4.0, and model outputs are governed by the terms of use set by the respective model providers.

## Citation
```bibtex
@inproceedings{miroyan2025searcharenaanalyzingsearchaugmented,
title={Search Arena: Analyzing Search-Augmented LLMs},
author={Mihran Miroyan and Tsung-Han Wu and Logan King and Tianle Li and Jiayi Pan and Xinyan Hu and Wei-Lin Chiang and Anastasios N. Angelopoulos and Trevor Darrell and Narges Norouzi and Joseph E. Gonzalez},
year={2026},
booktitle={The Thirteenth International Conference on Learning Representations},
url={https://openreview.net/forum?id=MMGRlDnhtI}
}

@misc{chiang2024chatbot,
title={Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference},
author={Wei-Lin Chiang and Lianmin Zheng and Ying Sheng and Anastasios Nikolas Angelopoulos and Tianle Li and Dacheng Li and Hao Zhang and Banghua Zhu and Michael Jordan and Joseph E. Gonzalez and Ion Stoica},
year={2024},
eprint={2403.04132},
archivePrefix={arXiv},
primaryClass={cs.AI}
}
```