ulab-ai/PersonaRoute-Bench

---
license: apache-2.0
task_categories:
- graph-ml
language:
- en
size_categories:
- 10K<n<100K
tags:
- graph-ml
- large-language-model
pretty_name: PersonaRoute-Bench
configs:
- config_name: Router_user_data_v1
data_files:
- split: train
path: router_user_train_data_v1.csv
- split: val
path: router_user_val_data_v1.csv
- split: test
path: router_user_test_data_v1.csv
- split: raw
path: router_user_data_v1.csv
- config_name: Router_user_data_v2
data_files:
- split: train
path: router_user_train_data_v2.csv
- split: val
path: router_user_val_data_v2.csv
- split: test
path: router_user_test_data_v2.csv
- split: raw
path: router_user_data_v2.csv
- config_name: Router_user_data_v1_large
data_files:
- split: train
path: router_user_train_data_v1_large.csv
- split: val
path: router_user_val_data_v1_large.csv
- split: test
path: router_user_test_data_v1_large.csv
- config_name: Router_user_data_v2_large
data_files:
- split: train
path: router_user_train_data_v2_large.csv
- split: val
path: router_user_val_data_v2_large.csv
- split: test
path: router_user_test_data_v2_large.csv
- config_name: LLM_judge_results
data_files:
- split: raw
path: raw/llm_judge_results.csv
- config_name: LLM_judge_results_large
data_files:
- split: raw
path: raw/llm_judge_results_large.csv
- config_name: Router_data_v1
data_files:
- split: raw
path: raw/router_data.csv
- config_name: Router_data_v2
data_files:
- split: raw
path: raw/router_data_v2.csv
- config_name: QA
data_files:
- split: raw
path: raw/unified_qa_data.csv
---

This repository contains the datasets presented in the paper **PersonalizedRouter**

In the project files, the suffix `v1` refers to the `Multi-cost-efficiency Simulation Strategy` described in the paper, while `v2` refers to the `LLM-as-a-Judge Simulation`, and `large` denotes the large-scale setting.

You can utilize `router_user_data_v1` (or `v2`) to train and test `PersonalizedRouter`.

In `router_user_data_v1`, we collected the responses of 10 candidate LLMs to 240 questions under 9 different performance and cost settings.

In `router_user_data_v2`, we collected the responses of 10 candidate LLMs to 240 questions and simulated the preferences of 9 different user groups for these responses.

In `router_user_data_v1_large`, we collected the responses of 10 candidate LLMs to 240 questions under 1000 different performance and cost settings.

In `router_user_data_v2_large`, we collected the responses of 10 candidate LLMs to 240 questions and simulated the preferences of 1200 different user groups for these responses.