optillm-router-dataset

This dataset was used to train the [optillm-modernbert-large-bert](https://huggingface.co/codelion/optillm-modernbert-large) and [optillm-bert-uncased](https://huggingface.co/codelion/optillm-bert-uncased) router classifier model. It was built by combining all instances of [Arena Hard Auto](https://huggingface.co/datasets/lmarena-ai/arena-hard-auto-v0.1) and [MixEval](https://huggingface.co/datasets/MixEval/MixEval) datasets and running them through the [optillm](https://github.com/codelion/optillm) proxy with `gpt-4o-mini`. We generated responses using optillm for all the approaches and then evaluated them using LLM-as-Judge for Arena Hard Auto and ground truth for MixEval. These responses were ranked and saved along with the number of tokens required for the completion of the response. The scripts for generating the dataset are available in the optillm GitHub repo [here](https://github.com/codelion/optillm/blob/main/scripts/gen_optillm_dataset.py) and [here](https://github.com/codelion/optillm/blob/main/scripts/gen_optillm_ground_truth_dataset.py). ## Citation If you use this in your work, please cite: ```bibtex @software{optillm, title = {Optillm: Optimizing inference proxy for LLMs}, author = {Asankhaya Sharma}, year = {2024}, publisher = {GitHub}, url = {https://github.com/codelion/optillm} } ```