invi-bhagyesh/ValueArena

--- license: mit tags: - value-alignment - eigenbench - elo-ranking - benchmark --- # ValueArena Data repository for [ValueArena](https://valuearena.github.io) — a leaderboard for [EigenBench](https://github.com/jchang153/EigenBench) value alignment experiments. EigenBench is a black-box framework for quantifying value alignment across language models. Models judge each other's responses in pairwise comparisons, fitted with a Bradley-Terry-Davison model and aggregated via EigenTrust into consensus alignment scores. ## Structure ``` index.json # manifest of all runs runs/ {group}/{persona}/ # e.g. oct-v2/goodness meta.json # spec + training log + eigentrust + git info summary.json # bootstrap Elo rankings images/ eigenbench.png training_loss.png uv_embeddings_pca.png bootstrap_elo.png ``` ## Usage This dataset is consumed by the [ValueArena website](https://valuearena.github.io) at page load. No download needed. To upload new results, use the upload script from the EigenBench repo: ```bash # Single run python3 scripts/upload_results.py --name "my-run" --run-dir runs/my_run/ # Batch python3 scripts/upload_results.py --batch-dir runs/matrix/ --name "oct-v2" --note "LoRA + API models" ``` ## Models Runs include character-trained LoRA models from [Open Character Training](https://huggingface.co/collections/maius/open-character-training), base models, and API models (GPT-4o, Claude, Gemini). ## Links - **Website:** [valuearena.github.io](https://valuearena.github.io) - **EigenBench:** [github.com/jchang153/EigenBench](https://github.com/jchang153/EigenBench) - **Website Source:** [github.com/ValueArena/ValueArena.github.io](https://github.com/ValueArena/ValueArena.github.io)