invi-bhagyesh/ValueArena

---
license: mit
tags:
- value-alignment
- eigenbench
- elo-ranking
- benchmark
---

# ValueArena

Data repository for [ValueArena](https://valuearena.github.io) — a leaderboard for [EigenBench](https://github.com/jchang153/EigenBench) value alignment experiments.

EigenBench is a black-box framework for quantifying value alignment across language models. Models judge each other's responses in pairwise comparisons, fitted with a Bradley-Terry-Davison model and aggregated via EigenTrust into consensus alignment scores.

## Structure

```
index.json # manifest of all runs
runs/
{group}/{persona}/ # e.g. oct-v2/goodness
meta.json # spec + training log + eigentrust + git info
summary.json # bootstrap Elo rankings
images/
eigenbench.png
training_loss.png
uv_embeddings_pca.png
bootstrap_elo.png
```

## Usage

This dataset is consumed by the [ValueArena website](https://valuearena.github.io) at page load. No download needed.

To upload new results, use the upload script from the EigenBench repo:

```bash
# Single run
python3 scripts/upload_results.py --name "my-run" --run-dir runs/my_run/

# Batch
python3 scripts/upload_results.py --batch-dir runs/matrix/ --name "oct-v2" --note "LoRA + API models"
```

## Models

Runs include character-trained LoRA models from [Open Character Training](https://huggingface.co/collections/maius/open-character-training), base models, and API models (GPT-4o, Claude, Gemini).

## Links

- **Website:** [valuearena.github.io](https://valuearena.github.io)
- **EigenBench:** [github.com/jchang153/EigenBench](https://github.com/jchang153/EigenBench)
- **Website Source:** [github.com/ValueArena/ValueArena.github.io](https://github.com/ValueArena/ValueArena.github.io)