aviralchawla/metaothello

---
license: mit
task_categories:
- text-generation
language:
- en
tags:
- othello
- mechanistic-interpretability
- world-models
- board-games
- transformer
- toy-model
pretty_name: MetaOthello Training Data
size_categories:
- 10M<n<100M
---

# MetaOthello: Training Data

Training data for **MetaOthello**, a controlled suite of Othello game variants used to study how transformers organize multiple world models internally.

> **Paper:** [MetaOthello: A Controlled Study of Multiple World Models in Transformers](pending_link)
>
> **Code:** [github.com/aviralchawla/metaothello](https://github.com/aviralchawla/metaothello)
>
> **Models & Probes:** [huggingface.co/aviralchawla/metaothello](https://huggingface.co/aviralchawla/metaothello)

## Dataset Contents

This repository contains 20 million complete game sequences for each of the four MetaOthello variants (80M games total), stored as Zarr datasets:

| File | Game Variant | Description |
|---|---|---|
| `train_classic_20M.zarr` | Classic | Standard Othello — flip all flanked pieces |
| `train_nomidflip_20M.zarr` | NoMidFlip | Flip only endpoints of flanked sequences |
| `train_delflank_20M.zarr` | DelFlank | Delete flanked pieces; open-spread init; neighbor validation |
| `train_iago_20M.zarr` | Iago | Identical rules to Classic, but with a scrambled token vocabulary |

## Data Format

Each Zarr store is an xarray dataset with the following variables:

| Variable | Shape | Dtype | Description |
|---|---|---|---|
| `seqs` | `(20_000_000, 60)` | `int32` | Tokenized move sequences (60 moves per game) |
| `board_state` | `(20_000_000, 60, 8, 8)` | `float64` | Board snapshots after each move (`-1` = Black, `1` = White, `0` = Empty) |

### Token vocabulary

| Token ID | Meaning |
|---|---|
| 0 | Padding |
| 1–64 | Board squares `a1`–`h8` |
| 65 | Pass move |

**Vocabulary size:** 66. Each game is exactly 60 moves.

## Usage

### Stream directly (no download required)

```python
import xarray as xr

ds = xr.open_zarr("hf://datasets/aviralchawla/metaothello/train_classic_20M.zarr")
seqs = ds["seqs"] # Lazy xarray DataArray (20M, 60)
boards = ds["board_state"] # Lazy xarray DataArray (20M, 60, 8, 8)
```

### Download via the MetaOthello CLI

```bash
# Clone the repository
git clone https://github.com/aviralchawla/metaothello.git
cd metaothello && pip install -e .

# Download all training data
make download-data

# Download a single game variant
make download-data-game GAME=classic
```

Data is placed into `data/{game}/train_{game}_20M.zarr`.

### Download with huggingface_hub

```python
from huggingface_hub import snapshot_download

snapshot_download(
repo_id="aviralchawla/metaothello",
repo_type="dataset",
allow_patterns=["train_classic_20M.zarr/**"],
local_dir="./data",
)
```

## Generating Data from Scratch

To regenerate training data locally (or generate additional splits):

```bash
# Generate N million games for a variant
make generate-data GAME=classic N_GAMES=20 SPLIT=train

# Generate all four variants (20M each)
make generate-data-all-train
```

## Game Variants

All four variants share the same 8x8 board and 64-square coordinate system but differ in their rules:

- **Classic**: Standard Othello. Placing a piece flips all opponent pieces flanked in any direction.
- **NoMidFlip**: Only the two endpoints of each flanked sequence are flipped, leaving interior pieces unchanged. This creates high game-tree overlap with Classic.
- **DelFlank**: Flanked pieces are deleted rather than flipped. Uses an open-spread initial board and neighbor-based move validation. Very different game dynamics from Classic.
- **Iago**: Identical rules to Classic but board squares are mapped to tokens via a fixed permutation. Serves as an isomorphic control — the model must learn the same latent structure through a different surface vocabulary.

## Upcoming

Game ID probes (linear classifiers that predict which game variant is being played), along with training scripts and analysis plotting scripts, are currently in development and will be added in a future update. See the [GitHub repository](https://github.com/aviralchawla/metaothello) for the latest status.

## Citation

```bibtex
@article{metaothello2025,
title = {MetaOthello: A Controlled Study of Multiple World Models in Transformers},
author = {Aviral Chawla, Galen Hall, Juniper Lovato},
journal = {arXiv preprint},
year = {2025}
}
```

## License

MIT