diamond-benchmark

<div align="center"> # 💎 Diamond Benchmark ### 750 real degraded speech recordings for evaluating restoration models [](.) [](.) [](.) [](https://huggingface.co/nineninesix/diamond-1.0) </div> --- These are **real, genuinely degraded** speech recordings — not synthetic, not clean audio re-degraded by a pipeline. Each clip carries the damage of an actual real-world capture: low-bitrate codec compression, narrow bandwidth, background noise, and clipping. Together they form a **fixed benchmark** for measuring how well a speech-restoration model recovers clean, studio-quality audio from real degradation. Every clip ships with its reference transcript, so restoration can be scored on **two axes at once** — perceptual quality *and* content preservation. ## What's inside | | | |---|---| | **Clips** | 750 degraded `.wav` recordings | | **Language** | English | | **Audio** | real degraded speech (codec · band-limit · noise · clipping) | | **Reference** | ground-truth transcript per clip | ## Structure ``` diamond-bench/ ├── audio/ │ └── <set>_<id>.wav # 750 degraded input clips └── manifest.csv # per-clip metadata ``` `manifest.csv` columns: | column | meaning | |---|---| | `set` | source subset the clip belongs to | | `id` | clip id within its set | | `emolia_id` | original recording id | | `audio_path` | relative path under `audio/` | | `sample_rate` | clip sample rate (Hz) | | `duration_sec` | clip length (seconds) | | `speaker` | speaker id | | `text` | reference transcript (used for CER) | ## How to score Two complementary metrics — one alone is not enough: - **DNSMOS-P.835** (`sig_bak_ovr.onnx`, raw windowed mean) — perceptual quality (SIG / BAK / OVRL). How good the audio *sounds*. - **CER** (ASR vs. `text`) — content preservation. A generative restorer can sound clean while smearing or dropping words; DNSMOS is blind to that, CER is not. A genuine gain means **OVRL rises without CER rising**. ## Loading ```python import pandas as pd import soundfile as sf from huggingface_hub import snapshot_download root = snapshot_download("nineninesix/diamond-benchmark", repo_type="dataset") meta = pd.read_csv(f"{root}/manifest.csv") row = meta.iloc[0] wav, sr = sf.read(f"{root}/{row.audio_path}") print(row.id, f"{row.duration_sec:.1f}s —", row.text) ``` ## Links - **Model:** [`nineninesix/diamond-1.0`](https://huggingface.co/nineninesix/diamond-1.0) - **Training:** [`diamond-train`](https://github.com/nineninesix-ai/diamond-train.git) - **Inference:** [`diamond-inference`](https://github.com/nineninesix-ai/diamond-inference.git)