Evaluating Objective Speech Quality Metrics for Neural Audio Codecs

Neural audio codecs have gained recent popularity for their use in generative modeling as they offer high-fidelity audio reconstruction at low bitrates. While human listening studies remain the gold standard for assessing perceptual quality, they are time-consuming and impractical. In this work, we examine the reliability of existing objective quality metrics in assessing the performance of recent neural audio codecs. To this end, we conduct a MUSHRA listening test on high-fidelity speech signals and analyze the correlation between subjective scores and widely used objective metrics. Our results show that, while some metrics align well with human perception, others struggle to capture relevant distortions. Our findings provide practical guidance for selecting appropriate evaluation metrics when using neural audio codecs for speech.

Here we provide the data for reproducing our MUSHRA listening experiments on neural audio codecs: the csv contains per-trial ratings and the corresponding audio stimuli presented to listeners. The audio dataset includes the reference, two anchors (mid/low), and the codec reconstructions for both clean speech and speech-with-background (“combined”) conditions.

The accompanying paper can be found at https://arxiv.org/abs/2511.19734

Our work builds on ODAQ and uses their dataset: https://zenodo.org/records/10405774

To cite this dataset use:

@article{lanzendorfer2025evaluating,
title={Evaluating Objective Speech Quality Metrics for Neural Audio Codecs},
author={Lanzend{\"o}rfer, Luca A and Gr{\"o}tschla, Florian},
journal={arXiv preprint arXiv:2511.19734},
year={2025}
}