Audiocards: Structured Metadata Improves Audio Language Models for Sound Design

Paper abstract Sound designers search for sounds in large sound effects libaries using aspects such as sound class or visual context. However, the metadata needed for such search is often missing or incomplete, and requires significant manual effort to add. Existing solutions to automate this task by generating metadata, i.e. captioning, and search using learned embeddings, i.e. text-audio retrieval, are not trained on metadata with the structure and information pertinent to sound design. To this end we propose audiocards, structured metadata grounded in acoustic attributes and sonic descriptors, by exploiting the world knowledge of LLMs. We show that training on audiocards improves downstream text-audio retrieval, descriptive captioning, and metadata generation on professional sound effects libraries. Moreover, audiocards also improve performance on general audio captioning and retrieval over the baseline single-sentence captioning approach. We release a curated dataset of sound effects audiocards to invite further research in audio language modeling for sound design. Dataset: ASFx Eval Audiocards We release ASFx Eval, a curated dataset of audiocards which are structured metadata with attributes pertinent to sound design such as noun-verb pairs and example visual context . Using Pixtral-12B-2409 we generated 500 audiocards for corresponding audio files in Adobe Audition Sound Effects and manually screened them for incoherence and hallucinations. These audiocards can be used to evaluate audio language models on tasks such as structured metadata generation, text-audio retrieval, and audio captioning, by pairing them with the corresponding audio file using the filename column in the csv below. For example, the audiocard with filename "Impacts/Impact Metal Spatula Hits 08.wav" corresponds to the "Impact Metal Spatula Hits 08.wav" sound effect under the "Impacts" category. Each of the remaining columns corresponds to an audiocard field. We also provide Universal Category System (UCS) category and subcategory tags. UCS is an ontology developed for sound design and commonly used to organize professional libraries. We generated UCS tags from a text classifier that achieved 96% accuracy on a separate dataset with human annotated tags. These tags can be used to evaluate audio language models and audio representations on UCS category and subcategory prediction for sound effects library management. Please see the companion website for more details.Referencing this work If you use this dataset in your work, please cite our paper: @inproceedings{sridhar2026audiocards, title={AUDIOCARDS: Structured Metadata Improves Audio Language Models for Sound Design}, author={Sridhar, Sripathi and Seetharaman, Prem and Nieto, Oriol and Cartwright, Mark and Salamon, Justin}, booktitle={ICASSP 2026-2026 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)}, pages={14522--14526}, year={2026}, organization={IEEE} }