lighthouse-emnlp2024/Clotho-Moment_CLAP_features

---
license: cc-by-4.0
language:
- en
tags:
- multimodal
- audio-retrieval
- moment-retrieval
size_categories:
- 10K<n<100K
task_categories:
- audio-text-to-text
---

# Clotho-Moment CLAP features
This repository contains audio and text features of [Clotho-Moment dataset](https://arxiv.org/abs/2409.15672) extracted by CLAP.
- Using these features, we can reproduce the audio moments retrieval using Clotho-Moment, which is used in [lighthouse](https://github.com/line/lighthouse).
- Please also check [demo page](https://h-munakata.github.io/Language-based-Audio-Moment-Retrieval/).

## How to Download?
Run the following script:
```python
from huggingface_hub import snapshot_download

repo_id = "lighthouse-emnlp2024/Clotho-Moment_CLAP_features"
local_dir = "./"

downloaded_path = snapshot_download(
repo_id=repo_id,
repo_type="dataset",
local_dir=local_dir,
allow_patterns="*.tar.gz",
)
```

## How to Use on Lighthouse
The `.tar.gz` files should be decompressed by following shell commands:
```bash
mkdir -p {LIGHTHOUSE_PATH}/features/clotho-moment/clap
mkdir -p {LIGHTHOUSE_PATH}/features/clotho-moment/clap_text
tar -zxvf clap.tar.gz -C {LIGHTHOUSE_PATH}/features/clotho-moment/clap
tar -zxvf clap_text.tar.gz -C {LIGHTHOUSE_PATH}/features/clotho-moment/clap_text
```

## Citation
```bibtex
@inproceedings{munakata2025language,
title={Language-based Audio Moment Retrieval},
author={Munakata, Hokuto and Nishimura, Taichi and Nakada, Shota and Komatsu, Tatsuya},
booktitle={ICASSP 2025-2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)},
pages={1--5},
year={2025},
organization={IEEE}
}
```