Trelis/eval-Voxtral-Mini-3B-2507-multimed-hard-20260408-1931

--- tags: - whisper - evaluation - speech - speech-to-text --- # Evaluation Results: Voxtral-Mini-3B-2507 Evaluation results from Whisper model evaluation. ## Summary | Model | WER | CER | |-------|-----|-----| | [mistralai/Voxtral-Mini-3B-2507](https://huggingface.co/mistralai/Voxtral-Mini-3B-2507) | 10.89% | 7.46% | ## Source Data - **Evaluation Dataset:** [Trelis/multimed-hard](https://huggingface.co/datasets/Trelis/multimed-hard) - **Model Evaluated:** [mistralai/Voxtral-Mini-3B-2507](https://huggingface.co/mistralai/Voxtral-Mini-3B-2507) ## Columns | Column | Description | |--------|-------------| | `audio` | Audio sample (if available from source dataset) | | `reference` | Ground truth transcription | | `prediction` | Model prediction | | `wer` | Word Error Rate for this sample | | `cer` | Character Error Rate for this sample | | `entities` | Entity annotations from source dataset | | `entity_cer` | Per-sample entity CER (-1.0 if no entities) | ## Entity CER **Overall Entity CER:** 27.34% | Category | CER | |----------|-----| | anatomy | 25.53% | | biomarker | 25.82% | | condition | 28.18% | | drug | 0.00% | | organisation | 35.64% | | procedure | 27.81% | --- *Generated by [Trelis Studio](https://studio.trelis.com)*