Where transcription errors remain
Semantic WER by prompt category · identical held-out text · Whisper-large-v3
0.0%
2.8%
5.6%
8.4%
11.2%
14.0%
Numbers
Micro 12.50%
Nano 9.38%
Rare Words
Micro 5.26%
Nano 8.77%
Short
Micro 2.86%
Nano 7.62%
Names Places
Micro 7.02%
Nano 5.26%
Homographs
Micro 2.86%
Nano 4.29%
Phonetic Balance
Micro 2.59%
Nano 3.16%
Technical
Micro 3.12%
Nano 3.12%
Narrative
Micro 0.28%
Nano 1.14%
Conversational
Micro 0.00%
Nano 0.97%
Long Range
Micro 0.00%
Nano 0.83%
Long
Micro 0.69%
Nano 0.69%
Question
Micro 0.00%
Nano 0.61%
Descriptive
Micro 0.44%
Nano 0.44%
Emotional
Micro 0.00%
Nano 0.30%
Punctuation
Micro 0.00%
Nano 0.00%
Category results diagnose weaknesses; they are not pooled with UTMOS22 or human preference.