qwen3-asr-1.7b-ar-run6b-soup70-sip-v5-grpo2
sip-v5-grpo1 refined with a second round of reinforcement learning, MWER-style: for each training clip the beam
search's 8 best transcripts are rewarded with 1 - WER against the label (Test Arabic ASR Benchmark normaliser), -1 for
a repetition loop or text far longer than the label, and the model is pushed towards the better ones (advantages
standardised within the clip's group), with a KL penalty to sip-v5-grpo1. LoRA r 16 on the language model's
projections (merged; adapter in adapter/).
Prompts (~7,100 clips, 10 h): clips where sip-v5-grpo1 makes mistakes, plus 20% it gets right and 300 clips without speech, from in-domain Arabic call-centre recordings (private), EdAcc and Earnings-22 (accented English conversation, CC BY-SA 4.0).
Licence note: part of sip-v5's training text is CC BY-NC 4.0 and EdAcc / Earnings-22 are CC BY-SA 4.0; check that your use is compatible.
Evaluation (bf16, WER %; lower is better)
| set | sip-v5 | sip-v5-grpo1 | sip-v5-grpo2 |
|---|---|---|---|
| Audioluna call samples (20 recordings) | 13.33 | 13.50 | 13.43 |
| Test Arabic ASR Benchmark (505 segments) | 9.99 | 9.59 | 10.39 |
| Arabic segments | 15.35 | 14.76 | 15.69 |
| English segments | 6.39 | 6.08 | 6.77 |
| FLEURS ar_eg test (428) | 5.25 | 5.06 | 5.61 |
| In-domain call-centre test split | 10.50 | 10.78 | 10.57 |
| Clips without speech that get text (of 400) | 107 | 104 | 101 |
Not recommended: this second (N-best) round is worse than sip-v5-grpo1 on the Test Arabic benchmark (+0.80) and FLEURS (+0.55) and no better on the call samples; use sip-v5-grpo1 instead.
- Downloads last month
- 2
Model tree for Arabic-STT/qwen3-asr-1.7b-ar-run6b-soup70-sip-v5-grpo2
Base model
Qwen/Qwen3-ASR-1.7B