cdli-whisper-ml-eng-lug-full-a40-5e-5

This is a multilingual model and is a fine-tuned version of Sunbird/asr-whisper-large-v3-salt on the Ugandan CDLI Atypical speech datasets. It achieves the following results on the evaluation set:

Loss: 1.2283
Wer: 0.4137
Cer: 0.2271

On the test set with repetition penalty of 1.3 and no_repeat_ngram_size of 2 it obtains:

test_cer = 0.1268
test_loss = 0.8137
test_runtime = 0:22:24.23
test_samples_per_second = 1.518
test_steps_per_second = 0.379
test_wer = 0.2851

English

Overall WER (normalized): 0.224
Overall CER (normalized): 0.135
Avg WER (normalized): 0.214
Avg CER (normalized): 0.133

Luganda

Overall WER (normalized): 0.414
Overall CER (normalized): 0.146
Avg WER (normalized): 0.354
Avg CER (normalized): 0.12

Model description

The training was resumed from epoch 7.2255 and the Wer reported after that is a bit dirty (CER instead of WER)

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 5e-05
train_batch_size: 4
eval_batch_size: 4
seed: 42
gradient_accumulation_steps: 8
total_train_batch_size: 32
optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
lr_scheduler_type: linear
lr_scheduler_warmup_steps: 150
training_steps: 4000

Training results

Training Loss	Epoch	Step	Cer	Validation Loss	Wer
0.6253	0.7228	250	0.2722	0.8156	0.4660
0.4188	1.4452	500	0.2362	0.8119	0.4247
0.2709	2.1677	750	0.2352	0.8229	0.4206
0.2571	2.8905	1000	0.2261	0.8141	0.4153
0.1581	3.6129	1250	0.2292	0.9097	0.4167
0.083	4.3354	1500	0.2271	0.9749	0.4177
0.0593	5.0578	1750	0.2266	1.0613	0.4107
0.0518	5.7806	2000	0.2235	1.0547	0.4108
0.0382	6.5031	2250	0.2249	1.1098	0.4095
0.0356	7.2255	2500	0.2238	1.1149	0.4087
0.0408	7.9483	2750	1.1168	0.4139	0.2261
0.0368	8.6737	3000	1.1499	0.4172	0.2279
0.0271	9.3961	3250	1.2052	0.4132	0.2271
0.0237	10.1185	3500	1.2107	0.4114	0.2263
0.0212	10.8413	3750	1.2275	0.4111	0.2250
0.0221	11.5638	4000	1.2283	0.4137	0.2271