---
language:
- tr
license: apache-2.0
base_model: facebook/wav2vec2-xls-r-300m
tags:
- wav2vec2
- automatic-speech-recognition
- phoneme-recognition
- turkish
- ctc
datasets:
- mozilla-foundation/common_voice_24_0
metrics:
- per
---

# telaffuz-yz — XLS-R-300M Turkish Phoneme CTC

Fine-tuned from [facebook/wav2vec2-xls-r-300m](https://huggingface.co/facebook/wav2vec2-xls-r-300m) for Turkish phoneme-level CTC recognition. Developed as the acoustic backbone of a Turkish pronunciation assessment system (MSc thesis, Istanbul University–Cerrahpaşa, 2026).

## Model Description

| Property | Value |
|---|---|
| Base model | `facebook/wav2vec2-xls-r-300m` |
| Architecture | Wav2Vec2ForCTC |
| Parameters | ~315 M |
| Feature encoder | Frozen (7-layer CNN) |
| Transformer layers | 24 |
| CTC vocabulary | 54 tokens (49 Turkish phonemes + `<pad>`, `<unk>`, `<blank>`, `<spc>`) |
| Sample rate | 16 kHz |
| Task | Turkish phoneme recognition (CTC) |

## Phoneme Inventory

The 54-token vocabulary covers the full Turkish phoneme inventory used in the thesis pipeline:

- **8 short vowels:** a e i ɯ o œ u y
- **8 long vowels:** aː eː iː ɯː oː œː uː yː
- **1 vowel allophone:** æ
- **20 consonant phonemes:** p b t d k ɡ t͡ʃ d͡ʒ f v s z ʃ ʒ h m n l ɾ j
- **7 base consonant allophones:** c ɟ ɲ ŋ ɫ β β̞
- **5 pedagogical allophones:** pʰ tʰ kʰ cʰ ɾ̞̊
- **1 suprasegmental:** ˈ (primary stress)
- **Special tokens:** `<pad>` `<unk>` `<blank>` `<spc>`

This inventory is aligned with the rule-based G2P pipeline that produces reference transcriptions.

## Training Data

Fine-tuned on ~310 hours of Turkish read speech from two sources:

| Corpus | Style | Hours used |
|---|---|---:|
| Mozilla Common Voice 24.0 | Crowdsourced read speech | ~126 h |
| ISSAI Turkish Speech Corpus (TSC) | Studio read speech | ~163 h |
| **Total** | | **~310 h** |

Only segments with verified publisher transcripts are used (Whisper-transcribed segments excluded). Duration filter: 0.5 s ≤ clip ≤ 15 s.

**Split composition (speaker-aware 90/5/5):**

| Split | Segments | Hours |
|---|---:|---:|
| train | 239,555 | 275.5 h |
| dev | 14,793 | 16.0 h |
| test | 15,277 | 18.6 h |

Speaker assignment uses SHA1 hashing on speaker IDs so no speaker appears in more than one split.

## Training Procedure

```
scripts/train_wav2vec.py --model xls-r-300m --epochs 8 --batch-size 4 --grad-accum 8 \
    --lr 1e-4 --warmup-steps 500 --early-stopping-patience 5 \
    --early-stopping-threshold 0.001 --seed 42
```

| Hyperparameter | Value |
|---|---|
| Effective batch size | 32 (per-device 4 × grad-accum 8) |
| Learning rate | 1 × 10⁻⁴ |
| LR schedule | Linear warmup → linear decay to 0 |
| LR warmup steps | 500 |
| Weight decay | 0.0 |
| Epochs planned / run | 8 / 8 (full schedule) |
| Early stopping patience | 5 evaluations |
| Feature encoder | Frozen throughout |
| Precision | FP16 |
| Gradient checkpointing | Enabled (`use_reentrant=False`) |
| Seed | 42 |

**Checkpoint selection:** Best dev-set PER across all evaluations (every 400 steps). HuggingFace `Trainer` with `load_best_model_at_end=True`.

**Hardware:** NVIDIA GeForce RTX 3090 (24 GB VRAM), AMD Ryzen 9 5900X, 64 GB RAM, CUDA 12.4, PyTorch 2.6.0+cu124, Transformers 4.57.6.

**Training time:** ~20 GPU-hours (59,888 steps, single GPU).

## Evaluation Results

Evaluated on the held-out test set (15,277 utterances, 661,008 reference phones) with atomic IPA token decoding (multi-character tokens like `t͡ʃ`, `aː` treated as single units):

| Dataset | PER |
|---|---:|
| Common Voice (test) | 3.88% |
| ISSAI TSC (test) | 4.41% |
| **Combined test** | **4.05%** |

The companion MMS-1B model achieves 4.13% PER on the same test set.

## Usage

```python
from transformers import Wav2Vec2ForCTC, Wav2Vec2Processor
import torch, soundfile as sf

model_id = "onesvat/telaffuz-yz-wav2vec-xlsr300m"
processor = Wav2Vec2Processor.from_pretrained(model_id)
model = Wav2Vec2ForCTC.from_pretrained(model_id)
model.eval()

audio, sr = sf.read("speech.wav", dtype="float32")
# resample to 16 kHz if needed

inputs = processor(audio, sampling_rate=16000, return_tensors="pt", padding=True)
with torch.no_grad():
    logits = model(**inputs).logits

pred_ids = torch.argmax(logits, dim=-1)
transcription = processor.batch_decode(pred_ids)
print(transcription)  # IPA phoneme sequence
```

## Intended Use

This model is intended for:
- Turkish pronunciation assessment and coaching systems
- Turkish phoneme recognition research
- Downstream tasks requiring Turkish phone-level posteriors

It is **not** intended for general Turkish ASR (word-level transcription). The vocabulary is phonemic, not graphemic.

## Limitations

- Trained on read speech only; performance on spontaneous/conversational speech may be lower.
- Long vowels (aː, eː, …) are treated as distinct phoneme classes; recognition accuracy varies by phoneme (see per-phoneme error analysis in the thesis).
- Duration filter (0.5–15 s) means very short or very long utterances were excluded from training.

## Citation

If you use this model, please cite:

```bibtex
@mastersthesis{nesvat2026telaffuz,
  author  = {Onur Nesvat},
  title   = {Türkçe Telaffuz Değerlendirme Sistemi},
  school  = {İstanbul Üniversitesi-Cerrahpaşa},
  year    = {2026},
}
```

## License

Apache-2.0 (inherited from `facebook/wav2vec2-xls-r-300m`).