duclvQ/qwen3-asr-1.7b-malay-merged

Qwen3-ASR-1.7B Malay (merged)

Qwen/Qwen3-ASR-1.7B with the Malay / Malay-English code-switch LoRA duclvQ/qwen3-asr-1.7b-malay-lora merged into the weights (bf16, safetensors). Drop-in replacement for the base model in qwen-asr (transformers or vLLM backend). Training data, results and caveats are on the adapter card; code and serving stack: github.com/duclvq/finetune_QwenASR.

WER % (lower is better): in-domain 37.03 (base 52.19), Revolab benchmark 13.39 (base 15.01), real code-switch on an unseen speaker 7.83 (base 17.94).

Serve with vLLM

pip install "qwen-asr[vllm]==0.0.6"
qwen-asr-serve duclvQ/qwen3-asr-1.7b-malay-merged --served-model-name qwen3-asr-malay --port 8100 \
  --dtype bfloat16 --quantization fp8 --max-model-len 4096 --max-num-seqs 96 --limit-mm-per-prompt '{"audio":1}'
curl -s http://localhost:8100/v1/audio/transcriptions -F [email protected] -F model=qwen3-asr-malay \
     -F language=ms -F max_completion_tokens=400

A Docker Compose version is in the repository's deploy/. Measured on an RTX 5060 Ti 16 GB with FP8: single-stream RTF 0.043, 128 concurrent requests at p95 RTF 0.29 (~685x real time), served WER 8.85 on the real code-switch set (bf16 8.56). Always send max_completion_tokens (400 for <= 30 s audio): the transcription endpoint otherwise decodes up to the context length, and a clip that falls into a repetition loop emits hundreds of words.

Transformers

import torch
from qwen_asr import Qwen3ASRModel
w = Qwen3ASRModel.from_pretrained("duclvQ/qwen3-asr-1.7b-malay-merged", dtype=torch.bfloat16, device_map={"": 0},
                                  max_inference_batch_size=8, max_new_tokens=400)
print(w.transcribe(audio=["clip.wav"], language="Malay")[0].text)   # starts with "language Malay<asr_text>"

Weight-only quantization on this checkpoint (HQQ, LLM linears): int8 is free (8.62 -> 8.65 WER), int4 costs ~1.4 WER, 3-bit and below collapse without calibration.