Gabriel/parakeet-tdt-0.6b-v2-onnx-webgpu

Parakeet TDT 0.6B v2 — int4 encoder for WebGPU

A 4-bit (MatMulNBits, block size 32, uint8 zero points) build of the encoder of nvidia/parakeet-tdt-0.6b-v2 (English, CC BY 4.0, by NVIDIA), quantized from the fp32 ONNX export in istupakov/parakeet-tdt-0.6b-v2-onnx (revision 0bbb45a3) with onnxruntime 1.31's MatMulNBitsQuantizer (quantize.py here).

uint8 zero points are what onnxruntime-web's WebGPU MatMulNBits kernel runs, so the whole encoder stays on the GPU. It is a drop-in replacement for encoder-model.int8.onnx in that repo: use it with its vocab.txt and decoder_joint-model.int8.onnx (onnx-asr layout).

nemo128.onnx here is that repo's mel front end with its STFT in float32 instead of float64 (stft_float32.py): onnxruntime-web's WebGPU build has no float64 Cast. The log-mel features differ by at most 2.2e-4, and on the test clips the tokens are the same as with the original.

Accuracy

LibriSpeech (73 utterances of hf-internal-testing/librispeech_asr_dummy, 8 min), onnx-asr greedy decoding, lower-cased without punctuation:

Encoder	WER	Size
fp32	2.87 %	2.48 GB
int8	2.70 %	652 MB
int4 (this)	3.04 %	688 MB

The 1×1 convolutions are left in fp32, hence the size.