---
license: cc-by-4.0
language:
- en
base_model: nvidia/parakeet-tdt-0.6b-v2
base_model_relation: quantized
pipeline_tag: automatic-speech-recognition
library_name: onnx
tags:
- onnx
- webgpu
- fastconformer
- tdt
- parakeet
---

# Parakeet TDT 0.6B v2 — int4 encoder for WebGPU

A 4-bit (`MatMulNBits`, block size 32, uint8 zero points) build of the encoder of
[nvidia/parakeet-tdt-0.6b-v2](https://huggingface.co/nvidia/parakeet-tdt-0.6b-v2) (English, CC BY 4.0, by NVIDIA),
quantized from the fp32 ONNX export in
[istupakov/parakeet-tdt-0.6b-v2-onnx](https://huggingface.co/istupakov/parakeet-tdt-0.6b-v2-onnx)
(revision `0bbb45a3`) with onnxruntime 1.31's `MatMulNBitsQuantizer` (`quantize.py` here).

uint8 zero points are what onnxruntime-web's WebGPU `MatMulNBits` kernel runs, so the whole encoder stays on the GPU.
It is a drop-in replacement for `encoder-model.int8.onnx` in that repo: use it with its `vocab.txt` and
`decoder_joint-model.int8.onnx` (onnx-asr layout).

`nemo128.onnx` here is that repo's mel front end with its STFT in float32 instead of float64 (`stft_float32.py`):
onnxruntime-web's WebGPU build has no float64 `Cast`. The log-mel features differ by at most 2.2e-4, and on the
test clips the tokens are the same as with the original.


## Accuracy

LibriSpeech (73 utterances of `hf-internal-testing/librispeech_asr_dummy`, 8 min), onnx-asr greedy decoding,
lower-cased without punctuation:

| Encoder | WER | Size |
|---|---|---|
| fp32 | 2.87 % | 2.48 GB |
| int8 | 2.70 % | 652 MB |
| int4 (this) | 3.04 % | 688 MB |

The 1×1 convolutions are left in fp32, hence the size.