---
language:
  - ja
license: cc-by-4.0
library_name: onnxruntime
pipeline_tag: automatic-speech-recognition
base_model: nvidia/parakeet-tdt_ctc-0.6b-ja
base_model_relation: quantized
tags:
  - automatic-speech-recognition
  - onnx
  - quantized
  - int8
  - cpu
  - japanese
  - tdt
  - ctc
  - fastconformer
---

# Parakeet TDT-CTC 0.6B Japanese ONNX Dynamic INT8


This is an **unofficial ONNX conversion and speed-oriented dynamic INT8
quantization** of
[`nvidia/parakeet-tdt_ctc-0.6b-ja`](https://huggingface.co/nvidia/parakeet-tdt_ctc-0.6b-ja).
It is not affiliated with or endorsed by NVIDIA.

The source model is pinned to revision
`44edb27eea9317daf89333e75eb830db4b1cc298`.

## What is included

This distribution uses one quantized FastConformer encoder shared by both
decoding paths. It deliberately omits the redundant monolithic CTC graph.

| File | Role | Precision |
|---|---|---|
| `encoder-model.int8.onnx` + `.data` | Shared FastConformer encoder | Dynamic per-channel QInt8 for constant-weight `MatMul` operations |
| `decoder_joint-model.onnx` | TDT predictor and joint network | FP32 |
| `ctc-head-model.onnx` + `_data` | CTC projection, transpose, and log-softmax | FP32 |
| `vocab.txt` | 3,072-token SentencePiece-derived vocabulary | Unchanged from the pinned export |

## Quantization and conversion

The source `.nemo` checkpoint was exported with NeMo. The standalone NeMo
encoder export was rejected because it was numerically inconsistent with the
PyTorch encoder. The shared encoder in this distribution was instead extracted
from the numerically faithful monolithic CTC export at the CTC decoder boundary.

The extracted encoder was quantized with:

- `onnxruntime.quantization.quantize_dynamic`
- `weight_type=QInt8`
- `per_channel=True`
- `op_types_to_quantize=["MatMul"]`

The TDT decoder/joint and the small CTC head remain FP32. See
[`MODIFICATIONS.md`](MODIFICATIONS.md) for the complete transformation record.

## Input and output

- Input audio: mono, 16 kHz PCM after frontend conversion.
- Language: Japanese.
- Output: punctuated transcription text through either TDT or CTC decoding.
- Architecture: hybrid FastConformer TDT-CTC, approximately 0.6B parameters.

## Usage

This is a split ONNX distribution intended for Parapper's Japanese Parakeet
runtime. It is **not** a drop-in replacement for the original NeMo checkpoint,
the Hugging Face ASR pipeline, or a generic single-file ONNX recognizer.

The runtime must:

1. reproduce the pinned NeMo 80-bin audio frontend;
2. run the shared encoder;
3. run either the TDT decoder/joint state machine or the CTC head;
4. decode IDs with `vocab.txt`.

The TDT path requires the NVIDIA-compatible token-and-duration decoding rules,
including a maximum duration of four encoder frames. Parapper also supports a
variable-width TDT DAG decoder, but that decoder is application code and is not
part of the model payload.

## Limitations

- Quantization can change recognized text relative to the source model.
- Evaluation does not cover every Japanese accent, domain, microphone, noise
  condition, or speaking style.
- Only CPUExecutionProvider with the tested ONNX Runtime contract has been
  verified for this distribution.
- The source model may reproduce biases or errors present in its training data.
- Transcriptions should not be treated as authoritative in safety-critical,
  legal, medical, or other high-impact settings.

Refer to the
[upstream model card](https://huggingface.co/nvidia/parakeet-tdt_ctc-0.6b-ja)
for the original architecture, training data, intended use, and limitations.

## License and attribution

The upstream checkpoint is licensed under
[Creative Commons Attribution 4.0 International](https://creativecommons.org/licenses/by/4.0/).
This quantized derivative is distributed under the same license. The conversion
is clearly identified as a modification. See [`ATTRIBUTION.md`](ATTRIBUTION.md)
and [`LICENSE`](LICENSE).