README.md

metadata

language:
  - ja
license: cc-by-4.0
library_name: onnxruntime
pipeline_tag: automatic-speech-recognition
base_model: nvidia/parakeet-tdt_ctc-0.6b-ja
base_model_relation: quantized
tags:
  - automatic-speech-recognition
  - onnx
  - quantized
  - int8
  - cpu
  - japanese
  - tdt
  - ctc
  - fastconformer

Parakeet TDT-CTC 0.6B Japanese ONNX Dynamic INT8

This is an unofficial ONNX conversion and speed-oriented dynamic INT8 quantization of nvidia/parakeet-tdt_ctc-0.6b-ja. It is not affiliated with or endorsed by NVIDIA.

The source model is pinned to revision 44edb27eea9317daf89333e75eb830db4b1cc298.

What is included

This distribution uses one quantized FastConformer encoder shared by both decoding paths. It deliberately omits the redundant monolithic CTC graph.

File	Role	Precision
encoder-model.int8.onnx + .data	Shared FastConformer encoder	Dynamic per-channel QInt8 for constant-weight MatMul operations
decoder_joint-model.onnx	TDT predictor and joint network	FP32
ctc-head-model.onnx + _data	CTC projection, transpose, and log-softmax	FP32
vocab.txt	3,072-token SentencePiece-derived vocabulary	Unchanged from the pinned export

Quantization and conversion

The source .nemo checkpoint was exported with NeMo. The standalone NeMo encoder export was rejected because it was numerically inconsistent with the PyTorch encoder. The shared encoder in this distribution was instead extracted from the numerically faithful monolithic CTC export at the CTC decoder boundary.

The extracted encoder was quantized with:

• onnxruntime.quantization.quantize_dynamic
• weight_type=QInt8
• per_channel=True
• op_types_to_quantize=["MatMul"]

The TDT decoder/joint and the small CTC head remain FP32. See MODIFICATIONS.md for the complete transformation record.

Input and output

• Input audio: mono, 16 kHz PCM after frontend conversion.
• Language: Japanese.
• Output: punctuated transcription text through either TDT or CTC decoding.
• Architecture: hybrid FastConformer TDT-CTC, approximately 0.6B parameters.

Usage

This is a split ONNX distribution intended for Parapper's Japanese Parakeet runtime. It is not a drop-in replacement for the original NeMo checkpoint, the Hugging Face ASR pipeline, or a generic single-file ONNX recognizer.

The runtime must:

1. reproduce the pinned NeMo 80-bin audio frontend;
2. run the shared encoder;
3. run either the TDT decoder/joint state machine or the CTC head;
4. decode IDs with vocab.txt.

The TDT path requires the NVIDIA-compatible token-and-duration decoding rules, including a maximum duration of four encoder frames. Parapper also supports a variable-width TDT DAG decoder, but that decoder is application code and is not part of the model payload.

Limitations

• Quantization can change recognized text relative to the source model.
• Evaluation does not cover every Japanese accent, domain, microphone, noise condition, or speaking style.
• Only CPUExecutionProvider with the tested ONNX Runtime contract has been verified for this distribution.
• The source model may reproduce biases or errors present in its training data.
• Transcriptions should not be treated as authoritative in safety-critical, legal, medical, or other high-impact settings.

Refer to the upstream model card for the original architecture, training data, intended use, and limitations.

License and attribution

The upstream checkpoint is licensed under Creative Commons Attribution 4.0 International. This quantized derivative is distributed under the same license. The conversion is clearly identified as a modification. See ATTRIBUTION.md and LICENSE.