kznkzn/Qwen3-ASR-1.7B-JA-speechcpp-GGUF

Qwen3-ASR-1.7B-JA for speech.cpp

A Q8_0 GGUF conversion for speech.cpp of Neosophie's Japanese fine-tune of Qwen3-ASR-1.7B. This repository converts and quantizes that checkpoint; it introduces no additional training.

The model uses speech.cpp layout 1, supported from speech.cpp 0.7.0. Conversion and inference were verified with 0.8.10. This is a speech.cpp-specific GGUF, and is not compatible with llama.cpp's GGUF model layout. The audio frontend, encoder, projector, decoder, and tokenizer are contained in one file; no separate mmproj file is needed.

Download and run

Download Qwen3-ASR-1.7B-JA-Q8_0.gguf from this repository's Files tab and pass its local path to speech.cpp:

speech asr ./Qwen3-ASR-1.7B-JA-Q8_0.gguf --language ja ./audio.wav

Use a speech.cpp build with Qwen3-ASR support. --language ja selects Japanese; the language tags on this model card describe the Japanese/English use case of the source fine-tune, and are not an English accuracy claim for this conversion.

To verify the downloaded model on macOS:

shasum -a 256 Qwen3-ASR-1.7B-JA-Q8_0.gguf

Expected SHA-256:

cfe17bfad6708fd4ea243ade250634b7191becb491579ae8f19f423fb2516676

Artifact and provenance

Item	Value
File	Qwen3-ASR-1.7B-JA-Q8_0.gguf
Size	2,176,109,280 bytes (about 2.18 GB / 2.03 GiB)
Quantization	Q8_0, using speech.cpp's model-specific quantization rules
Format	GGUF, speech.cpp layout 1
Source checkpoint	neosophie/Qwen3-ASR-1.7B-JA
Source revision	987bda160f2dabfa6757550bcff7cdda2ba0648c
Source model.safetensors SHA-256	4e378e29a060f4a8464a59976a6b84661dc2f88f23a3f43634651de3fc84ccb4
Converter/runtime	speech.cpp 0.8.10
speech.cpp revision	a72f1bbedd50e293d5b3f4c2bf0fb44ec003227d

The base model is Qwen/Qwen3-ASR-1.7B. Japanese fine-tuning was performed by Neosophie; consult the source model card for its training information and intended uses. Machine-readable conversion metadata is in model-manifest.json, and file checksums are in SHA256SUMS.

Validation and limitations

The following checks passed with speech.cpp 0.8.10 on Apple Silicon:

• The checkpoint's 707 tensor names, shapes, and dtypes matched the expected base-model schema, with the omitted duplicate tied output matrix handled by the adapter. The source checkpoint passed its pinned file integrity checks.
• Conversion to F32 GGUF and quantization to Q8_0 completed successfully.
• The portable adapter included here was rerun end-to-end against the pinned source with the same locked environment and speech.cpp 0.8.10. Its Q8_0 output matched the published model's size and SHA-256 exactly.
• GPU inference on one synthetic Japanese utterance, 6.264 seconds long, returned identical text for the F32 and Q8_0 GGUFs.
• An integrated NeMo runtime loaded the Q8_0 model through libspeech while two streaming ASR sessions ran concurrently. Qwen's output matched the speech.cpp CLI reference and remained the same during concurrent inference; the server survived without spawning a separate speech process.

These are conversion and runtime smoke tests. No accuracy benchmark was run. They do not establish parity with the original BF16 model, improved accuracy over the base Qwen model, English accuracy, or performance on real meetings. The synthetic test audio is not included in this repository.

Reproduce the conversion

The converter directory includes the pinned upstream converter, dependency lockfile, input integrity manifest, tensor schema, and a small Japanese-checkpoint adapter. It provides commands to download the pinned source, convert to F32, and quantize with speech.cpp 0.8.10. A separately exported safetensors header is optional; the adapter can read it directly from the model.

The adapters account for the fine-tune's repository identity, its omitted duplicate tied output weights, and an omitted do_sample field whose default is greedy decoding. They also label the output as the Japanese fine-tune and load the tokenizer from the local checkpoint. They do not change the numerical conversion operations.

License and credits

The model is distributed under Apache-2.0, as declared by the source model. Credit goes to the Qwen team / Alibaba Cloud for Qwen3-ASR, Neosophie for the Japanese fine-tune, and the speech.cpp contributors for the inference runtime and converter. The included converter source is distributed under the upstream MIT license. See NOTICE for modification notices and provenance.