Bonenk/Nemotron-3-Diarization-GGUF

Nemotron-3-Diarization — GGUF (F32 / F16 / Q8_0)

GGUF / ggml conversions of nvidia/Nemotron-3-Diarization for the two native C/C++ runtimes that speak its layout:

• CrispStrobe/CrispASR — --diarize-method sortformer
• NVIDIA/NeMo-Speech.cpp — nemo-speech diarize

Nemotron-3-Diarization is a ~100 M-parameter streaming Sortformer v3 diarizer: one end-to-end network maps 16 kHz audio straight to a per-speaker activity probability every 10 ms, for up to 8 speakers numbered in order of first appearance, with overlapping speech handled natively. No embedding model, no clustering, no speaker-count estimate. Weights are NVIDIA OpenMDW-1.1 (commercial use allowed).

🥇 #1 on the Voice Arena Diarization Bench — 14.72 % DER

On the independent Voice Arena Diarization Bench (Version 1), Nemotron-3-Diarization is the best speaker-diarization model measured, at 14.72 % DER at a 0 ms collar — and it wins at every collar:

Collar	0 ms	100 ms	250 ms
DER	14.72 %	8.85 %	4.29 %
JER	25.53 %	18.84 %	12.10 %

At the strict 0 ms collar the error splits into 3.63 % missed speech, 8.88 % false alarm and 2.21 % speaker confusion (3.63 + 8.88 + 2.21 = 14.72). Broken out by setting: 15.24 % DER on in-person recordings and 11.65 % DER on online (VoIP) recordings.

Full Version 1 leaderboard, DER % at the 0 ms collar, pooled over all 139 sessions (lower is better):

#	System	Provider	DER %
1	Nemotron 3 Diarization (offline)	NVIDIA	14.72
2	DiariZen WavLM Large s80 MD v2	BUT FIT	19.34
3	Precision-3	pyannoteAI	20.56
4	Precision-2	pyannoteAI	23.41
5	Streaming Sortformer Diarizer 4spk v2	NVIDIA	24.61
6	Community-1	pyannote	30.61
7–12	Scribe v2, Muse Voice Transcribe, Universal-3.5 Pro, Soniox v5 Async, Speechmatics Enhanced, Nova-3	ElevenLabs, Meta, AssemblyAI, Soniox, Speechmatics, Deepgram	40.71 – 67.13

That is a 4.6-point absolute (≈ 24 % relative) lead over the runner-up (DiariZen) and 9.9 points over the next-best NVIDIA model (Streaming Sortformer 4spk v2). The commercial ASR-with-diarization APIs sit 25–52 points behind.

What the bench measures

Voice Arena Version 1 scores 12 open-source and commercial systems against one strict, human-verified RTTM reference:

• 139 far-field English conversations, 21.97 hours, 280+ unique speakers (115 in-person, 24 online), English across Indian, US, Vietnamese and European accents.
• DER uses DIHARD scoring: (missed speech + false alarm + speaker confusion) ÷ total reference speech time, pooled by scored minutes over every session.
• Collars of 0, 100 and 250 ms; overlapping speech is scored; the system performs its own speech-activity detection; the speaker count is never given to the system.
• Version 1 ranks the batch/offline track: streaming-capable systems are run in batch mode and scored on their final timeline. JER and speaker-count accuracy are published alongside DER.

See the bench methodology for the corpus design, reference construction and scoring convention.

Because collars, overlap handling and references differ, these numbers are not directly comparable to NVIDIA's own evaluation (DIHARD III, CALLHOME, AMI, AliMeeting, NOTSOFAR1 — see the upstream model card), where the offline configuration reports 12.73 % DER on DIHARD III Eval and 9.10 % on CALLHOME-Part2 (0.25 s collar).

Files

File	Size	Notes
Nemotron-3-Diarization.f32.gguf	397 MB	All F32. Exact reference, matches the Transformers checkpoint stage-for-stage
Nemotron-3-Diarization.f16.gguf	202 MB	F16 weights; norms, biases, the mel filterbank and a few numerically sensitive projections stay F32 (see below)
Nemotron-3-Diarization.q8_0.gguf	109 MB	Q8_0, re-quantised from the F32 reference. The 31 layers' attention/FFN projections plus encoder_proj and encoder.pre_encode.proj are Q8_0; norms, biases, the output head, the subpixel conv and the mel filterbank stay F32

All share the identical sortformer GGUF layout (tensor names, shapes and metadata), so either runtime loads any of them — pick the size/quality point you want. NVIDIA's own Nemotron-3-Diarization.q8_0.gguf (107 MB) stays available in the upstream repo.

Which file should I use?

• Nemotron-3-Diarization.f16.gguf — recommended default. Half the size of F32 with diarization decisions that match the F32 reference on the CrispASR parity clips.
• Nemotron-3-Diarization.f32.gguf — the exact reference. Use it when you are diffing against Transformers/NeMo or want the tightest numerical fidelity.
• Nemotron-3-Diarization.q8_0.gguf — smallest. Reproduces the F32/F16 diarization on the parity clip, but like any 8-bit quant it can flip a borderline speaker decision.

Quick start

CrispASR

git clone https://github.com/CrispStrobe/CrispASR && cd CrispASR
cmake -B build -DCMAKE_BUILD_TYPE=Release && cmake --build build -j

huggingface-cli download Bonenk/Nemotron-3-Diarization-GGUF \
    Nemotron-3-Diarization.f16.gguf --local-dir .

# Transcribe + diarize (speaker-labelled segments)
./build/bin/crispasr -m auto --backend cohere -f meeting.wav \
    --diarize --diarize-method sortformer \
    --diarize-model Nemotron-3-Diarization.f16.gguf \
    --merge-speaker-turns -osrt -of meeting

# Live-style local streaming schedule (0.32 s input buffer)
./build/bin/crispasr -m auto --backend cohere -f meeting.wav \
    --diarize --diarize-method sortformer \
    --diarize-model Nemotron-3-Diarization.f16.gguf \
    --sortformer-mode ultra_low_latency -ojf

--diarize-method sortformer (alias nemotron3-diar) needs only the diarizer GGUF; --diarize-model auto (the default) downloads NVIDIA's q8_0 instead. --sortformer-mode selects offline (default), low_latency, very_low_latency or ultra_low_latency.

NVIDIA NeMo-Speech.cpp

# Standalone diarization — no ASR model required
nemo-speech diarize meeting.wav \
    --diar-model ./Nemotron-3-Diarization.f16.gguf \
    --format rttm --output meeting.rttm

# Word-level speaker tags in a transcription
nemo-speech transcribe meeting.wav --diarize \
    --diar-model ./Nemotron-3-Diarization.f16.gguf --json

Or point a config at it:

# config/diar.yaml
diar:
  model_path: ./Nemotron-3-Diarization.f16.gguf
  preset: v3-offline

nemo-speech diarize meeting.wav --config config/diar.yaml

These GGUFs are written in exactly the layout NeMo-Speech.cpp's conversion/diarization.py emits, including the v3 resolution/silence contract (sortformer.high_resolution, sortformer.output_subsampling_factor, sortformer.learnable_silence) that its loader validates before accepting a v3 checkpoint.

GGUF layout

general.architecture = "sortformer", sortformer.version = "v3", with the tensor set NVIDIA's NeMo-Speech.cpp reads:

encoder.pre_encode.proj.weight       # 8x feature stacking, 1024 -> 512
encoder.embed_norm.{weight,bias}
encoder.layers.{0..30}/
    norm1.*  attn.w_qkv.weight (q;k;v fused)  attn.out_proj.*  norm2.*
    ffn.net.0.*  ffn.net.3.*
encoder.final_norm.{weight,bias}
encoder_proj.{weight,bias}           # 512 -> 192
subpixel_upsample.{weight,bias}      # Conv1d 192 -> 1536, k=3 -> 8 x 10 ms frames
head.first_hidden_to_hidden.*
head.single_hidden_to_spks.*         # -> 8 speaker logits
learnable_sil_emb
preprocessor.fb                      # 128 slaney mel filters

nemotron3diar.offline.* and nemotron3diar.streaming.{low_latency, very_low_latency,ultra_low_latency}.* carry Transformers' offline chunk schedule and the streaming presets; sortformer.streaming.* mirrors the checkpoint's own geometry and is provenance only.

Why some tensors stay F32 in the F16 build

• encoder.pre_encode.proj — in streaming it projects only a chunk plus look-ahead (13 rows per chunk). CUDA's small-batch F16 matmul rounds those activations to F16 and flips a few borderline decisions, so it is kept F32.
• subpixel_upsample, head.*, the mel filterbank and every 1-D norm/bias — small, numerically sensitive, and already cheap at F32.

Quantising the large 2-D weights is what saves the space; everything that is cheap to keep exact is kept exact. Net effect: 125 of 356 tensors are F16 in the F16 build (the 31 layers' QKV / output-projection / feed-forward weights plus encoder_proj).

Validation

• Layout parity — every tensor name and shape is identical to NVIDIA's shipped Nemotron-3-Diarization.q8_0.gguf; the only metadata absent is the four unused head.activity_head.* tensors NVIDIA's converter writes but no runtime reads.
• NeMo-Speech.cpp contract — the v3 loader checks (high_resolution == true, output_subsampling_factor == 1, upsample_factor == encoder.subsampling_factor, learnable_silence == true) pass, and all 356 tensors the v3 graph requires are present.
• CrispASR parity (crispasr-diff nemotron3-diar, CPU and CUDA): F32 and F16 give cos 1.000000 on mel, embeddings, logits and probabilities, and 100 % of the p > 0.5 decisions and the segment list match the Transformers reference on the parity clips; NVIDIA's q8_0 agrees on 99.91–99.99 %.
• End-to-end — F32, F16 and this repo's Q8_0 produce byte-identical diarized transcripts through CrispASR on samples/multispeaker.wav (7 turns, 2 speakers); NVIDIA's upstream q8_0 is identical on the same clip.

License

OpenMDW-1.1, inherited from nvidia/Nemotron-3-Diarization. Commercial and non-commercial use permitted. This repo only re-containers NVIDIA's weights; it adds no new weights and makes no model-quality claims beyond the measurements cited above.

References

• Upstream model: nvidia/Nemotron-3-Diarization
• NVIDIA blog: Nemotron Diarization
• C++ runtime (GPL-3.0 project): CrispStrobe/CrispASR
• C++ runtime (Apache-2.0): NVIDIA/NeMo-Speech.cpp
• Independent benchmark: Voice Arena Diarization Bench