# Chatterbox Multilingual oMLX Q6 versus Q8 and FP32 Fresh three-way measurements on 9 October 2026, MacBook Air M5 (Mac17,4), 24 GiB unified memory, macOS build 25G76. Twelve measured generations per variant, 36 total, using the same private Valentina reference, six prompts (3 English, 3 Spanish), seeds 42 and 123, runtime and decoding parameters. Voice recordings are not distributed. ## Findings Q6's main checkpoint is **18.2% smaller than Q8** and **73.5% smaller than FP32**. In this short run it used 9.8% less process peak RSS and 7.1% less peak MLX memory than Q8. Mean generation time was 4.0% shorter than Q8, a modest difference that requires replication before claiming a general speed improvement. | Measurement | FP32 | Q8 affine G64 | Q6 affine G64 | |---|---:|---:|---:| | Main checkpoint, decimal MB | 2711.114 | 877.794 | 718.359 | | Model load, s | 0.371 | 0.686 | 0.745 | | Reference conditioning, s | 0.141 | 0.129 | 0.240 | | Separate warm-up generation, s | 2.425 | 1.770 | 1.894 | | Mean warm generation per clip, s | 4.925 | 2.687 | 2.580 | | Mean generated audio length, s | 3.553 | 3.633 | 3.660 | | Aggregate real-time factor | 1.386 | 0.740 | 0.705 | | Process peak RSS over loading/generation, GiB | 3.226 | 1.521 | 1.372 | | Maximum per-generation MLX peak, GiB | 4.197 | 2.029 | 1.884 | | Mean CPU, percent of one core | 22.793 | 28.582 | 29.450 | | Whole-chip CPU+GPU+ANE mean power during generation, W | 7.354 | 8.505 | 8.741 | | Sampled chip energy during generation, J | 428.573 | 266.497 | 257.704 | | Strict pooled Parakeet WER | 7.33% (11/150 edits) | 6.67% (10/150 edits) | 8.67% (13/150 edits) | RTF = generation time / generated audio length; below 1 is faster than real time. RSS and MLX allocations overlap and must not be added. Main checkpoint sizes exclude the separate 494.87 MB S3Tokenizer disk cache. Both quantized loaders additionally quantize that dependency in memory at their respective bit widths. Unsupported tensors remain FP32. These values come from a **new run**, rather than combining Q6 with older Q8/FP32 timings. This run used **battery power at 100%, low-power mode off**, an active desktop and fixed order FP32, Q8, Q6, each in a separate sequential process. Terminal, WindowServer, Chrome, NoMachine and video-decoding activity were present. FP32's first half had notably higher generation latency than its second half; workload variation and order can bias apparent speed and power differences. This is not an isolated TTS or statistically controlled performance result. The earlier Q8 report ran on AC and must not be treated as the same operating condition. ## Exact quantization and provenance Q6 was generated directly from the cached **original FP32** source, not from Q8. Source: [mlx-community/chatterbox-multilingual-v3](https://huggingface.co/mlx-community/chatterbox-multilingual-v3/tree/03565773edd72e949572557597af8063bb49a18a), revision 03565773edd72e949572557597af8063bb49a18a, main SHA-256 e702f2c441e040bd360e59a86c85d462539759d665e41cfd1938adadd74187a3. Q8 comparison artifact: [groxaxo/chatterbox-multilingual-MLX-Q8-G64](https://huggingface.co/groxaxo/chatterbox-multilingual-MLX-Q8-G64), SHA-256 aa938735e36edfa807531d274baa608ec168f8f06ad6c835a3c4071effcf172e. - oMLX revision 70f44cfd652d2013dc0f9776dca7518e32101b00, function _quantize_chunked, native MLX **affine 6-bit, group 64**, no importance weights or calibration. This is the native packing primitive, not the stock oQ mixed-precision LLM calibration workflow. - **651 checkpoint matrices**: 222 T3, 428 S3Gen, 1 voice encoder. - **37 external S3Tokenizer matrices** additionally become Q6 in memory, **688 Q6 matrices in execution**. Q8 uses the same matrix set at 8 bits; FP32 uses zero quantized matrices. - **1,836 untouched tensors / 159,837,884 bytes** checked byte-for-byte identical to the FP32 source. This is mixed FP32/Q6 storage, not an all-integer model. - Q6 checkpoint SHA-256: **392f8758a072a70a44ec5c70351a9502877880a196377c74d90cf006a8f06c8c**; bytes: 718359417. - External tokenizer revision e0c9886f0e1c35ae85b1f27277416fb19fc72bec, SHA-256 928726bc1f206a613d36b8f49e297eae9c5593a21bf9b92ddfe2c23f85eb92cc; its cached file matched the Hub hash and was not overwritten. - The included standalone [quantize_q6.py](../../scripts/quantize_q6.py) was executed on Apple Silicon / Metal and reproduced the checkpoint **byte-identically**, including the full SHA-256. See [rebuild-validation.json](rebuild-validation.json). Platform/backend/library differences can change packed bytes; the published checksum identifies the evaluated artifact. Detailed checks: [weight-validation.json](weight-validation.json). No LoRA merge, fine-tuning or private voice is baked into the model. ## Detected audio and transcription differences All 36 outputs were finite, non-empty 24 kHz PCM16 audio. Q6's waveform hash differed from both references in every one of the 12 pairs. Relative audio-duration changes were -7.8% to 11.4% versus Q8, and -8.0% to 14.5% versus FP32. Different autoregressive/acoustic output under the same seed does not by itself establish perceptual degradation. Per-clip RMS, amplitude and clipping fractions are in the data. | Language | FP32 WER | Q8 WER | Q6 WER | |---|---:|---:|---:| | en | 13.51% | 13.51% | 14.86% | | es | 1.32% | 0.00% | 2.63% | Q6 had **13 word edits in 150 reference tokens**, versus 10 for Q8 and 11 for FP32. The Q6/Q8 gap is only three edits. Q6's seed-42 weather transcript omitted “today” and contracted “I am” to “I'm”; the contraction alone counts as two strict edits. In Spanish the seed-42 microphone clip was recognized as “Puedes” instead of “Podés”, and seed 123 omitted “se”. Both Q6 recordings of the English tested/saved-recordings prompt had four edits, also a difficult prompt for Q8 and FP32. These are ASR observations: they cannot isolate synthesis from recognition error. No statistical significance, blind listening, naturalness/MOS or voice-similarity claim is made. Every paired transcript, including unfavorable cases: | Language / prompt / seed | FP32 | Q8 | Q6 | Word edits FP32 / Q8 / Q6 | |---|---|---|---|---| | en / 0 / 42 | The weather is pleasant today and I'm going for a walk. | The weather is pleasant today and I am going for a walk. | The weather is pleasant and I'm going for a walk. | 2 / 0 / 3 | | en / 0 / 123 | The weather is pleasant today and I am going for a walk. | The weather is pleasant today and I'm going for a walk. | The weather is pleasant today and I am going for a walk. | 0 / 2 / 0 | | es / 1 / 42 | Hoy hace buen tiempo y voy a salir a caminar por el parque. | Hoy hace buen tiempo y voy a salir a caminar por el parque. | Hoy hace buen tiempo y voy a salir a caminar por el parque. | 0 / 0 / 0 | | es / 1 / 123 | Hoy hace buen tiempo y voy a salir a caminar por el parque. | Hoy hace buen tiempo y voy a salir a caminar por el parque. | Hoy hace buen tiempo y voy a salir a caminar por el parque. | 0 / 0 / 0 | | en / 2 / 42 | Could you check the microphone and tell me whether the sound is clear? | Could you check the microphone and tell me whether the sound is clear? | Could you check the microphone and tell me whether the sound is clear? | 0 / 0 / 0 | | en / 2 / 123 | Could you check the microphone and tell me whether the sound is good? | Could you check the microphone and tell me whether the sound is clear? | Could you check the microphone and tell me whether the sound is clear? | 1 / 0 / 0 | | es / 3 / 42 | ¿Podés revisar el micrófono y decirme si mi voz se escucha claramente? | ¿Podés revisar el micrófono y decirme si mi voz se escucha claramente? | ¿Puedes revisar el micrófono y decirme si mi voz se escucha claramente? | 0 / 0 / 1 | | es / 3 / 123 | ¿Podés revisar el micrófono y decirme si mi voz escucha claramente? | ¿Podés revisar el micrófono y decirme si mi voz se escucha claramente? | ¿Podés revisar el micrófono y decirme si mi voz escucha claramente? | 1 / 0 / 1 | | en / 4 / 42 | We test the smaller model twice and save both recording for comparison. | We test the smaller model twice and save Boss Recording for comparison. | We test the Smaller Model Twice and save Boss Recording for comparison. | 3 / 4 / 4 | | en / 4 / 123 | We test the smaller model twice, and save Boss Recording for comparison. | We test the smaller model twice, and save voice recording for comparison. | We test the smaller model twice and save Boss Recording for comparison. | 4 / 4 / 4 | | es / 5 / 42 | Probamos el modelo más pequeño dos veces y guardamos las grabaciones para compararlas. | Probamos el modelo más pequeño dos veces y guardamos las grabaciones para compararlas. | Probamos el modelo más pequeño dos veces y guardamos las grabaciones para compararlas. | 0 / 0 / 0 | | es / 5 / 123 | Probamos el modelo más pequeño dos veces y guardamos las grabaciones para compararlas. | Probamos el modelo más pequeño dos veces y guardamos las grabaciones para compararlas. | Probamos el modelo más pequeño dos veces y guardamos las grabaciones para compararlas. | 0 / 0 / 0 | ## Method and resource limitations Reference SHA-256: ace8518ab1d3b45e702a4c54700cab378a3d6e8172dfa8cd3dd51d8e176cfac3; the local profile was Valentina. Reference audio and generated clips remain private. Parameters: exaggeration=0.5, cfg_weight=0.5, temperature=0.8, repetition_penalty=1.2, min_p=0.05, top_p=0.95, max_new_tokens=400; MLX cache limit 512 MiB. Versions: {"mlx": "0.32.3", "mlx-audio": "0.5.7", "numpy": "2.4.6", "soundfile": "0.14.0", "huggingface-hub": "1.33.0"}. Each variant performed one separate warm-up. Timing includes audio materialization/synchronization, excludes loading, conditioning, warm-up, WAV writing and downloads. Loads use cached files and exclude imports; these are not cold HTTP service latencies. CPU percent is process CPU time / wall time, with 100% representing one core. ASR ran after all TTS generations: Parakeet TDT 0.6B v3 ONNX INT8 on CPU, four intra-op and one inter-op thread, offline local cache. Versions: {"onnx-asr": "0.12.0", "onnxruntime": "1.27.0", "numpy": "1.26.4"}. WER uses Unicode NFC, case-folding, punctuation removal, accents retained and contractions unexpanded; Levenshtein word edits / reference tokens. Two seeds per phrase are repeated linguistic examples, not 12 independent prompts. Power: macOS powermetrics, approximately 1-second samples weighted by overlap with generation intervals. Coverage seconds / total generation seconds: fp32 58.28/59.10, q8 31.33/32.25, q6 29.48/30.96. Thermal levels: {"fp32": ["Nominal"], "q8": ["Nominal"], "q6": ["Nominal"]}. Whole-chip CPU+GPU+ANE power includes other desktop work and excludes display, battery losses and charger power. Rounded timestamps make energy approximate. Startup, conditioning, warm-up and idle are excluded. This does not isolate TTS watts. No long-text, sustained thermal, streaming-latency or broad multilingual evaluation was performed. ## Reproduction and data Run [benchmark.py](benchmark.py) in separate processes, variants fp32, q8 and q6, with both quantized model folders and the same authorized reference. Score with [score_audio.py](score_audio.py). All prompts, seeds, metrics, transcript edits and paired differences: [results.json](results.json), [results.csv](results.csv). Runtime module lists: fp32-results.json, q8-results.json, q6-results.json. Power overlap samples: [power-samples.json](power-samples.json). ```bash python docs/performance/benchmark.py --variant q6 --source /path/to/fp32 \ --q8 /path/to/q8 --q6 /path/to/q6 --reference /path/to/your/reference.wav --out ./audio ``` The linked Hugging Face publication contains generic weights, model metadata, loader, checksums, scripts and measurement data. This experiment archive excludes weights and retains the reports, exact quantization policy and reproduction scripts. Private reference/generated voice audio, tokens, caches and local service configuration are excluded. The upstream MIT license and Resemble AI attribution are retained.