--- library_name: mentria-engine base_model: prism-ml/Ternary-Bonsai-2-9B-gguf base_model_relation: quantized license: apache-2.0 language: - en pipeline_tag: image-text-to-text tags: - safetensors - qwen3_5 - mentria-engine - webgpu - ternary - q2g128 - on-device - multimodal --- # Ternary-Bonsai-2-9B-mentria prism-ml's **Ternary Bonsai 2 9B**, a ternary (−1 / 0 / +1) vision-language model built on Qwen3.5-9B, repacked into `mentria-q2g128-v1` safetensors for **mentria-engine**, a custom WebGPU runtime that runs the model entirely in your browser. 9 billion parameters in **2.41 GB**, plus a **477 MB vision tower** for image input. **Status:** the engine runs this model and its outputs match the reference runtime (below), and on 2026-10-28 it was **chosen to replace our [Qwen3.5-4B-mentria](https://huggingface.co/mentriaai/Qwen3.5-4B-mentria) on mentria.ai**, as the model for the top desktop tier. The website rollout is under way. ## Files | Path | Size | Purpose | |---|---|---| | `bonsai2-9b-q2g128-00001-of-00002.safetensors` | ~1.9 GB | Language model, shard 1 of 2. Split under 2 GB so each shard fits a JS ArrayBuffer | | `bonsai2-9b-q2g128-00002-of-00002.safetensors` | ~509 MB | Language model, shard 2 of 2 | | `qwen3.5-9b-vl-q4.safetensors` | ~477 MB | **Vision tower** (optional): Q4_0 with MSE-optimal scales, with the MLP input matrices and one output matrix kept at BF16 (see below) | | `tokenizer.json` | ~12 MB | Qwen3.5 BPE tokenizer (248,320 vocab) | | `tokenizer_config.json` | ~16 KB | Special-token IDs, chat-template metadata | | `chat_template.jinja` | ~8 KB | Standalone chat template | | `preprocessor_config.json` | ~400 B | Image preprocessing constants | ## Format: mentria-q2g128-v1 Each quantized weight is a ternary code (−1, 0 or +1) with one FP16 scale per 128-weight group, about 2.13 bits per weight. The weights are repacked from the upstream GGUF, not re-quantized. Bonsai 2 stores its weights in a Hadamard-rotated basis; the engine applies the matching rotation to each matmul input in f32 (block size 4,096 on this model), so the rotation is exact. ## Correctness Teacher-forced golden test against the PrismML llama.cpp reference: **373 / 373 positions agree, zero decisive disagreements.** Teacher-forced means both runtimes see the same context at every position and we compare their top-1 token, so one early difference cannot snowball. A decisive disagreement is one where the reference's top two candidates are more than 0.15 probability apart. Greedy output was identical to the reference on all four free-running prompts. The browser build that mentria.ai serves gives the same predictions, token for token (Chrome). ## Why mentria.ai switched to this model Each Bonsai 2 model was paired with the 4-bit model of about the same download size that mentria.ai served before (our Q4 conversions of Qwen3.5). Both models ran in mentria-engine, on the same questions, scored by the same scripts, under a test plan written before the runs. This model replaces Qwen3.5-4B (2,645 MB against this model's 2,419 MB, language model plus tokenizer). ### Benchmarks: a pilot only A 197-item pilot (n = 60 / 57 / 60). Greedy decoding, no system prompt, no repetition penalty, Apple M4 Pro. The Q4 model ran in mentria-engine; this model ran in PrismML's llama.cpp fork, whose predictions mentria-engine matches at every position of the golden test above. The full test sets were not run for this pair. | Benchmark (n) | Q4 Qwen3.5-4B | **Bonsai 2 9B (this model)** | Difference, points (95% CI) | p | |---|---:|---:|---|---:| | GSM8K, maths word problems (60) | 90.0% | 85.0% | −5.0 (−13.3, 1.7) | 0.38 | | MMLU-Redux, knowledge (57) | 82.5% | 80.7% | −1.8 (−8.8, 5.3) | 1.0 | | IFEval, following instructions, prompt-level strict (60) | 65.0% | **80.0%** | **+15.0** (3.3, 26.7) | 0.035 | Maths and knowledge are level within the pilot's noise; instruction following is clearly better. In a 20-prompt blind chat pilot (five languages, one Claude Opus judge) this model was preferred 58.8% of the time (95% CI 39–78), not a significant difference. ### Speed in mentria-engine (Mac, Apple M4 Pro, Chrome) | | Q4 Qwen3.5-4B | **Bonsai 2 9B** | |---|---:|---:| | Reply (decode), 512-token prompt | 23.9–24.1 tok/s | **57.4–58.4 tok/s** (2.4×) | | First word, 512-token prompt | **2.0 s** | 3.1 s | | First word, 23-token prompt | 0.29 s | **0.26 s** | - A 128-token reply to a 512-token prompt takes about 5.3 s on this model against 7.3 s on the Q4 model (computed from the measured parts). - With this model all three website tiers run one model family, Bonsai 2. - Every speed-up in the engine is exact: on each device, the faster kernels give bit-for-bit the same output as the reference path (checked on the device when the model loads). ## Vision tower The 9B tower is the stock Qwen3.5-9B vision tower (PrismML's BF16 mmproj equals it on all 334 tensors): 27 blocks, width 1152, 16 heads of 72, about 0.46 B parameters, projecting to the 4,096-wide text embeddings. Most of the tower is Q4_0 with MSE-optimal per-block scales. Two kinds of matrix stay at BF16: - the 27 MLP input matrices, because their inner dimension (4,304) is not a multiple of the Q4 block size of 32; - the last block's MLP output matrix. One extreme output channel in it caused almost all of the drift in a plain Q4 build; keeping it at BF16 costs 6.8 MB. Measured against the BF16 tower on the same images (teacher-forced: the same image rows and text at every position): | Tower | Size | Image embedding cosine | Same next token as BF16 | Decisive misses | |---|---:|---:|---:|---:| | **This file** | 477 MB | 0.974 | 96.1% | 4 | | Plain Q4 build | 470 MB | 0.700 | 93.0% | 16 | | BF16 | 912 MB | 1.000 | 99.8% | 0 | With identical image rows, the engine matches the reference runtime at 610 / 610 tokens. ## Model details | | | |---|---| | **Base model** | [prism-ml/Ternary-Bonsai-2-9B-gguf](https://huggingface.co/prism-ml/Ternary-Bonsai-2-9B-gguf) (built on [Qwen/Qwen3.5-9B](https://huggingface.co/Qwen/Qwen3.5-9B)) | | **Architecture** | Qwen3.5 hybrid: 32 layers, 24 Gated-DeltaNet (16 key / 32 value heads) + 8 full-attention (every 4th layer; 16Q / 4KV, head dim 256), hidden 4096, FFN 12288 | | **Quantization** | mentria-q2g128-v1: ternary weights, group size 128, FP16 scale per group, Hadamard-rotated basis | | **Tokenizer** | Qwen3.5 BPE, 248,320 vocab | | **License** | Apache 2.0 (upstream license preserved) | ## About mentria.ai [Mentria](https://mentria.ai) is AI that runs in your browser, with built-in tools, an extensions library and card decks for learning. Add your own extensions and decks. Your data stays private.