Ternary-Bonsai-2-9B-mentria

prism-ml's Ternary Bonsai 2 9B, a ternary (−1 / 0 / +1) vision-language model built on Qwen3.5-9B, repacked into mentria-q2g128-v1 safetensors for mentria-engine, a custom WebGPU runtime that runs the model entirely in your browser. 9 billion parameters in 2.41 GB, plus a 477 MB vision tower for image input.

Status: the engine runs this model and its outputs match the reference runtime (below), and on 2026-10-28 it was chosen to replace our Qwen3.5-4B-mentria on mentria.ai, as the model for the top desktop tier. The website rollout is under way.

Files

Path	Size	Purpose
bonsai2-9b-q2g128-00001-of-00002.safetensors	~1.9 GB	Language model, shard 1 of 2. Split under 2 GB so each shard fits a JS ArrayBuffer
bonsai2-9b-q2g128-00002-of-00002.safetensors	~509 MB	Language model, shard 2 of 2
qwen3.5-9b-vl-q4.safetensors	~477 MB	Vision tower (optional): Q4_0 with MSE-optimal scales, with the MLP input matrices and one output matrix kept at BF16 (see below)
tokenizer.json	~12 MB	Qwen3.5 BPE tokenizer (248,320 vocab)
tokenizer_config.json	~16 KB	Special-token IDs, chat-template metadata
chat_template.jinja	~8 KB	Standalone chat template
preprocessor_config.json	~400 B	Image preprocessing constants

Format: mentria-q2g128-v1

Each quantized weight is a ternary code (−1, 0 or +1) with one FP16 scale per 128-weight group, about 2.13 bits per weight. The weights are repacked from the upstream GGUF, not re-quantized. Bonsai 2 stores its weights in a Hadamard-rotated basis; the engine applies the matching rotation to each matmul input in f32 (block size 4,096 on this model), so the rotation is exact.

Correctness

Teacher-forced golden test against the PrismML llama.cpp reference: 373 / 373 positions agree, zero decisive disagreements. Teacher-forced means both runtimes see the same context at every position and we compare their top-1 token, so one early difference cannot snowball. A decisive disagreement is one where the reference's top two candidates are more than 0.15 probability apart. Greedy output was identical to the reference on all four free-running prompts.

The browser build that mentria.ai serves gives the same predictions, token for token (Chrome).

Why mentria.ai switched to this model

Each Bonsai 2 model was paired with the 4-bit model of about the same download size that mentria.ai served before (our Q4 conversions of Qwen3.5). Both models ran in mentria-engine, on the same questions, scored by the same scripts, under a test plan written before the runs. This model replaces Qwen3.5-4B (2,645 MB against this model's 2,419 MB, language model plus tokenizer).

Benchmarks: a pilot only

A 197-item pilot (n = 60 / 57 / 60). Greedy decoding, no system prompt, no repetition penalty, Apple M4 Pro. The Q4 model ran in mentria-engine; this model ran in PrismML's llama.cpp fork, whose predictions mentria-engine matches at every position of the golden test above. The full test sets were not run for this pair.

Benchmark (n)	Q4 Qwen3.5-4B	Bonsai 2 9B (this model)	Difference, points (95% CI)	p
GSM8K, maths word problems (60)	90.0%	85.0%	−5.0 (−13.3, 1.7)	0.38
MMLU-Redux, knowledge (57)	82.5%	80.7%	−1.8 (−8.8, 5.3)	1.0
IFEval, following instructions, prompt-level strict (60)	65.0%	80.0%	+15.0 (3.3, 26.7)	0.035

Maths and knowledge are level within the pilot's noise; instruction following is clearly better. In a 20-prompt blind chat pilot (five languages, one Claude Opus judge) this model was preferred 58.8% of the time (95% CI 39–78), not a significant difference.

Speed in mentria-engine (Mac, Apple M4 Pro, Chrome)

	Q4 Qwen3.5-4B	Bonsai 2 9B
Reply (decode), 512-token prompt	23.9–24.1 tok/s	57.4–58.4 tok/s (2.4×)
First word, 512-token prompt	2.0 s	3.1 s
First word, 23-token prompt	0.29 s	0.26 s

• A 128-token reply to a 512-token prompt takes about 5.3 s on this model against 7.3 s on the Q4 model (computed from the measured parts).
• With this model all three website tiers run one model family, Bonsai 2.
• Every speed-up in the engine is exact: on each device, the faster kernels give bit-for-bit the same output as the reference path (checked on the device when the model loads).

Vision tower

The 9B tower is the stock Qwen3.5-9B vision tower (PrismML's BF16 mmproj equals it on all 334 tensors): 27 blocks, width 1152, 16 heads of 72, about 0.46 B parameters, projecting to the 4,096-wide text embeddings.

Most of the tower is Q4_0 with MSE-optimal per-block scales. Two kinds of matrix stay at BF16:

• the 27 MLP input matrices, because their inner dimension (4,304) is not a multiple of the Q4 block size of 32;
• the last block's MLP output matrix. One extreme output channel in it caused almost all of the drift in a plain Q4 build; keeping it at BF16 costs 6.8 MB.

Measured against the BF16 tower on the same images (teacher-forced: the same image rows and text at every position):

Tower	Size	Image embedding cosine	Same next token as BF16	Decisive misses
This file	477 MB	0.974	96.1%	4
Plain Q4 build	470 MB	0.700	93.0%	16
BF16	912 MB	1.000	99.8%	0

With identical image rows, the engine matches the reference runtime at 610 / 610 tokens.

Model details

Base model	prism-ml/Ternary-Bonsai-2-9B-gguf (built on Qwen/Qwen3.5-9B)
Architecture	Qwen3.5 hybrid: 32 layers, 24 Gated-DeltaNet (16 key / 32 value heads) + 8 full-attention (every 4th layer; 16Q / 4KV, head dim 256), hidden 4096, FFN 12288
Quantization	mentria-q2g128-v1: ternary weights, group size 128, FP16 scale per group, Hadamard-rotated basis
Tokenizer	Qwen3.5 BPE, 248,320 vocab
License	Apache 2.0 (upstream license preserved)

About mentria.ai

Mentria is AI that runs in your browser, with built-in tools, an extensions library and card decks for learning. Add your own extensions and decks. Your data stays private.