Ternary-Bonsai-2-4B-mentria

prism-ml's Ternary Bonsai 2 4B, a ternary (−1 / 0 / +1) vision-language model built on Qwen3.5-4B, repacked into mentria-q2g128-v1 safetensors for mentria-engine, a custom WebGPU runtime that runs the model entirely in your browser. 4 billion parameters in 1.14 GB, plus a 210 MB vision tower for image input.

Status: the engine runs this model and its outputs match the reference runtime (below), and on 2026-10-28 it was chosen to replace our Qwen3.5-2B-mentria on mentria.ai, as the model for the middle tier. The website rollout is under way.

Files

Path	Size	Purpose
bonsai2-4b-q2g128.safetensors	~1.14 GB	Language model
qwen3.5-4b-vl-q4.safetensors	~210 MB	Vision tower (optional): the stock Qwen3.5-4B tower in Q4_0, the same file as in Qwen3.5-4B-mentria
tokenizer.json	~12 MB	Qwen3.5 BPE tokenizer (248,320 vocab)
tokenizer_config.json	~16 KB	Special-token IDs, chat-template metadata
chat_template.jinja	~8 KB	Standalone chat template
preprocessor_config.json	~400 B	Image preprocessing constants

Format: mentria-q2g128-v1

Each quantized weight is a ternary code (−1, 0 or +1) with one FP16 scale per 128-weight group, about 2.13 bits per weight. The weights are repacked from the upstream GGUF, not re-quantized. Bonsai 2 stores its weights in a Hadamard-rotated basis; the engine applies the matching rotation to each matmul input in f32 (block size 512 on this model), so the rotation is exact.

Correctness

Teacher-forced golden test against the PrismML llama.cpp reference: 433 / 433 positions agree, zero decisive disagreements. Teacher-forced means both runtimes see the same context at every position and we compare their top-1 token, so one early difference cannot snowball. A decisive disagreement is one where the reference's top two candidates are more than 0.15 probability apart. Greedy output was identical to the reference on all four free-running prompts.

The browser build that mentria.ai serves gives the same predictions, token for token (Chrome).

Why mentria.ai switched to this model

Each Bonsai 2 model was paired with the 4-bit model of about the same download size that mentria.ai served before (our Q4 conversions of Qwen3.5). Both models ran in mentria-engine, on the same questions, scored by the same scripts, under a test plan written before the runs. This model replaces Qwen3.5-2B (1,191 MB against this model's 1,148 MB, language model plus tokenizer).

Benchmarks: full test sets

Protocol: greedy decoding, thinking off, no system prompt, no repetition penalty, 2,048-token context; dawn.node on an Apple M4 Pro. Differences are Bonsai minus Q4 in points, with 95% bootstrap intervals and an exact McNemar test on the paired answers.

Benchmark (n)	Q4 Qwen3.5-2B	Bonsai 2 4B (this model)	Difference, points (95% CI)	p
GSM8K, maths word problems (1,319)	73.1%	89.8%	+16.7 (14.4, 19.0)	< 0.001
IFEval, following instructions, prompt-level strict (541)	64.7%	76.9%	+12.2 (7.9, 16.5)	< 0.001
MMLU-Redux, knowledge, 20 per subject × 57 subjects (1,140)	59.5%	70.1%	+10.6 (7.7, 13.6)	< 0.001

The lead does not depend on our engine: against the same Q4 weights run in MLX instead, a 197-item pilot read GSM8K +15, IFEval +18 and MMLU-Redux 0 points.

Chat: a pilot only

20 hard prompts in English, Spanish, Portuguese (BR), French and Japanese, both orders, one blind Claude Opus judge: Bonsai 2 4B's reply was preferred 83.8% of the time (95% CI 68–96). The full 650-prompt chat test was not run for this pair.

Speed in mentria-engine (Mac, Apple M4 Pro, Chrome)

	Q4 Qwen3.5-2B	Bonsai 2 4B
Reply (decode), 512-token prompt	59.7 tok/s	88.3 tok/s
First word, 512-token prompt	0.70 s	1.62 s

• Bonsai shows its first word later on a long prompt, but writes 1.48× faster, so realistic chat replies finish 12–28% sooner.
• Every speed-up in the engine is exact: on each device, the faster kernels give bit-for-bit the same output as the reference path (checked on the device when the model loads).

Model details

Base model	prism-ml/Ternary-Bonsai-2-4B-gguf (built on Qwen/Qwen3.5-4B)
Architecture	Qwen3.5 hybrid: 32 layers, 24 Gated-DeltaNet (16 key / 32 value heads) + 8 full-attention (every 4th layer; 16Q / 4KV, head dim 256), hidden 2560, FFN 9216
Quantization	mentria-q2g128-v1: ternary weights, group size 128, FP16 scale per group, Hadamard-rotated basis
Vision tower	24-layer ViT, 1024 hidden, patch 16, spatial merge 2, projects to the 2,560-wide text embeddings
Tokenizer	Qwen3.5 BPE, 248,320 vocab
License	Apache 2.0 (upstream license preserved)

About mentria.ai

Mentria is AI that runs in your browser, with built-in tools, an extensions library and card decks for learning. Add your own extensions and decks. Your data stays private.