Ornith-1.5-9B-MACaQ-3bit-MLX

Calibrated MACaQ Low-Precision Checkpoint for Apple Silicon (MLX & Basalt)

bas-alt.vercel.app GitHub Run with Basalt

Overview

goodmai/Ornith-1.5-9B-MACaQ-3bit-MLX is a MACaQ (Model Agnostic Calibrated Quantization, arXiv:2611.10649) checkpoint quantized from ornith-ai/Ornith-1.5-9B-MLX-4bit for Apple Silicon Metal / MLX via Basalt (bas-alt.vercel.app).

  • Base Model: ornith-ai/Ornith-1.5-9B-MLX-4bit
  • Architecture / Family: qwen3_5 (text)
  • Quantization: MACaQ 3-bit affine (group_size=64) (macaq_quantization v1, channel-balance $\alpha = 0.75$, Sylvester block-Hadamard $B = 64$, active-set Hessian coordinate-descent code refinement)
  • Total Checkpoint Size: 4.20 GB
  • Runtime: Native single-simdgroup barrier-free Metal Walsh-Hadamard (macaq_single_simd_wht) + Apple Silicon quantizedMM in Basalt

Measured Benchmarks (Apple M4 Pro 24 GB)

Metric Source (Ornith-1.5-9B-MLX-4bit) 3-bit RTN Ornith-1.5-9B-MACaQ-3bit-MLX
Checkpoint Size 4.69 GB 3.58 GB 4.20 GB
Perplexity (1025-tok held-out) 6.125 8.096 6.493 (-19.8% vs RTN, 51.31 tok/s)
Hadamard Block (macaq_single_simd_wht) — — B = 64
Conversion Time (M4 Pro 24 GB) — — 986.0s
Raw Basalt PerformanceBenchmark & Generation Verification Output
Converted ornith-ai/Ornith-1.5-9B-MLX-4bit -> /private/tmp/Ornith-1.5-9B-MACaQ-3bit-MLX (MACaQ affine:3:64, B=64)
  MACaQ:     200 layer(s), 181861681 code flip(s), H-loss -39.7% in 1 shard(s)

Benchmark — model: /private/tmp/Ornith-1.5-9B-MACaQ-3bit-MLX
  iterations: 3  tokens: 80  warmup: true

Loading model… done

Warmup… done

  #     TPS         TTFT (s)    Time (s)
  ──────────────────────────────────────────
  1     53.94       0.129       1.62      
  2     53.71       0.138       1.64      
  3     53.86       0.135       1.63      

  TPS         avg=53.835    min=53.708    max=53.939    σ=0.096     tok/s
  TTFT        avg=0.134     min=0.129     max=0.138     σ=0.004     s
  GenTime     avg=1.629     min=1.621     max=1.637     σ=0.006     s
  Finish      length 3

--- OUTPUT ---
# The Importance of Open Source Models

Open source models represent a significant shift in how AI technology is developed, distributed, and accessed. Here are the key reasons they matter:

## 1. Democratization of Access

Open source models break down barriers to entry. Instead of requiring expensive API subscriptions or large budgets, organizations—especially smaller ones and researchers in resource-limited settings—can access powerful
--- END OUTPUT ---

  Memory (active): 3982 MB
  Hardware profile updated: /Users/alexeiboklag/.gem/hardware_profile.json (measured ✓)

Quick Start with Basalt

# Verify integrity and quantization manifest
basalt models inspect goodmai/Ornith-1.5-9B-MACaQ-3bit-MLX

# Serve OpenAI-compatible API + MCP on Apple Silicon
basalt serve --model goodmai/Ornith-1.5-9B-MACaQ-3bit-MLX

Method Reference

Quantized using Basalt's native Swift/Metal implementation of Huawei CSL's MACaQ algorithm:

  1. Channel Balance: $p_j = \mathrm{std}(W_{:,j})^{0.75} / \mathrm{rms}(X_{:,j})^{0.25}$ equalizes weight and activation outlier dynamic ranges prior to block rotation.
  2. Sylvester Walsh-Hadamard Frame: $W' = (W ,\mathrm{diag}(p)^{-1}) H_{64}$, $H' = H_{64}^\top (\mathrm{diag}(p), H, \mathrm{diag}(p)) H_{64}$.
  3. Active-Set Hessian Flip Refinement: Monotonic greedy $\pm 1$ discrete grid coordinate descent minimizing $\mathrm{tr}(\Delta W, H', \Delta W^\top)$ (with Hessian-weighted E8M0 exponent offset screening ${+1, 0, -1}$ for mxfp4).
Downloads last month
63
Safetensors
Model size
7B params
Tensor type
U32
·
BF16
·
MLX
Hardware compatibility
Log In to add your hardware

4-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for goodmai/Ornith-1.5-9B-MACaQ-3bit-MLX

Quantized
(2)
this model

Paper for goodmai/Ornith-1.5-9B-MACaQ-3bit-MLX