Ornith-1.5-35B-A3B-MACaQ-3bit-MLX

Calibrated MACaQ Low-Precision Checkpoint for Apple Silicon (MLX & Basalt)

bas-alt.vercel.app GitHub Run with Basalt

Overview

goodmai/Ornith-1.5-35B-A3B-MACaQ-3bit-MLX is a MACaQ (Model Agnostic Calibrated Quantization, arXiv:2611.10649) checkpoint quantized from ornith-ai/Ornith-1.5-35B-A3B-MLX for Apple Silicon Metal / MLX via Basalt (bas-alt.vercel.app).

  • Base Model: ornith-ai/Ornith-1.5-35B-A3B-MLX
  • Architecture / Family: qwen3_5_moe (text)
  • Quantization: MACaQ 3-bit MoE (group_size=64, 256 experts) (macaq_quantization v1, channel-balance $\alpha = 0.75$, Sylvester block-Hadamard $B = 64$, active-set Hessian coordinate-descent code refinement)
  • Total Checkpoint Size: 15.36 GB
  • Runtime: Native single-simdgroup barrier-free Metal Walsh-Hadamard (macaq_single_simd_wht) + Apple Silicon quantizedMM in Basalt

Measured Benchmarks (Apple M4 Pro 24 GB)

Metric Official BF16 (ornith-ai/Ornith-1.5-35B-A3B-MLX) Ornith-1.5-35B-A3B-MACaQ-3bit-MLX
Checkpoint Size 69.32 GB 15.36 GB (4.51× compression — fits on 24 GB Macs!)
Architecture qwen3_5_moe (40 layers, 256 experts, A3B active) MACaQ 3-bit (B=64, α=0.75) + 4-bit embeddings/head
Quantized Modules — 250 dense + 120 stacked 256-expert MACaQSwitchLinear (773,928,363 flips)
Integrity (basalt models inspect) intact=true intact=true, quantizationMode=macaq (5356.6s on M4 Pro)
Raw Basalt PerformanceBenchmark & Live Generation Output (M4 Pro 24 GB)
Benchmark — model: /private/tmp/Ornith-1.5-35B-A3B-MACaQ-3bit-MLX
  iterations: 3  tokens: 80  warmup: true

Loading model… done

Warmup… done

  #     TPS         TTFT (s)    Time (s)
  ──────────────────────────────────────────
  1     77.35       0.128       1.17      
  2     78.23       0.135       1.16      
  3     78.12       0.116       1.14      

  TPS         avg=77.899    min=77.349    max=78.228    σ=0.391     tok/s
  TTFT        avg=0.126     min=0.116     max=0.135     σ=0.008     s
  GenTime     avg=1.157     min=1.143     max=1.166     σ=0.010     s
  Finish      length 3

--- OUTPUT ---
Open source models have become increasingly important across software, AI, and various technical domains. Here's an overview of why they matter:

## Access and Affordability

Open source models are typically free or low-cost to access, which democratizes technology. This lowers barriers to entry for individuals, startups, and organizations that might not afford expensive proprietary alternatives. It allows wider participation in innovation regardless of budget
--- END OUTPUT ---

  Memory (active): 14623 MB
  Hardware profile updated: /Users/alexeiboklag/.gem/hardware_profile.json (measured ✓)

Quick Start with Basalt

# Verify 14 shards and MACaQ MoE quantization manifest
basalt models inspect goodmai/Ornith-1.5-35B-A3B-MACaQ-3bit-MLX

# Run 35B-A3B MoE natively on 24 GB+ Apple Silicon Macs!
basalt serve --model goodmai/Ornith-1.5-35B-A3B-MACaQ-3bit-MLX

Method Reference

Quantized using Basalt's native Swift/Metal implementation of Huawei CSL's MACaQ algorithm:

  1. Channel Balance: $p_j = \mathrm{std}(W_{:,j})^{0.75} / \mathrm{rms}(X_{:,j})^{0.25}$ equalizes weight and activation outlier dynamic ranges prior to block rotation.
  2. Sylvester Walsh-Hadamard Frame: $W' = (W ,\mathrm{diag}(p)^{-1}) H_{64}$, $H' = H_{64}^\top (\mathrm{diag}(p), H, \mathrm{diag}(p)) H_{64}$.
  3. Active-Set Hessian Flip Refinement: Monotonic greedy $\pm 1$ discrete grid coordinate descent minimizing $\mathrm{tr}(\Delta W, H', \Delta W^\top)$ (with Hessian-weighted E8M0 exponent offset screening ${+1, 0, -1}$ for mxfp4).
Downloads last month
120
Safetensors
Model size
26B params
Tensor type
U32
·
BF16
·
MLX
Hardware compatibility
Log In to add your hardware

4-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for goodmai/Ornith-1.5-35B-A3B-MACaQ-3bit-MLX

Quantized
(2)
this model

Paper for goodmai/Ornith-1.5-35B-A3B-MACaQ-3bit-MLX