---
license: apache-2.0
base_model: ornith-ai/Ornith-1.5-35B-A3B-MLX
pipeline_tag: text-generation
library_name: mlx
tags:
- mlx
- basalt
- apple-silicon
- quantization
- macaq
- hadamard
- qwen3_5_moe
---
Ornith-1.5-35B-A3B-MACaQ-3bit-MLX
Calibrated MACaQ Low-Precision Checkpoint for Apple Silicon (MLX & Basalt)
## Overview
`goodmai/Ornith-1.5-35B-A3B-MACaQ-3bit-MLX` is a **MACaQ** (*Model Agnostic Calibrated Quantization*, [arXiv:2611.10649](https://arxiv.org/abs/2611.10649)) checkpoint quantized from [`ornith-ai/Ornith-1.5-35B-A3B-MLX`](https://huggingface.co/ornith-ai/Ornith-1.5-35B-A3B-MLX) for Apple Silicon Metal / MLX via **[Basalt](https://github.com/goodmai/basalt)** ([bas-alt.vercel.app](https://bas-alt.vercel.app)).
- **Base Model**: [`ornith-ai/Ornith-1.5-35B-A3B-MLX`](https://huggingface.co/ornith-ai/Ornith-1.5-35B-A3B-MLX)
- **Architecture / Family**: `qwen3_5_moe` (`text`)
- **Quantization**: `MACaQ 3-bit MoE (group_size=64, 256 experts)` (`macaq_quantization` v1, channel-balance $\alpha = 0.75$, Sylvester block-Hadamard $B = 64$, active-set Hessian coordinate-descent code refinement)
- **Total Checkpoint Size**: `15.36 GB`
- **Runtime**: Native single-simdgroup barrier-free Metal Walsh-Hadamard (`macaq_single_simd_wht`) + Apple Silicon `quantizedMM` in [Basalt](https://github.com/goodmai/basalt)
## Measured Benchmarks (Apple M4 Pro 24 GB)
| Metric | Official BF16 (`ornith-ai/Ornith-1.5-35B-A3B-MLX`) | **`Ornith-1.5-35B-A3B-MACaQ-3bit-MLX`** |
|---|---:|---:|
| Checkpoint Size | `69.32 GB` | **`15.36 GB` (`4.51×` compression — fits on 24 GB Macs!)** |
| Architecture | `qwen3_5_moe` (`40` layers, `256` experts, `A3B` active) | **MACaQ 3-bit (`B=64, α=0.75`) + 4-bit embeddings/head** |
| Quantized Modules | — | **`250` dense + `120` stacked 256-expert `MACaQSwitchLinear` (`773,928,363` flips)** |
| Integrity (`basalt models inspect`) | `intact=true` | **`intact=true`, `quantizationMode=macaq` (`5356.6s` on M4 Pro)** |
Raw Basalt PerformanceBenchmark & Live Generation Output (M4 Pro 24 GB)
```text
Benchmark — model: /private/tmp/Ornith-1.5-35B-A3B-MACaQ-3bit-MLX
iterations: 3 tokens: 80 warmup: true
Loading model… done
Warmup… done
# TPS TTFT (s) Time (s)
──────────────────────────────────────────
1 77.35 0.128 1.17
2 78.23 0.135 1.16
3 78.12 0.116 1.14
TPS avg=77.899 min=77.349 max=78.228 σ=0.391 tok/s
TTFT avg=0.126 min=0.116 max=0.135 σ=0.008 s
GenTime avg=1.157 min=1.143 max=1.166 σ=0.010 s
Finish length 3
--- OUTPUT ---
Open source models have become increasingly important across software, AI, and various technical domains. Here's an overview of why they matter:
## Access and Affordability
Open source models are typically free or low-cost to access, which democratizes technology. This lowers barriers to entry for individuals, startups, and organizations that might not afford expensive proprietary alternatives. It allows wider participation in innovation regardless of budget
--- END OUTPUT ---
Memory (active): 14623 MB
Hardware profile updated: /Users/alexeiboklag/.gem/hardware_profile.json (measured ✓)
```
## Quick Start with Basalt
```bash
# Verify 14 shards and MACaQ MoE quantization manifest
basalt models inspect goodmai/Ornith-1.5-35B-A3B-MACaQ-3bit-MLX
# Run 35B-A3B MoE natively on 24 GB+ Apple Silicon Macs!
basalt serve --model goodmai/Ornith-1.5-35B-A3B-MACaQ-3bit-MLX
```
## Method Reference
Quantized using Basalt's native Swift/Metal implementation of Huawei CSL's **MACaQ** algorithm:
1. **Channel Balance**: $p_j = \mathrm{std}(W_{:,j})^{0.75} / \mathrm{rms}(X_{:,j})^{0.25}$ equalizes weight and activation outlier dynamic ranges prior to block rotation.
2. **Sylvester Walsh-Hadamard Frame**: $W' = (W \,\mathrm{diag}(p)^{-1}) H_{64}$, $H' = H_{64}^\top (\mathrm{diag}(p)\, H\, \mathrm{diag}(p)) H_{64}$.
3. **Active-Set Hessian Flip Refinement**: Monotonic greedy $\pm 1$ discrete grid coordinate descent minimizing $\mathrm{tr}(\Delta W\, H'\, \Delta W^\top)$ (with Hessian-weighted E8M0 exponent offset screening $\{+1, 0, -1\}$ for `mxfp4`).