MACaQ: Fast, Model Agnostic Calibrated Quantization for Low-Precision Weights and Activations
Paper • 2611.10649 • Published • 30
How to use goodmai/Ornith-1.5-9B-MACaQ-3bit-MLX with MLX:
# Make sure mlx-lm is installed
# pip install --upgrade mlx-lm
# Generate text with mlx-lm
from mlx_lm import load, generate
model, tokenizer = load("goodmai/Ornith-1.5-9B-MACaQ-3bit-MLX")
prompt = "Write a story about Einstein"
messages = [{"role": "user", "content": prompt}]
prompt = tokenizer.apply_chat_template(
messages, add_generation_prompt=True
)
text = generate(model, tokenizer, prompt=prompt, verbose=True)How to use goodmai/Ornith-1.5-9B-MACaQ-3bit-MLX with Pi:
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "goodmai/Ornith-1.5-9B-MACaQ-3bit-MLX"
# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
"providers": {
"mlx-lm": {
"baseUrl": "http://localhost:8080/v1",
"api": "openai-completions",
"apiKey": "none",
"models": [
{
"id": "goodmai/Ornith-1.5-9B-MACaQ-3bit-MLX"
}
]
}
}
}# Start Pi in your project directory: pi
How to use goodmai/Ornith-1.5-9B-MACaQ-3bit-MLX with MLX LM:
# Install MLX LM uv tool install mlx-lm # Interactive chat REPL mlx_lm.chat --model "goodmai/Ornith-1.5-9B-MACaQ-3bit-MLX"
# Install MLX LM
uv tool install mlx-lm
# Start the server
mlx_lm.server --model "goodmai/Ornith-1.5-9B-MACaQ-3bit-MLX"
# Calling the OpenAI-compatible server with curl
curl -X POST "http://localhost:8080/v1/chat/completions" \
-H "Content-Type: application/json" \
--data '{
"model": "goodmai/Ornith-1.5-9B-MACaQ-3bit-MLX",
"messages": [
{"role": "user", "content": "Hello"}
]
}'How to use goodmai/Ornith-1.5-9B-MACaQ-3bit-MLX with Hermes Agent:
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "goodmai/Ornith-1.5-9B-MACaQ-3bit-MLX"
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default goodmai/Ornith-1.5-9B-MACaQ-3bit-MLX
hermes
How to use goodmai/Ornith-1.5-9B-MACaQ-3bit-MLX with OpenClaw:
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "goodmai/Ornith-1.5-9B-MACaQ-3bit-MLX"
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "goodmai/Ornith-1.5-9B-MACaQ-3bit-MLX" \ --custom-provider-id mlx-lm \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
openclaw agent --local --agent main --message "Hello from Hugging Face"
Calibrated MACaQ Low-Precision Checkpoint for Apple Silicon (MLX & Basalt)
goodmai/Ornith-1.5-9B-MACaQ-3bit-MLX is a MACaQ (Model Agnostic Calibrated Quantization, arXiv:2611.10649) checkpoint quantized from ornith-ai/Ornith-1.5-9B-MLX-4bit for Apple Silicon Metal / MLX via Basalt (bas-alt.vercel.app).
ornith-ai/Ornith-1.5-9B-MLX-4bitqwen3_5 (text)MACaQ 3-bit affine (group_size=64) (macaq_quantization v1, channel-balance $\alpha = 0.75$, Sylvester block-Hadamard $B = 64$, active-set Hessian coordinate-descent code refinement)4.20 GBmacaq_single_simd_wht) + Apple Silicon quantizedMM in Basalt| Metric | Source (Ornith-1.5-9B-MLX-4bit) |
3-bit RTN | Ornith-1.5-9B-MACaQ-3bit-MLX |
|---|---|---|---|
| Checkpoint Size | 4.69 GB |
3.58 GB |
4.20 GB |
Perplexity (1025-tok held-out) |
6.125 |
8.096 |
6.493 (-19.8% vs RTN, 51.31 tok/s) |
Hadamard Block (macaq_single_simd_wht) |
— | — | B = 64 |
| Conversion Time (M4 Pro 24 GB) | — | — | 986.0s |
Converted ornith-ai/Ornith-1.5-9B-MLX-4bit -> /private/tmp/Ornith-1.5-9B-MACaQ-3bit-MLX (MACaQ affine:3:64, B=64)
MACaQ: 200 layer(s), 181861681 code flip(s), H-loss -39.7% in 1 shard(s)
Benchmark — model: /private/tmp/Ornith-1.5-9B-MACaQ-3bit-MLX
iterations: 3 tokens: 80 warmup: true
Loading model… done
Warmup… done
# TPS TTFT (s) Time (s)
──────────────────────────────────────────
1 53.94 0.129 1.62
2 53.71 0.138 1.64
3 53.86 0.135 1.63
TPS avg=53.835 min=53.708 max=53.939 σ=0.096 tok/s
TTFT avg=0.134 min=0.129 max=0.138 σ=0.004 s
GenTime avg=1.629 min=1.621 max=1.637 σ=0.006 s
Finish length 3
--- OUTPUT ---
# The Importance of Open Source Models
Open source models represent a significant shift in how AI technology is developed, distributed, and accessed. Here are the key reasons they matter:
## 1. Democratization of Access
Open source models break down barriers to entry. Instead of requiring expensive API subscriptions or large budgets, organizations—especially smaller ones and researchers in resource-limited settings—can access powerful
--- END OUTPUT ---
Memory (active): 3982 MB
Hardware profile updated: /Users/alexeiboklag/.gem/hardware_profile.json (measured ✓)
# Verify integrity and quantization manifest
basalt models inspect goodmai/Ornith-1.5-9B-MACaQ-3bit-MLX
# Serve OpenAI-compatible API + MCP on Apple Silicon
basalt serve --model goodmai/Ornith-1.5-9B-MACaQ-3bit-MLX
Quantized using Basalt's native Swift/Metal implementation of Huawei CSL's MACaQ algorithm:
mxfp4).4-bit
Base model
ornith-ai/Ornith-1.5-9B-MLX-4bit