Text Generation
MLX
Safetensors
qwen3_5_moe
basalt
apple-silicon
quantization
macaq
hadamard
conversational
4-bit precision
Instructions to use goodmai/Ornith-1.5-35B-A3B-MACaQ-3bit-MLX with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use goodmai/Ornith-1.5-35B-A3B-MACaQ-3bit-MLX with MLX:
# Make sure mlx-lm is installed # pip install --upgrade mlx-lm # Generate text with mlx-lm from mlx_lm import load, generate model, tokenizer = load("goodmai/Ornith-1.5-35B-A3B-MACaQ-3bit-MLX") prompt = "Write a story about Einstein" messages = [{"role": "user", "content": prompt}] prompt = tokenizer.apply_chat_template( messages, add_generation_prompt=True ) text = generate(model, tokenizer, prompt=prompt, verbose=True) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- Pi
How to use goodmai/Ornith-1.5-35B-A3B-MACaQ-3bit-MLX with Pi:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "goodmai/Ornith-1.5-35B-A3B-MACaQ-3bit-MLX"
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "mlx-lm": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "goodmai/Ornith-1.5-35B-A3B-MACaQ-3bit-MLX" } ] } } }Run Pi
# Start Pi in your project directory: pi
- MLX LM
How to use goodmai/Ornith-1.5-35B-A3B-MACaQ-3bit-MLX with MLX LM:
Generate or start a chat session
# Install MLX LM uv tool install mlx-lm # Interactive chat REPL mlx_lm.chat --model "goodmai/Ornith-1.5-35B-A3B-MACaQ-3bit-MLX"
Run an OpenAI-compatible server
# Install MLX LM uv tool install mlx-lm # Start the server mlx_lm.server --model "goodmai/Ornith-1.5-35B-A3B-MACaQ-3bit-MLX" # Calling the OpenAI-compatible server with curl curl -X POST "http://localhost:8080/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "goodmai/Ornith-1.5-35B-A3B-MACaQ-3bit-MLX", "messages": [ {"role": "user", "content": "Hello"} ] }' - Hermes Agent
How to use goodmai/Ornith-1.5-35B-A3B-MACaQ-3bit-MLX with Hermes Agent:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "goodmai/Ornith-1.5-35B-A3B-MACaQ-3bit-MLX"
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default goodmai/Ornith-1.5-35B-A3B-MACaQ-3bit-MLX
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use goodmai/Ornith-1.5-35B-A3B-MACaQ-3bit-MLX with OpenClaw:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "goodmai/Ornith-1.5-35B-A3B-MACaQ-3bit-MLX"
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "goodmai/Ornith-1.5-35B-A3B-MACaQ-3bit-MLX" \ --custom-provider-id mlx-lm \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
Ornith-1.5-35B-A3B-MACaQ-3bit-MLX
Calibrated MACaQ Low-Precision Checkpoint for Apple Silicon (MLX & Basalt)
Overview
goodmai/Ornith-1.5-35B-A3B-MACaQ-3bit-MLX is a MACaQ (Model Agnostic Calibrated Quantization, arXiv:2611.10649) checkpoint quantized from ornith-ai/Ornith-1.5-35B-A3B-MLX for Apple Silicon Metal / MLX via Basalt (bas-alt.vercel.app).
- Base Model:
ornith-ai/Ornith-1.5-35B-A3B-MLX - Architecture / Family:
qwen3_5_moe(text) - Quantization:
MACaQ 3-bit MoE (group_size=64, 256 experts)(macaq_quantizationv1, channel-balance $\alpha = 0.75$, Sylvester block-Hadamard $B = 64$, active-set Hessian coordinate-descent code refinement) - Total Checkpoint Size:
15.36 GB - Runtime: Native single-simdgroup barrier-free Metal Walsh-Hadamard (
macaq_single_simd_wht) + Apple SiliconquantizedMMin Basalt
Measured Benchmarks (Apple M4 Pro 24 GB)
| Metric | Official BF16 (ornith-ai/Ornith-1.5-35B-A3B-MLX) |
Ornith-1.5-35B-A3B-MACaQ-3bit-MLX |
|---|---|---|
| Checkpoint Size | 69.32 GB |
15.36 GB (4.51× compression — fits on 24 GB Macs!) |
| Architecture | qwen3_5_moe (40 layers, 256 experts, A3B active) |
MACaQ 3-bit (B=64, α=0.75) + 4-bit embeddings/head |
| Quantized Modules | — | 250 dense + 120 stacked 256-expert MACaQSwitchLinear (773,928,363 flips) |
Integrity (basalt models inspect) |
intact=true |
intact=true, quantizationMode=macaq (5356.6s on M4 Pro) |
Raw Basalt PerformanceBenchmark & Live Generation Output (M4 Pro 24 GB)
Benchmark — model: /private/tmp/Ornith-1.5-35B-A3B-MACaQ-3bit-MLX
iterations: 3 tokens: 80 warmup: true
Loading model… done
Warmup… done
# TPS TTFT (s) Time (s)
──────────────────────────────────────────
1 77.35 0.128 1.17
2 78.23 0.135 1.16
3 78.12 0.116 1.14
TPS avg=77.899 min=77.349 max=78.228 σ=0.391 tok/s
TTFT avg=0.126 min=0.116 max=0.135 σ=0.008 s
GenTime avg=1.157 min=1.143 max=1.166 σ=0.010 s
Finish length 3
--- OUTPUT ---
Open source models have become increasingly important across software, AI, and various technical domains. Here's an overview of why they matter:
## Access and Affordability
Open source models are typically free or low-cost to access, which democratizes technology. This lowers barriers to entry for individuals, startups, and organizations that might not afford expensive proprietary alternatives. It allows wider participation in innovation regardless of budget
--- END OUTPUT ---
Memory (active): 14623 MB
Hardware profile updated: /Users/alexeiboklag/.gem/hardware_profile.json (measured ✓)
Quick Start with Basalt
# Verify 14 shards and MACaQ MoE quantization manifest
basalt models inspect goodmai/Ornith-1.5-35B-A3B-MACaQ-3bit-MLX
# Run 35B-A3B MoE natively on 24 GB+ Apple Silicon Macs!
basalt serve --model goodmai/Ornith-1.5-35B-A3B-MACaQ-3bit-MLX
Method Reference
Quantized using Basalt's native Swift/Metal implementation of Huawei CSL's MACaQ algorithm:
- Channel Balance: $p_j = \mathrm{std}(W_{:,j})^{0.75} / \mathrm{rms}(X_{:,j})^{0.25}$ equalizes weight and activation outlier dynamic ranges prior to block rotation.
- Sylvester Walsh-Hadamard Frame: $W' = (W ,\mathrm{diag}(p)^{-1}) H_{64}$, $H' = H_{64}^\top (\mathrm{diag}(p), H, \mathrm{diag}(p)) H_{64}$.
- Active-Set Hessian Flip Refinement: Monotonic greedy $\pm 1$ discrete grid coordinate descent minimizing $\mathrm{tr}(\Delta W, H', \Delta W^\top)$ (with Hessian-weighted E8M0 exponent offset screening ${+1, 0, -1}$ for
mxfp4).
- Downloads last month
- 120
Model size
26B params
Tensor type
U32
·
BF16 ·
Hardware compatibility
Log In to add your hardware
4-bit
Model tree for goodmai/Ornith-1.5-35B-A3B-MACaQ-3bit-MLX
Base model
ornith-ai/Ornith-1.5-35B-A3B-MLXPaper for goodmai/Ornith-1.5-35B-A3B-MACaQ-3bit-MLX
Paper • 2611.10649 • Published • 31