mlx-community/Qwen3.8-Flash-Next-Uncensored-oQ5e-mtp

Qwen3.8-Flash-Next-Uncensored (MLX, oQ5e, MTP)

Calibrated 5-bit mixed-precision MLX quantization of orcarouter/Qwen3.8-Flash-Next-Uncensored, the BF16 abliterated build of Qwen/Qwen3.8-Flash-Next. Produced with oMLX oQe (oQ level 5 with imatrix weighting). 5.72 bits per weight effective, 128.54 GB on disk (119.7 GiB) across 25 safetensors shards. Everything from the source is kept: the routed experts, the 51B-parameter n-gram embedding table (quantized, not left in BF16), the Lightning MTP head, and the full vision + video tower.

This is the build that runs the model's full 262,144-token context on a 128 GB Apple Silicon Mac. A real 250,073-token request completed in 214 s on a MacBook Pro M5 Max with Lightning MTP and TurboQuant KV on (numbers below).

The source model has had its safety alignment removed by abliteration. It will comply with requests the original Qwen3.8-Flash-Next refuses. Released for research, red-teaming and interpretability work. Read the source model card's disclaimer before use and add your own moderation if you deploy it. You are responsible for what you do with it.

Model details

Quantized by	timteh673
Model type	Qwen4ExpForConditionalGeneration (qwen4_exp), sparse MoE vision-language model with thinking mode, tool calling and an MTP head
Source	orcarouter/Qwen3.8-Flash-Next-Uncensored (BF16, 360 GB)
Upstream base	Qwen/Qwen3.8-Flash-Next
Parameters	~180B total as stored (125B transformer + 51.2B n-gram table), ~6B active per token
Architecture	48 layers, hidden 2560; hybrid attention: 36 Gated-DeltaNet linear layers + 12 full-attention layers (interval 4, 24 query heads, 2 KV heads, head dim 256); 512 routed experts, top-10, plus a shared expert; 3-gram PLE embedding table; QSA sparse-attention indexer (budget 2048); 1-layer MTP head
Context	262,144 tokens native
Languages	English, Chinese (multilingual capable)
License	Qwen Community License 1.0, inherited from the upstream model (LICENSE in this repo). The OrcaRouter source card labels itself Apache-2.0; the upstream Qwen repo and the LICENSE file shipped with the weights say Qwen Community License 1.0, so that is what is declared here.
Quantization tool	oMLX v0.7.0.dev2, quantize_oq_streaming (oQe)
Format	Standard MLX affine safetensors, no custom kernels or loader

What is in this repo

Files	Size
model-00001-of-00025.safetensors ... model-00025-of-00025.safetensors	128.54 GB (119.72 GiB)
config.json (with the full per-tensor quantization map), model.safetensors.index.json, oq_imatrix_report.json	0.7 MB
tokenizer, chat template, preprocessor config, LICENSE	23 MB

The size shown by oMLX's estimator and in its upload dialog (119.7 "GB") is GiB. The decimal figure is 128.5 GB.

Quantization layout

oQ is sensitivity-driven mixed precision: every layer's quantization error is measured on calibration data, and bits go where the data says they matter. The "e" (oQe) adds an importance matrix so that inside each quantization group the clipping favors the weights that carry activation energy. Output is plain affine quantization that any MLX runtime can read.

Component	Precision	Bytes
Routed experts (512 per layer, 48 layers)	5-bit, group 64, imatrix-weighted	83.05 GB
PLE n-gram embedding table (51.2B params)	5-bit, group 32	38.40 GB
Attention, GDN, shared experts, routers, norms	mixed 5 / 6 / 8-bit by measured sensitivity (routers 8-bit)	3.04 GB
MTP head (mtp.*, 76 tensors)	6-bit, group 64	1.81 GB
Embeddings and lm_head	8-bit	1.35 GB
Vision + video tower (333 tensors)	BF16, untouched	0.90 GB
Total	5.72 bpw effective	128.54 GB

Bit histogram over the 871 quantized text tensors: 209 at 8-bit, 498 at 6-bit, 164 at 5-bit. 128 n-gram shards at 5-bit. The exact per-tensor map is in config.json under quantization.

How it was made

Built on a MacBook Pro M5 Max (128 GB) with oMLX 0.7.0.dev2 from the 360 GB BF16 source, streamed tensor by tensor (the full model never sits in memory).

• Sensitivity pass: measured on a uniform 4-bit MLX build of the same model (the orcarouter 4-bit release), since the BF16 source is 2.8x the machine's memory. 48 layers scored, 128 samples x 256 tokens.
• Importance matrix: collected on a temporary uniform 4-bit proxy that oMLX builds and deletes itself. Calibration set oqe_code_multilingual (code, English, Korean, Chinese, Japanese, tool calls, reasoning). The adaptive collector ran to its maximum of 1,024 samples x 512 tokens: 937 tensor entries, 6,144 layer forwards, 128 MTP-head forwards. Expert coverage 75,243 of 75,264 (21 experts were never routed on the calibration set and fall back to plain oQ). Three tensors have no imatrix entry by design (lm_head and the two hyper-connection mixer projections) and are quantized with plain oQ. Full report in oq_imatrix_report.json.
• Time: about 80 minutes end to end, of which 46 minutes was imatrix collection. The imatrix cache is keyed by source, calibration set and sample counts, not by bit width, so the same matrix was reused unchanged for the sibling oQ6e build.

Requirements

Apple Silicon only. Tested with oMLX 0.7.0.dev2, which ships its own qwen4_exp implementation (Lightning MTP, the PLE table on SSD, TurboQuant KV). The checkpoint is standard MLX safetensors, but stock mlx-lm / mlx-vlm need qwen4_exp support to load it and were not tested here.

Unified memory	What you get
128 GB	Runs, full 262k context. oMLX keeps the 38 GB n-gram table on SSD (mmap) and holds about 84 GiB resident. This is the configuration measured below.
192 GB and up	Everything resident, including the n-gram table (~135 GB). Faster decode, since n-gram rows are no longer gathered from disk.
96 GB or less	Does not fit. Use a smaller oQ level.

Disk: 129 GB for the model, plus room for oMLX's KV cache spillover.

Running it with oMLX

Drop the folder into ~/.omlx/models/ (or download it there) and the model is discovered on the next rescan. Settings used for the numbers below, on a 128 GB M5 Max:

Setting	Value	Why
SSD N-gram Offload (Qwen4)	on	oMLX forces this on automatically whenever the resident estimate exceeds its memory ceiling. Required on 128 GB.
Lightning MTP	on, 3 draft tokens	88 percent draft acceptance measured. Roughly doubles tokens per decode cycle.
TurboQuant KV cache	on, 4-bit, skip last layer	KV for the full 262k window drops from 6.4 GB to 2.0 GB. Costs about 5 to 8 percent prefill speed. Optional on 128 GB, useful headroom.
Context window	262144	Native maximum.
Thinking	on, preserved	Vendor default.
Sampling (thinking)	temperature 1.0, top_p 0.95, top_k 20, presence 0	Vendor defaults. Instruct mode: 0.7 / 0.8 / 20 / presence 1.5.
Memory guard	Aggressive tier (or custom around 120 GB)	Leaves the prefill sizer room.

Any OpenAI-compatible client then works against http://localhost:8000/v1. Vision input goes in user messages as image_url parts.

Measured performance

MacBook Pro M5 Max, 128 GB, 40-core GPU, oMLX 0.7.0.dev2, n-gram table on SSD, Lightning MTP with 3 drafts, TurboQuant KV 4-bit unless noted. Single request.

Prefill (prompt processing)

Prompt tokens	Prefill tok/s	Time to first token
4,096	1,090 to 1,153	3.6 to 3.8 s
32,768	1,296	25 s
65,536	1,228 to 1,340	49 to 53 s
131,072	1,203	109 s
200,000	1,081 to 1,236	162 to 185 s
250,073 (real chat request)	1,165	214 s

The 250k request ran with no memory-guard throttling; every prefill chunk stayed at the full 2,048-token step. Without TurboQuant KV the prefill numbers are 5 to 8 percent higher.

Decode (generation)

Context at the time	Decode tok/s	MTP acceptance
Short prompt, 900-token answer	59	87.7 percent, 2.56 tokens per cycle
250,073-token prompt	about 30	88.9 percent

Decode is bound by the n-gram gathers from SSD in this configuration. On a machine where the table is resident, expect materially faster decode; oMLX's own estimate for the SSD path is about 2.5x slower than resident.

Compared with the sibling builds

Build	Disk	Resident on 128 GB (table on SSD)	262k context on 128 GB
oQ3.5e-mtp (not published)	89.8 GB	~70 GiB, or fully resident at ~94 GiB	yes
oQ5e-mtp (this repo)	128.5 GB	~84 GiB	yes, measured
oQ6e-mtp	150.5 GB	~101 GiB with MTP off	no, about 131k with MTP off

oQ5e is the highest level that keeps the full context window on a 128 GB machine. oQ6e trades that for 6-bit experts; see its card for the exact limits.

Limitations

• Apple Silicon and MLX only.
• Safety alignment removed by the source model's abliteration. No guardrails.
• 21 of 75,264 routed experts and 3 shell tensors were quantized without imatrix weighting (plain oQ), as noted above.
• The n-gram table runs from SSD on 128 GB machines, which caps decode speed; this is a property of the machine, not the quantization.
• No formal perplexity or benchmark evaluation was run on this quant. The measurements above are throughput and a correctness spot check (long-context retrieval answer, tool calls and vision were exercised through oMLX).

Credits

• Base model: Qwen (Qwen3.8-Flash-Next).
• Abliterated source: OrcaRouter.
• Quantization tooling: oMLX by jundot, oQ specification.

@misc{omlx,
  title  = {oMLX: LLM inference optimized for Apple Silicon},
  author = {jundot},
  year   = {2026},
  url    = {https://github.com/jundot/omlx}
}