Qwen3.8-Flash-Next-Abliterated-Sushi-3bpw

Experimental Sushi-format quantization made directly from the BF16 weights of orcarouter/Qwen3.8-Flash-Next-Uncensored, revision e096800036ec20da7e2442dcd4044a004d4e99fa. The 3-bit sibling of Qwen3.8-Flash-Next-Abliterated-Sushi-2bpw. The abliteration comes from that upstream checkpoint; this conversion adds no further abliteration or fine-tuning. Refusal behavior has not been independently characterized here.

Format and conversion

• Routed experts: uniform EXL3 K3, MCG codebook, window15, including MTP experts, calibrated from BF16 (64 rows x 1024 tokens).
• Everything else is byte-identical to the 2bpw pack (same sha256): affine8 dense trunk, vision tower, MTP trunk, tokenizer, and the affine4/group32 n-gram sidecar ngram_table.bin. Only the expert shards and config.json (expert_quant.k) differ.
• Converter based on ExLlamaV3 151539c77abc7ab7425d30da7a4e8e3c5c154e7b, with an isolated MCG-window15 encoder adaptation; its K2 and K3 output was checked bit-exact against Sushi v1.2.1's own EXL3 decoder before conversion.

This is a Sushi pack, not a general ExLlamaV3 or Transformers checkpoint. 3bpw refers to the routed-expert code rate. No lossless/BF16-equivalent quality claim is made. The official Sushi-3bpw uses a private converter at window14; this pack does not reproduce that recipe, and no KLD measurement has been made for it.

Running

The complete download is about 79.22 GiB (including the 29.8 GiB disk-backed n-gram sidecar). If you already have the 2bpw pack, only the expert shards differ.

hf download sanasol2008/Qwen3.8-Flash-Next-Abliterated-Sushi-3bpw \
  --local-dir ~/.sushi/models/Qwen3.8-Flash-Next-Abliterated-Sushi-3bpw

Expert weights are about 1.5x the 2bpw pack. On a 48 GiB Mac they are streamed from SSD (--ssd-budget-gb), so decode is slower than 2bpw there; 64 GiB or more keeps much more of the pack resident. The image check below passed on the public Sushi v1.2.1 engine with --vision under a 22 GiB streaming budget; the patched engine from the 2bpw card was not needed for it.

sushi serve --model ~/.sushi/models/Qwen3.8-Flash-Next-Abliterated-Sushi-3bpw \
  --host 127.0.0.1 --port 12345 --ssd-budget-gb 22 --no-mtp --vision \
  --kv-quant 8 --ctx-size 262144 --prefill-chunk 2048 --metrics \
  --max-tokens 8192 --prefix-cache-entries 2 --prefix-cache-mem 1GB \
  --prefix-cache-disk 10GB --temp 1

The 256k context is a configured ceiling, not a measured full-context guarantee.

Converter-reported calibration error on decoder layer 0, the one layer measured in both runs: relative Frobenius error 0.029 / SQNR 43.4 dB at K3 versus 0.064 / 38.0 dB for the 2bpw pack.

Validation

Tested on Apple M3 Max / 48 GiB unified memory with the public Sushi v1.2.1 engine (--ssd-budget-gb 22 --no-mtp --kv-quant 8).

Functional checks (thinking off, temperature 0): arithmetic, a Russian instruction, an image OCR check that transcribed all three fields including numbers, and a tool call returning get_weather with city=Belgrade all passed. These establish basic functionality, not model-wide quality, BF16 equivalence or a measured refusal rate.

Controlled decode comparison: a fresh server per pack, 16k context, prefill chunk 512, prefix cache off, the same 40-token prompt, 256 generated tokens, nothing else running.

Pack	Cold decode	Warm decode, mean of runs 2–3
Abliterated 2bpw	18.36 tok/s	28.91 tok/s
This 3bpw pack	11.88 tok/s	12.88 tok/s

Short-prompt expert-cache measurements on a 48 GiB machine that cannot hold the 3bpw experts resident; they are not long-context agent throughput, and a Mac with more memory will see a different ratio. MTP, video and long-context behavior were not validated.

License and attribution

The upstream LICENSE is included verbatim: Qwen Community License 1.0. The upstream model card labels Apache 2.0, but its actual bundled license differs; this repository preserves the bundled license rather than asserting a new Apache license. Credit to Qwen for the base model, OrcaRouter for the BF16 derivative, turboderp-org for ExLlamaV3, and beamivalice for Sushi.