77.8 GB
63 files
Updated 2 months ago
Name
Size
2-bit
4-bit
6-bit
8-bit
.gitattributes1.75 kB
xet
LICENSE11.5 kB
xet
README.md9.14 kB
xet
README.md
OrcaRouter

Qwen3.8-27B-Uncensored-MLX

An abliterated (refusal-removed) MLX build of Qwen's Qwen3.8-27B — 2 / 4 / 6 / 8-bit for Apple Silicon

Website Model Catalog Model Card License MLX Quants Vision

One Gateway. Every Model. — Route Smarter · Ship Safer · Spend Less.

Website · Model Catalog · Model API · GitHub · Discord · X


An abliterated (refusal-removed) build of Qwen/Qwen3.8-27B — a 27B-parameter dense, hybrid-attention (Gated DeltaNet linear + full attention) native vision-language model with thinking control, tool-calling and an MTP head — quantized to MLX format for Apple Silicon. Four precisions are provided as subfolders: 2 / 4 / 6 / 8-bit (affine, group size 64). The vision tower, norms and conv layers are kept in BF16; only the language-model linear weights (including embed_tokens / lm_head) are quantized. Browse all models in the OrcaRouter Model Catalog. This model is deployed as API here.


⚠️ Disclaimer & risks — read before use

This model has had its safety alignment substantially removed via abliteration (orthogonalizing the refusal direction out of the residual stream). As a direct consequence:

  • It will comply with harmful, unethical, offensive, or illegal requests that the original Qwen3.8-27B would refuse. It has no meaningful built-in guardrails.
  • It is released strictly for legitimate research — interpretability, AI-safety and refusal-mechanism study, red-teaming, robustness evaluation, and controlled experiments.
  • You assume full responsibility and liability for how you use it and for everything it generates. Add your own safety, moderation and abuse-prevention layers before any deployment.
  • Use must comply with the Apache 2.0 License inherited from the base model, and all laws and regulations that apply to you.
  • The authors and uploaders accept no liability for any misuse or harm. Outputs do not reflect the views of the uploaders or of Qwen / Alibaba.

Specific risks

  • Harmful content on demand — it will produce instructions for malware, exploits, weapons, fraud and other illegal or dangerous activity when asked.
  • No refusals — jailbreak / safety probes "succeed" trivially; do not mistake this for a passing safety evaluation.
  • Confident falsehoods & bias — it can generate false, defamatory, biased or offensive text and present it authoritatively.
  • Expanded attack surface — preserved vision, tool-calling and 262K context mean these risks extend to image understanding and autonomous / agentic use.
  • Quantization noise — lower-bit builds (esp. 2-bit) add instability on top of the above; outputs can be degraded or nonsensical.

Intended use vs out of scope

  • Intended: AI-safety and interpretability research, refusal-mechanism study, red-teaming, guardrail and robustness evaluation, controlled academic experiments.
  • Out of scope: any deployment to end users, minors, or production without your own moderation / safety layer; any unlawful, harmful, or rights-infringing use.

By downloading or using this model you acknowledge and accept the above.


Available quantizations

Folder Bits/weight Size Shards Min Mac RAM Quality vs BF16 source
8-bit/ 8.627 ~27.5 GB 6 32 GB Near-lossless — recommended for quality
6-bit/ 6.661 ~22 GB 5 24–32 GB Excellent — strong quality/size balance
4-bit/ 4.695 ~15 GB 3 24 GB Very good — recommended default
2-bit/ 2.729 ~8.7 GB 2 16 GB ⚠️ Severely degraded — archival only

2-bit warning: at 27B, 2-bit quantization collapses generation quality (repetition loops, garbled output). It is included only as an extreme-compression archive; do not use it for real work — prefer 4-bit or higher.


Verification & test results

All builds were quantized from the same abliterated BF16 source and verified numerically (dequantized weights vs. source) plus tested by generation on GPU.

Precision Numerical fidelity (cosine) Text / Chinese / Code Refusal probes Vision
8-bit cos 0.9997 ✅ ✅ 0 refusals ✅
6-bit cos 0.9996 ✅ ✅ 0 refusals ✅
4-bit cos 0.996 ✅ ✅ 0 refusals ✅
2-bit cos 0.92 ⚠️ breaks down ⚠️ garbled (not refusal) partial
  • Uncensored preserved: red-team probes (exploit walkthrough, controversial argument) return substantive content with zero refusals on 4 / 6 / 8-bit.
  • Multimodal preserved: shapes, colors, position, background and text in a probe image are described correctly on 4 / 6 / 8-bit.
  • Speed: ~32–37 tok/s steady-state on a single H200 (MLX CUDA backend). MLX's native target is Apple Silicon (Metal).

Note: on 6-bit, mlx's offline mx.dequantize mis-unpacks these weights (a library edge case), so correctness is verified by clean generation — inference is unaffected.


Usage (mlx-vlm, Apple Silicon)

pip install -U mlx-vlm    # needs mlx-vlm >= 0.6.13, mlx >= 0.32

# download one precision (e.g. 4-bit) from the subfolder
hf download orcarouter/Qwen3.8-27B-Uncensored-MLX --include "4-bit/*" \
    --local-dir ./Qwen3.8-27B-Uncensored-MLX

# text
python -m mlx_vlm generate \
    --model ./Qwen3.8-27B-Uncensored-MLX/4-bit \
    --prompt "Explain quantum entanglement in one sentence." --max-tokens 256

# vision (image + text)
python -m mlx_vlm generate \
    --model ./Qwen3.8-27B-Uncensored-MLX/4-bit \
    --image path/to/image.png \
    --prompt "Describe this image." --max-tokens 256

# OpenAI-compatible server
python -m mlx_vlm server --model ./Qwen3.8-27B-Uncensored-MLX/4-bit --port 8080

On Apple Silicon the Metal backend is used automatically — no CUDA setup needed. (On a Linux CUDA backend, vision requires MLX_CUDA_USE_CUDNN_SDPA=0; this does not apply on macOS.)


Model details

Base model Qwen/Qwen3.8-27B
Architecture Qwen3_5ForConditionalGeneration — 64 layers, hidden 5120, hybrid Gated DeltaNet (48 linear + 16 full attention, interval 4), native VL tower
Modification Abliteration (refusal-direction removal), then MLX affine quantization
Quantization MLX affine, group size 64, per-precision 2 / 4 / 6 / 8-bit
Kept in BF16 vision tower, all norms, linear-attention conv1d
Quantized language-model linear layers incl. embed_tokens and lm_head
Context 262,144 tokens

Mirror note

This repository is a mirror of orcarouter/Qwen3.8-27B-Uncensored-MLX (Apache 2.0), uploaded by Zomi AI Labs for internal deployment. The model files are byte-identical to the source repository. The only additions are this LICENSE file (Apache 2.0 full text, which the source repo declares via metadata but does not ship) and this note. All credit for the model and its card belongs to OrcaRouter and the Qwen team.

Total size
77.8 GB
Files
63
Last updated
Aug 19
Pre-warmed CDN
US EU US EU

Contributors