---
license: apache-2.0
base_model: huihui-ai/Huihui-Qwen3.8-27B-abliterated
base_model_relation: quantized
library_name: transformers
pipeline_tag: image-text-to-text
tags:
- qwen3.8
- qwen
- huihui
- awq
- w4a16
- multimodal
- v100
- sm70
- tensor-parallel
- abliterated
- uncensored
---

# Qwen3.8-27B Huihui W4A16 AWQ — 19.55 GB / 18.21 GiB

This is an activation-aware AWQ checkpoint of
[`huihui-ai/Huihui-Qwen3.8-27B-abliterated`](https://huggingface.co/huihui-ai/Huihui-Qwen3.8-27B-abliterated).
The current `main` branch follows upstream revision
`739e3c5b89849f6c238ce1e5b70008612ae42cdd`.

The language-model GEMMs are packed as asymmetric **W4A16** with group size
128. Vision, native MTP, the LM head, and linear-attention `in_proj_a` and
`in_proj_b` remain in source precision.

> [!IMPORTANT]
> The current revision passed complete artifact audit and native GPTQModel
> semantic generation on an RTX 3090. It has not yet replaced the live model
> for an exact 4x V100 runtime smoke. The older V100-validated checkpoint is
> permanently available at [`v1-d42ca897`](https://huggingface.co/leoncca/Qwen3.8-27B-Huihui-AWQ/tree/v1-d42ca897).

This is a community quantization, not an official Qwen or Huihui release.

## Versions

| Version | Mother revision | Validation | Download |
|---|---|---|---|
| `main` / `v2-739e3c5b` | `739e3c5b…` | Artifact audit + RTX 3090 native semantic smoke | [files](https://huggingface.co/leoncca/Qwen3.8-27B-Huihui-AWQ/tree/v2-739e3c5b) |
| `v1-d42ca897` | `d42ca897…` | Artifact audit + native smoke + 4x V100 TP4 runtime campaign | [files](https://huggingface.co/leoncca/Qwen3.8-27B-Huihui-AWQ/tree/v1-d42ca897) |

## What changed in the mother checkpoint

Compared with the mother revision used for v1, upstream narrowed the
abliteration scope. A tensor-level comparison found 28 matrices restored to
official Qwen values in layer indices 15, 16, and 52–63: 14 MLP
`down_proj`, 10 linear-attention `out_proj`, and 4 self-attention `o_proj`
matrices. The remaining 70 Huihui-modified matrices are in indices 17–51.
Vision, MTP, configuration, tokenizer, template, and weight index were
unchanged.

## Frozen source and quantizer

| Component | Repository | Revision |
|---|---|---|
| BF16 mother | [`huihui-ai/Huihui-Qwen3.8-27B-abliterated`](https://huggingface.co/huihui-ai/Huihui-Qwen3.8-27B-abliterated) | `739e3c5b89849f6c238ce1e5b70008612ae42cdd` |
| Quantizer | [`modelcloud/GPTQModel`](https://github.com/modelcloud/gptqmodel) 7.3.2 | `f6f1ab12fe8115f84c52253871ca039ac6c425ee` |

The pinned mother contains 18 weight shards, 1,199 indexed tensors, and
55,562,855,904 indexed tensor bytes. `SOURCE-MANIFEST-BF16-MOTHER.json`
records the pinned source files and hashes.

## Quantization recipe

| Item | Value |
|---|---|
| Method | Activation-aware AWQ, GEMM W4A16 |
| Weight bits | 4 |
| Group size | 128 |
| Zero point | Asymmetric, enabled |
| Activation order | Disabled (`desc_act=false`) |
| Packed dtype | INT32 `qweight` and `qzeros` |
| Scale dtype | BF16 |
| Quantized modules | 400 language-model Linear modules |
| Calibration | 320 frozen records; corpus SHA-256 `de8c3c856c55afd065ade04929659cf990cde7b8cf5fd846c6ee1867b3a13690` |
| Additional sensitivity exemptions | None |
| Retained scope | Vision, native MTP, LM head, linear-attention `in_proj_a` and `in_proj_b` |

The recipe uses joint `in_proj_qkv`/`in_proj_z` AWQ smoothing plus
zero-centered RMSNorm-aware BF16 compensation for the retained split-GDN
`in_proj_a`/`in_proj_b` projections.

## Validation of the current version

The v2 release passed these gates:

- 400/400 target modules packed;
- 400 `qweight`, 400 `qzeros`, and 400 `scales` tensors stored as INT32,
  INT32, and BF16;
- 799 retained source tensors present;
- 350 structurally protected tensors verified byte-for-byte;
- 96 retained split-GDN BF16 tensors compensated and audited;
- all targets used 202,750 calibration token samples;
- native GPTQModel 7.3.2 `BACKEND.TORCH` FP16 load and semantic smoke passed
  on an RTX 3090, including exact `NATIVE_AWQ_OK` output and a natural Chinese
  response;
- every release file is covered by `SHA256SUMS`.

The current v2 weights do **not** yet claim exact 1Cat-vLLM/SM70/V100 runtime
validation. The v1 tag retains the earlier V100-validated artifact and its
historical results.

## Download

Current version:

```bash
hf download leoncca/Qwen3.8-27B-Huihui-AWQ \
  --revision v2-739e3c5b \
  --local-dir Qwen3.8-27B-Huihui-AWQ-v2
```

Previous V100-validated version:

```bash
hf download leoncca/Qwen3.8-27B-Huihui-AWQ \
  --revision v1-d42ca897 \
  --local-dir Qwen3.8-27B-Huihui-AWQ-v1
```

Verify either download from inside its model directory:

```bash
sha256sum -c SHA256SUMS
```

Runtime flags depend on the inference engine. Do not assume that a loader
which supports older dense AWQ models already supports Qwen3.8 multimodal,
linear-attention, native-MTP, and SM70 execution.

## License and use

The included model files are released under Apache-2.0, following the source
repository. Review the upstream model card and license before use.

The Huihui mother checkpoint intentionally changes refusal behavior. This
checkpoint is not safety-tuned, and its outputs may be inaccurate, offensive,
or unsafe. Downstream users are responsible for evaluation, safeguards, and
legal compliance in their own deployment context.

## Acknowledgements

- [Qwen](https://huggingface.co/Qwen)
- [huihui-ai](https://huggingface.co/huihui-ai)
- [GPTQModel](https://github.com/modelcloud/gptqmodel)
- [1Cat-vLLM](https://github.com/1CatAI/1Cat-vLLM)