Configuration Parsing Warning:In config.json: "quantization_config.bits" must be an integer

Jackrong / Qwopus3.6-27B-v2

QUANTIZED BY: UnstableLlama

Information

exl3 quantizations of Jackrong's Qwopus3.6-27B-v2 via exllamav3.
repo generated automatically with ezexl3.

Repo Data

REVISION	GiB	KL DIV	PPL
2.50bpw (h4)	11.16	0.1822	7.4460
2.90bpw (h4)	12.29	0.1421	7.2710
3.08bpw	12.90	0.0749	6.9450
4.15bpw	15.93	0.0416	6.9000
6.00bpw	21.17	0.0137	6.8480
bf16	51.75	0.0000	6.8370

(h4) = head layer quantized to 4bpw; all other revisions use a 6bpw head.

CLI Download

hf download UnstableLlama/Qwopus3.6-27B-v2-exl3-2.50bpw-h4 --local-dir ./Qwopus3.6-27B-v2-exl3-2.50bpw-h4

Safetensors

Model size

7B params

Tensor type

BF16

F16

I16

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for UnstableLlama/Qwopus3.6-27B-v2-exl3-2.50bpw-h4

Base model

Quantized

(59)

this model