The newly added Huihui-Qwen3.8-27B-abliterated-GSQ-RCO is awesome

#19
by Abl1ttheworld - opened

@huihui-ai Hi just want to say thank you for the newly added Huihui-Qwen3.8-27B-abliterated-GSQ-RCO is amazing and seems to be even faster than the Unsloth Quant

my hardware is not the best only 8GB Vram and 4070

anyone know when GSQ-RCO iq4_xs comes out?
(or gsq-rco 14.4 gigs)
same for dw-ud iq4_xs, i really wanna test kld between both since theyre currently considered the front runners for quantization.

Edit:
Firstly thank you to the fellas who showed me the full pic about gsq-rco over 3.7bpw.
Now as for the ud-dw iq4_xs matter - I still don't understand the reason y'all never produced it, I mean ud-iq4_xs is pretty good and I can run it while only consuming 15.5 gigs of vram on an optimized system, that leaves 500MB avaliable which should be more then enough for the ud-dw variant since dw variants usually add about 300MB to file size.
The difference in performance is just to big to not feel like I'm missing out.
Note:
I know many pepole say leave 1 gig of room avaliable and I've got a few things to say about that:
Firstly unless your runing agentic workloads requiring massive eval and physical batches you don't need more then 500MB of free vram
Secondly I usually do leave the extra 500MB of space for eval and physical batches but ud-dw iq4_xs benchmarks have shown to genuinely rival q6_k and trutly optimized systems will still have around 300MB left for eval and physical batches which should be just about enough.
To conclude:
GIVE US UD-DW IQ4_XS PLEAS OH HUIHUI LORDS FROM ABOVE.
/
|
This part was satire i haven't gone insane fellas dw.

Abl1ttheworld changed discussion status to closed
Abl1ttheworld changed discussion status to open

@huihui-ai Not sure if you already saw this, but I thought it was interesting.
A group is claiming a free inference harness (ledger-based self-orchestration) lifts Qwen3.8-27B on LiveCodeBench Hard enough to match or slightly beat Claude Fable 5’s single-call score. They released the harness and the paper.

Reddit: https://www.reddit.com/r/Qwen_AI/s/jocmCf3ZLr
Paper: https://arxiv.org/abs/2608.26480
Code: https://github.com/slee-persis/GVS5H

Those of you who have these accounts can do a comparison test on huihui-ai/Huihui-Qwen3.8-27B-abliterated.

anyone know when GSQ-RCO iq4_xs comes out?
(or gsq-rco 14.4 gigs)
same for dw-ud iq4_xs, i really wanna test kld between both since theyre currently considered the front runners for quantization.

When the model quantization level is above 3.7 bpw, the GSQ RCO algorithm no longer provides additional gains.

IMG_20260911_113621

ok I tried it, and it failed to undertake any tool calls. It is good for chat, and does not code at all.

Abl1ttheworld changed discussion status to closed

Sign up or log in to comment