How to use from
Docker Model Runner
docker model run hf.co/nace-ai/Qwen3.8-27B-GGUF:Q4_K_XL
Quick Links

Qwen3.8-27B ยท Nace GGUF

GGUF file of Qwen/Qwen3.8-27B for llama.cpp, LM Studio and Ollama. Q4_K_XL (17.29 GB, mean KL 0.005) is as close to the BF16 model as Unsloth UD-Q4_K_XL (17.30 GB, 0.005), a file of the same size or larger, in 512-token text, and closer in 128K-token text (KL gap โˆ’0.0002, 95% interval โˆ’0.0002 to โˆ’0.0001), counting only the model's own reply tokens in chat text.

Built from Unsloth's Qwen3.8-27B UD-Q4_K_XL GGUF (unsloth/Qwen3.8-27B-GGUF, Apache-2.0) with our own output-layer and FFN precision changes; base model Qwen/Qwen3.8-27B (Apache-2.0).

Mean KL from BF16 at 128K context, averaged over prose, chat and code, against language-model size: the Nace files on this card, highlighted, and GGUF and EXL3 files of Qwen3.8-27B from other publishers in the same size range, each at its measured size and mean KL. Chat counts only the model's own reply tokens.

The chart is at 128K context. Mean KL values in the text and tables are at 512 tokens unless they say 128K.

Pick a file

File (tag) LM GB Mean KL All tokens p99.9 KL Top-1 accuracy % Baseline (GB, mean KL) KL gap (95% CI)
Q4_K_XL 17.29 0.005 0.012 0.172 97.4 Unsloth UD-Q4_K_XL (17.30, 0.005) โˆ’0.0001 (โˆ’0.0003 to โˆ’0.0000)

KL is the divergence from the BF16 model per token, so lower is closer. The gap is ours minus the baseline's mean KL; a negative gap means ours is closer, and the interval is a paired 95% bootstrap over windows. Top-1 accuracy is the share of tokens where the file's most likely next token is the same as the BF16 model's.

Mean KL from BF16 at 512 tokens against language-model size: the Nace files on this card, highlighted, and GGUF and EXL3 files of Qwen3.8-27B from other publishers in the same size range, each at its measured size and mean KL

File names mark the size tier, not a single llama.cpp type: each file mixes tensor types from 3.4 to 8.5 bits per weight, all in stock llama.cpp. By share of file bytes, Q4_K_XL is 52% Q5_K, 21% IQ4_XS, 13% Q6_K, 10% Q4_K and 4% other types.

At 512 tokens, Q4_K_XL has a lower mean KL than 5 of the 6 GGUF files from other publishers within 0.98 to 1.06 times its size: mradermacher i1-Q4_1 (17.07 GB), bartowski IQ4_NL (17.19 GB), bartowski Q4_K_M (17.19 GB), Unsloth Q4_1 (17.24 GB) and bartowski Q4_1 (17.58 GB).

Counting every token, Q4_K_XL is also closer than Unsloth UD-Q4_K_XL at 128K (โˆ’0.003, โˆ’0.006 to โˆ’0.001).

Test text: English Wikipedia articles, chat and tool-call data, and GitHub code, all created after 2026-09-20; separate from the text used to fit and develop the files; used for release evaluation. 64 windows, 32,704 tokens, the same llama.cpp build (0a94a3f2) for every GGUF file. Size is language-model bytes, vision and MTP excluded, the same rule for every file. In chat and tool-call text, KL on this card counts only the model's own reply tokens (assistant turns), at 512 and 128K tokens; the All tokens values beside it count every token, user and tool turns included. Prose and code count every token.

128K context

KL from BF16 on 128K-token windows, same reference and same build.

prose

File LM GB Mean KL p99.9 KL Top-1 accuracy %
Q4_K_XL 17.29 0.005 0.218 97.1

chat and tool calls

File LM GB Mean KL All tokens p99.9 KL Top-1 accuracy %
Q4_K_XL 17.29 0.005 0.707 0.443 97.8

code

File LM GB Mean KL p99.9 KL Top-1 accuracy %
Q4_K_XL 17.29 0.004 0.162 98.2

At 128K, over all three kinds of text, Q4_K_XL has a lower mean KL than all 6 GGUF files from other publishers within 0.98 to 1.06 times its size that we scored at that length: mradermacher i1-Q4_1 (17.07 GB), bartowski IQ4_NL (17.19 GB), bartowski Q4_K_M (17.19 GB), Unsloth Q4_1 (17.24 GB), Unsloth UD-Q4_K_XL (17.30 GB) and bartowski Q4_1 (17.58 GB).

Task check

Accuracy against the baseline file named in the table above, scored with llama.cpp 0a94a3f2 llama-perplexity (questions: MMLU 13,943, ARC-Challenge 299, Winogrande 1,267). Each cell gives ours against the baseline: better or worse only when the paired 95% bootstrap interval over questions excludes 0, then the gap in percentage points with that interval. Each answer is scored by its likelihood, with no chat template or few-shot prompt, so the accuracies are not comparable to published scores: compare the files with each other, not with a leaderboard. All three ran with -np 16: one ARC-Challenge question has 5 answers, and the default of 4 parallel sequences stops the run there.

File (tag) Baseline MMLU ARC-Challenge Winogrande
Q4_K_XL Unsloth UD-Q4_K_XL about the same, โˆ’0.0 (โˆ’0.2 to +0.1) about the same, โˆ’1.0 (โˆ’2.7 to +0.3) about the same, +0.2 (โˆ’0.4 to +0.8)

Quickstart

The examples use Q4_K_XL; swap in any file from the table.

llama.cpp

Install a current llama.cpp, then start a local chat server. It downloads the file on first use:

llama-server -hf nace-ai/Qwen3.8-27B-GGUF:Q4_K_XL -c 8192 --port 8080

Open http://127.0.0.1:8080 to chat, or call the OpenAI-compatible API:

curl -s http://127.0.0.1:8080/v1/chat/completions -H "Content-Type: application/json" -d '{
  "messages": [{"role": "user", "content": "What is 17 * 23?"}]
}'

The model thinks before it answers, and the API returns the thinking in reasoning_content. A small max_tokens can run out inside the thinking and leave the answer empty, so leave the cap off or turn thinking off: add "chat_template_kwargs": {"enable_thinking": false} to the request, or --reasoning off to llama-cli.

LM Studio

Update LM Studio and its llama.cpp runtime. Search for nace-ai/Qwen3.8-27B-GGUF under Discover, download the Q4_K_XL file, and load it in Chat. If you already have the file, add it with lms import --user-repo nace-ai/Qwen3.8-27B-GGUF Qwen3.8-27B-Q4_K_XL.gguf.

Ollama

ollama run hf.co/nace-ai/Qwen3.8-27B-GGUF:Q4_K_XL

Add --think=false to get the answer without the thinking.

Kapitsa

On a Mac with Kapitsa, download the file (hf comes with pip install -U huggingface_hub) and start it. The first start converts the file into Kapitsa's own format; then open http://127.0.0.1:8000 to chat, or send the curl request above to port 8000:

hf download nace-ai/Qwen3.8-27B-GGUF Qwen3.8-27B-Q4_K_XL.gguf --local-dir .
kapitsa serve --model ./Qwen3.8-27B-Q4_K_XL.gguf

Kapitsa stores the tensor types it has no kernels for at a higher bit width, so the converted model is larger than the file, and the measurements on this card, taken in llama.cpp, don't apply to it exactly.

Files

File Download
Qwen3.8-27B-Q4_K_XL.gguf 17.65 GB
mmproj-bf16.gguf 0.93 GB

Download sizes, not memory use: the context and the engine need more. mmproj-bf16.gguf is the vision encoder, for image input.

Original model and license: Qwen/Qwen3.8-27B.

Downloads last month
1,086
GGUF
Model size
27B params
Architecture
qwen35
Hardware compatibility
Log In to add your hardware

4-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for nace-ai/Qwen3.8-27B-GGUF

Base model

Qwen/Qwen3.8-27B
Quantized
(1,873)
this model

Collection including nace-ai/Qwen3.8-27B-GGUF