Instructions to use nace-ai/Qwen3.8-27B-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use nace-ai/Qwen3.8-27B-GGUF with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf nace-ai/Qwen3.8-27B-GGUF:Q4_K_XL # Run inference directly in the terminal: llama cli -hf nace-ai/Qwen3.8-27B-GGUF:Q4_K_XL
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf nace-ai/Qwen3.8-27B-GGUF:Q4_K_XL # Run inference directly in the terminal: llama cli -hf nace-ai/Qwen3.8-27B-GGUF:Q4_K_XL
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf nace-ai/Qwen3.8-27B-GGUF:Q4_K_XL # Run inference directly in the terminal: ./build/bin/llama cli -hf nace-ai/Qwen3.8-27B-GGUF:Q4_K_XL
Use Docker
docker model run hf.co/nace-ai/Qwen3.8-27B-GGUF:Q4_K_XL
- Llama new
- LM Studio
- Jan
- vLLM
How to use nace-ai/Qwen3.8-27B-GGUF with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "nace-ai/Qwen3.8-27B-GGUF" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "nace-ai/Qwen3.8-27B-GGUF", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/nace-ai/Qwen3.8-27B-GGUF:Q4_K_XL
- Ollama
How to use nace-ai/Qwen3.8-27B-GGUF with Ollama:
ollama run hf.co/nace-ai/Qwen3.8-27B-GGUF:Q4_K_XL
- Unsloth Desktop
- Pi
How to use nace-ai/Qwen3.8-27B-GGUF with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf nace-ai/Qwen3.8-27B-GGUF:Q4_K_XL
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:9931/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "nace-ai/Qwen3.8-27B-GGUF:Q4_K_XL" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use nace-ai/Qwen3.8-27B-GGUF with Docker Model Runner:
docker model run hf.co/nace-ai/Qwen3.8-27B-GGUF:Q4_K_XL
- Lemonade
How to use nace-ai/Qwen3.8-27B-GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull nace-ai/Qwen3.8-27B-GGUF:Q4_K_XL
Run and chat with the model
lemonade run user.Qwen3.8-27B-GGUF-Q4_K_XL
List all available models
lemonade list
- Hermes Agent
How to use nace-ai/Qwen3.8-27B-GGUF with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf nace-ai/Qwen3.8-27B-GGUF:Q4_K_XL
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:9931/v1 hermes config set model.default nace-ai/Qwen3.8-27B-GGUF:Q4_K_XL
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use nace-ai/Qwen3.8-27B-GGUF with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf nace-ai/Qwen3.8-27B-GGUF:Q4_K_XL
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:9931/v1 \ --custom-model-id "nace-ai/Qwen3.8-27B-GGUF:Q4_K_XL" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
Qwen3.8-27B ยท Nace GGUF
GGUF file of Qwen/Qwen3.8-27B for llama.cpp, LM Studio and Ollama. Q4_K_XL (17.29 GB, mean KL 0.005) is as close to the BF16 model as Unsloth UD-Q4_K_XL (17.30 GB, 0.005), a file of the same size or larger, in 512-token text, and closer in 128K-token text (KL gap โ0.0002, 95% interval โ0.0002 to โ0.0001), counting only the model's own reply tokens in chat text.
Built from Unsloth's Qwen3.8-27B UD-Q4_K_XL GGUF (unsloth/Qwen3.8-27B-GGUF, Apache-2.0) with our own output-layer and FFN precision changes; base model Qwen/Qwen3.8-27B (Apache-2.0).
The chart is at 128K context. Mean KL values in the text and tables are at 512 tokens unless they say 128K.
Pick a file
| File (tag) | LM GB | Mean KL | All tokens | p99.9 KL | Top-1 accuracy % | Baseline (GB, mean KL) | KL gap (95% CI) |
|---|---|---|---|---|---|---|---|
Q4_K_XL |
17.29 | 0.005 | 0.012 | 0.172 | 97.4 | Unsloth UD-Q4_K_XL (17.30, 0.005) | โ0.0001 (โ0.0003 to โ0.0000) |
KL is the divergence from the BF16 model per token, so lower is closer. The gap is ours minus the baseline's mean KL; a negative gap means ours is closer, and the interval is a paired 95% bootstrap over windows. Top-1 accuracy is the share of tokens where the file's most likely next token is the same as the BF16 model's.
File names mark the size tier, not a single llama.cpp type: each file mixes tensor types from 3.4 to 8.5 bits per weight, all in stock llama.cpp. By share of file bytes, Q4_K_XL is 52% Q5_K, 21% IQ4_XS, 13% Q6_K, 10% Q4_K and 4% other types.
At 512 tokens, Q4_K_XL has a lower mean KL than 5 of the 6 GGUF files from other publishers within 0.98 to 1.06 times its size: mradermacher i1-Q4_1 (17.07 GB), bartowski IQ4_NL (17.19 GB), bartowski Q4_K_M (17.19 GB), Unsloth Q4_1 (17.24 GB) and bartowski Q4_1 (17.58 GB).
Counting every token, Q4_K_XL is also closer than Unsloth UD-Q4_K_XL at 128K (โ0.003, โ0.006 to โ0.001).
Test text: English Wikipedia articles, chat and tool-call data, and GitHub code, all created after 2026-09-20; separate from the text used to fit and develop the files; used for release evaluation. 64 windows, 32,704 tokens, the same llama.cpp build (0a94a3f2) for every GGUF file. Size is language-model bytes, vision and MTP excluded, the same rule for every file. In chat and tool-call text, KL on this card counts only the model's own reply tokens (assistant turns), at 512 and 128K tokens; the All tokens values beside it count every token, user and tool turns included. Prose and code count every token.
128K context
KL from BF16 on 128K-token windows, same reference and same build.
prose
| File | LM GB | Mean KL | p99.9 KL | Top-1 accuracy % |
|---|---|---|---|---|
| Q4_K_XL | 17.29 | 0.005 | 0.218 | 97.1 |
chat and tool calls
| File | LM GB | Mean KL | All tokens | p99.9 KL | Top-1 accuracy % |
|---|---|---|---|---|---|
| Q4_K_XL | 17.29 | 0.005 | 0.707 | 0.443 | 97.8 |
code
| File | LM GB | Mean KL | p99.9 KL | Top-1 accuracy % |
|---|---|---|---|---|
| Q4_K_XL | 17.29 | 0.004 | 0.162 | 98.2 |
At 128K, over all three kinds of text, Q4_K_XL has a lower mean KL than all 6 GGUF files from other publishers within 0.98 to 1.06 times its size that we scored at that length: mradermacher i1-Q4_1 (17.07 GB), bartowski IQ4_NL (17.19 GB), bartowski Q4_K_M (17.19 GB), Unsloth Q4_1 (17.24 GB), Unsloth UD-Q4_K_XL (17.30 GB) and bartowski Q4_1 (17.58 GB).
Task check
Accuracy against the baseline file named in the table above, scored with llama.cpp 0a94a3f2 llama-perplexity (questions: MMLU 13,943, ARC-Challenge 299, Winogrande 1,267). Each cell gives ours against the baseline: better or worse only when the paired 95% bootstrap interval over questions excludes 0, then the gap in percentage points with that interval. Each answer is scored by its likelihood, with no chat template or few-shot prompt, so the accuracies are not comparable to published scores: compare the files with each other, not with a leaderboard. All three ran with -np 16: one ARC-Challenge question has 5 answers, and the default of 4 parallel sequences stops the run there.
| File (tag) | Baseline | MMLU | ARC-Challenge | Winogrande |
|---|---|---|---|---|
Q4_K_XL |
Unsloth UD-Q4_K_XL | about the same, โ0.0 (โ0.2 to +0.1) | about the same, โ1.0 (โ2.7 to +0.3) | about the same, +0.2 (โ0.4 to +0.8) |
Quickstart
The examples use Q4_K_XL; swap in any file from the table.
llama.cpp
Install a current llama.cpp, then start a local chat server. It downloads the file on first use:
llama-server -hf nace-ai/Qwen3.8-27B-GGUF:Q4_K_XL -c 8192 --port 8080
Open http://127.0.0.1:8080 to chat, or call the OpenAI-compatible API:
curl -s http://127.0.0.1:8080/v1/chat/completions -H "Content-Type: application/json" -d '{
"messages": [{"role": "user", "content": "What is 17 * 23?"}]
}'
The model thinks before it answers, and the API returns the thinking in reasoning_content. A small max_tokens can run out inside the thinking and leave the answer empty, so leave the cap off or turn thinking off: add "chat_template_kwargs": {"enable_thinking": false} to the request, or --reasoning off to llama-cli.
LM Studio
Update LM Studio and its llama.cpp runtime. Search for nace-ai/Qwen3.8-27B-GGUF under Discover, download the Q4_K_XL file, and load it in Chat. If you already have the file, add it with lms import --user-repo nace-ai/Qwen3.8-27B-GGUF Qwen3.8-27B-Q4_K_XL.gguf.
Ollama
ollama run hf.co/nace-ai/Qwen3.8-27B-GGUF:Q4_K_XL
Add --think=false to get the answer without the thinking.
Kapitsa
On a Mac with Kapitsa, download the file (hf comes with pip install -U huggingface_hub) and start it. The first start converts the file into Kapitsa's own format; then open http://127.0.0.1:8000 to chat, or send the curl request above to port 8000:
hf download nace-ai/Qwen3.8-27B-GGUF Qwen3.8-27B-Q4_K_XL.gguf --local-dir .
kapitsa serve --model ./Qwen3.8-27B-Q4_K_XL.gguf
Kapitsa stores the tensor types it has no kernels for at a higher bit width, so the converted model is larger than the file, and the measurements on this card, taken in llama.cpp, don't apply to it exactly.
Files
| File | Download |
|---|---|
Qwen3.8-27B-Q4_K_XL.gguf |
17.65 GB |
mmproj-bf16.gguf |
0.93 GB |
Download sizes, not memory use: the context and the engine need more. mmproj-bf16.gguf is the vision encoder, for image input.
Original model and license: Qwen/Qwen3.8-27B.
- Downloads last month
- 1,086
4-bit
Model tree for nace-ai/Qwen3.8-27B-GGUF
Base model
Qwen/Qwen3.8-27B

docker model run hf.co/nace-ai/Qwen3.8-27B-GGUF:Q4_K_XL