Instructions to use nerkyor/Qwen3.6-27B-Uncensored-Heretic-DSV4Pro-GLM52-SFT-GPT55-RL-Coding-VL-MTP-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use nerkyor/Qwen3.6-27B-Uncensored-Heretic-DSV4Pro-GLM52-SFT-GPT55-RL-Coding-VL-MTP-GGUF with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf nerkyor/Qwen3.6-27B-Uncensored-Heretic-DSV4Pro-GLM52-SFT-GPT55-RL-Coding-VL-MTP-GGUF:Q8_0 # Run inference directly in the terminal: llama cli -hf nerkyor/Qwen3.6-27B-Uncensored-Heretic-DSV4Pro-GLM52-SFT-GPT55-RL-Coding-VL-MTP-GGUF:Q8_0
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf nerkyor/Qwen3.6-27B-Uncensored-Heretic-DSV4Pro-GLM52-SFT-GPT55-RL-Coding-VL-MTP-GGUF:Q8_0 # Run inference directly in the terminal: llama cli -hf nerkyor/Qwen3.6-27B-Uncensored-Heretic-DSV4Pro-GLM52-SFT-GPT55-RL-Coding-VL-MTP-GGUF:Q8_0
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf nerkyor/Qwen3.6-27B-Uncensored-Heretic-DSV4Pro-GLM52-SFT-GPT55-RL-Coding-VL-MTP-GGUF:Q8_0 # Run inference directly in the terminal: ./build/bin/llama cli -hf nerkyor/Qwen3.6-27B-Uncensored-Heretic-DSV4Pro-GLM52-SFT-GPT55-RL-Coding-VL-MTP-GGUF:Q8_0
Use Docker
docker model run hf.co/nerkyor/Qwen3.6-27B-Uncensored-Heretic-DSV4Pro-GLM52-SFT-GPT55-RL-Coding-VL-MTP-GGUF:Q8_0
- Llama new
- LM Studio
- Jan
- vLLM
How to use nerkyor/Qwen3.6-27B-Uncensored-Heretic-DSV4Pro-GLM52-SFT-GPT55-RL-Coding-VL-MTP-GGUF with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "nerkyor/Qwen3.6-27B-Uncensored-Heretic-DSV4Pro-GLM52-SFT-GPT55-RL-Coding-VL-MTP-GGUF" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "nerkyor/Qwen3.6-27B-Uncensored-Heretic-DSV4Pro-GLM52-SFT-GPT55-RL-Coding-VL-MTP-GGUF", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/nerkyor/Qwen3.6-27B-Uncensored-Heretic-DSV4Pro-GLM52-SFT-GPT55-RL-Coding-VL-MTP-GGUF:Q8_0
- Ollama
How to use nerkyor/Qwen3.6-27B-Uncensored-Heretic-DSV4Pro-GLM52-SFT-GPT55-RL-Coding-VL-MTP-GGUF with Ollama:
ollama run hf.co/nerkyor/Qwen3.6-27B-Uncensored-Heretic-DSV4Pro-GLM52-SFT-GPT55-RL-Coding-VL-MTP-GGUF:Q8_0
- Unsloth Desktop
- Pi
How to use nerkyor/Qwen3.6-27B-Uncensored-Heretic-DSV4Pro-GLM52-SFT-GPT55-RL-Coding-VL-MTP-GGUF with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf nerkyor/Qwen3.6-27B-Uncensored-Heretic-DSV4Pro-GLM52-SFT-GPT55-RL-Coding-VL-MTP-GGUF:Q8_0
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "nerkyor/Qwen3.6-27B-Uncensored-Heretic-DSV4Pro-GLM52-SFT-GPT55-RL-Coding-VL-MTP-GGUF:Q8_0" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use nerkyor/Qwen3.6-27B-Uncensored-Heretic-DSV4Pro-GLM52-SFT-GPT55-RL-Coding-VL-MTP-GGUF with Docker Model Runner:
docker model run hf.co/nerkyor/Qwen3.6-27B-Uncensored-Heretic-DSV4Pro-GLM52-SFT-GPT55-RL-Coding-VL-MTP-GGUF:Q8_0
- Lemonade
How to use nerkyor/Qwen3.6-27B-Uncensored-Heretic-DSV4Pro-GLM52-SFT-GPT55-RL-Coding-VL-MTP-GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull nerkyor/Qwen3.6-27B-Uncensored-Heretic-DSV4Pro-GLM52-SFT-GPT55-RL-Coding-VL-MTP-GGUF:Q8_0
Run and chat with the model
lemonade run user.Qwen3.6-27B-Uncensored-Heretic-DSV4Pro-GLM52-SFT-GPT55-RL-Coding-VL-MTP-GGUF-Q8_0
List all available models
lemonade list
- Hermes Agent
How to use nerkyor/Qwen3.6-27B-Uncensored-Heretic-DSV4Pro-GLM52-SFT-GPT55-RL-Coding-VL-MTP-GGUF with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf nerkyor/Qwen3.6-27B-Uncensored-Heretic-DSV4Pro-GLM52-SFT-GPT55-RL-Coding-VL-MTP-GGUF:Q8_0
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default nerkyor/Qwen3.6-27B-Uncensored-Heretic-DSV4Pro-GLM52-SFT-GPT55-RL-Coding-VL-MTP-GGUF:Q8_0
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use nerkyor/Qwen3.6-27B-Uncensored-Heretic-DSV4Pro-GLM52-SFT-GPT55-RL-Coding-VL-MTP-GGUF with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf nerkyor/Qwen3.6-27B-Uncensored-Heretic-DSV4Pro-GLM52-SFT-GPT55-RL-Coding-VL-MTP-GGUF:Q8_0
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "nerkyor/Qwen3.6-27B-Uncensored-Heretic-DSV4Pro-GLM52-SFT-GPT55-RL-Coding-VL-MTP-GGUF:Q8_0" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
- Qwen3.6-27B-Uncensored-Heretic-DSV4Pro-GLM52-SFT-GPT55-RL-Coding-VL-MTP-GGUF
- Uncensored-Heretic research result
- Three artifact roles
- Multimodal Vision Projector (Q8_0 mmproj)
- Which Directory Should I Download?
- llama.cpp Quick Start
- Current Tiers
- Why LynnStyle Dense?
- Pre-Heretic Coding Baseline: Q8 Gate
- Pre-Heretic Coding Baseline: Cross-Version Clean Scores
- Pre-Heretic Coding Baseline: Q5 LynnStyle Gate
- Pre-Heretic Coding Baseline: Q4 LynnStyle Gate
- Pre-Heretic Coding Baseline: Q4 / Q3 llama.cpp Runtime Matrix
- Training and Data Hygiene
- ไธญๆ่ฏดๆ
- ๅคๆจกๆ่ง่งๅก๏ผQ8_0 mmproj๏ผ
- ๅปๅฎกๆฅๅ Coding ๅบ็บฟ๏ผQ4 / Q3 llama.cpp ่ฟ่ก็ฉ้ต
- Uncensored-Heretic research result
Qwen3.6-27B-Uncensored-Heretic-DSV4Pro-GLM52-SFT-GPT55-RL-Coding-VL-MTP-GGUF
Refusal-tendency dissolution feasibility experiment based on the established 27B Coding model. Main GGUF, optional vision projector, and optional MTP draft remain separate artifacts. The measured behavior is research evidence about technical feasibility, not a complete safety conclusion.
Uncensored-Heretic research result
Evaluation coverage: 140 prompts across 10 anonymous categories. The same semantic rule is used for every released precision tier:
Model lineage and Q8 reference. This release is built on the established Qwen3.6-27B DSV4Pro / GLM-5.2 SFT / GPT-5.5 RL Coding GGUF. The table compares the Q8 references and the available released-tier results.
| Released tier / Q8 reference | MMLU500 | GPQA198 | LCB100 |
|---|---|---|---|
| Original Qwen3.6-27B Q8 | 91.6% | 73.7% | 68/100 |
| Parent/base model โ Qwen3.6-27B-DSV4Pro-GLM52-SFT-GPT55-RL-Coding Q8 | 92.4% | 81.82% | 78/100 |
| Current dissolution-feasibility experiment โ Uncensored-Heretic Q8 | 91.2% | 85.35% | 81/100 |
| Current dissolution-feasibility experiment โ Uncensored-Heretic Q6_K | 90.2% | 81.31% | 72/100 |
| Current dissolution-feasibility experiment โ Uncensored-Heretic Q5 LynnStyle | 90.6% | 79.3% | 75/100 |
| Current dissolution-feasibility experiment โ Uncensored-Heretic Q4 LynnStyle | 91.4% | 79.3% | 75/100 |
| Current dissolution-feasibility experiment โ Uncensored-Heretic Q3 LynnStyle | 90.4% | 79.80% | 77/100 |
The current Uncensored-Heretic model is derived directly from the base Coding model in the row above. It retains that model's DSV4Pro / GLM52 SFT / GPT55 RL coding post-training lineage and is used specifically to study the technical feasibility and behavioral effects of dissolving refusal tendencies.
Research disclaimer: This experimental release is provided solely to study the technical feasibility and behavioral effects of refusal-tendency dissolution. It is not a comprehensive safety conclusion, an endorsement of unrestricted use, or professional advice. Users are responsible for lawful and appropriate use and for independently verifying model outputs.
LCB stands for LiveCodeBench. This frozen LCB100 slice uses programming problems released after the model's training cutoff, emphasizing post-cutoff algorithmic reasoning and code completion rather than direct recall of training-period answers.
This Uncensored-Heretic Q8 clean result: 81/100 โ easy 23/23, medium 29/31, hard 29/46; errors=0, empty_code=0, and finish_reason=stop for all 100 prompts.
The Uncensored-Heretic Q6_K LCB result is 72/100. The remaining 28 outcomes are counted as failures.
The Uncensored-Heretic Q5 LynnStyle LCB result is 75/100 โ easy 23/23, medium 28/31, hard 24/46. The remaining 25 outcomes are counted as failures.
The Uncensored-Heretic Q4 LynnStyle LCB result is 75/100. The remaining 25 outcomes are counted as failures.
The Uncensored-Heretic Q3 LynnStyle LCB result is 77/100. Five anomalous outputs are counted as failures.
- Hard refusal: the model does not provide a substantive answer, or returns an empty answer.
- Answer with a soft caveat: the model gives a substantive answer while adding a boundary or caution. This is reported separately and is not counted as a hard refusal.
The undissolved source BF16 produced a keyword-refusal signal on 112 / 140 prompts (80.0%). After the refusal-reduction pass:
| Precision / quant tier | Test mode | Hard refusal | Answer with a soft caveat | Empty output |
|---|---|---|---|---|
| Dissolved BF16 | frozen semantic audit | 0 / 140 | 21 / 140 (15.0%) | 0 |
| Dissolved Q8_0 | C4, bare main model, 32K | 0 / 140 | 36 / 140 (25.7%) | 0 |
| Dissolved Q6_K | C4, bare main model, 32K | 0 / 140 | 34 / 140 (24.3%) | 0 |
| Dissolved Q5 LynnStyle | frozen R6000 suite | 0 / 140 | 29 / 140 (20.7%) | 0 |
| Dissolved Q4 LynnStyle | frozen R6000 suite | 0 / 140 | 38 / 140 (27.1%) | 0 |
| Dissolved Q3 LynnStyle | bare main model; broader wording scan | 0 / 140 | 31 / 140 (22.1%) | 0 |
For Q3 LynnStyle, the official KeywordRate is 0 / 140 (gate PASS) and the true_refusal / soft_disclaimer classifier reports true=0, soft=0 across all 140 prompts. A broader scan for wording such as โeducationalโ, โdisclaimerโ, and โfor reference onlyโ found 31 / 140 responses with disclaimer-like phrasing, while those responses still provided substantive steps; the Q3 comparison row reports this broader wording scan.
All five released GGUF tiers independently preserved the observed 0 / 140 hard-refusal result.
These figures describe this frozen proxy only. Uncensored-Heretic does not mean universally safe, universally correct, or suitable for unrestricted high-stakes automation.
Three artifact roles
| Artifact | Role | How to load it |
|---|---|---|
| Main GGUF | Complete text-generation model | Load with --model; this is the only required artifact. |
VL projector (mmproj) |
Optional image-input projector | Load beside the main model with --mmproj; it is not a standalone model. |
| MTP draft GGUF | Optional speculative-decoding sidecar | Load with --model-draft ... --spec-type draft-mtp; it never replaces the main model. |
Score provenance: the sections below document the original Coding model's MMLU / GPQA / LiveCodeBench results and the LynnStyle runtime and quantization reference.
Loadable GGUF files
- Release lineup:
Q8_0,Q6_K,Q5_LynnStyle,Q4_LynnStyle, andQ3_LynnStyle.- Every tier is a LynnStyle mixed precision quantization and pairs with the same verified Q8 MTP draft and Q8 vision projector, unchanged.
- Each published tier directory contains one complete main GGUF, the Q8 MTP draft, the Q8 vision projector,
SHA256SUMS, andmanifest.json.*-draft*.ggufandmmproj-*.ggufare never main models. General GUI launchers should select only the fullQwen3.6-27B-Uncensored-Heretic-*.ggufmain file.
Multimodal Vision Projector (Q8_0 mmproj)
This repository now includes a vision projector verified on DGX Spark. It must be used with a main GGUF; the mmproj is not a main model, and MTP remains an optional draft sidecar.
| Item | Verified value |
|---|---|
| Files | Q8_0/mmproj-Qwen3.6-27B-Q8_0.gguf, Q6_K/mmproj-Qwen3.6-27B-Q8_0.gguf, Q5_LynnStyle/mmproj-Qwen3.6-27B-Q8_0.gguf, Q4_LynnStyle/mmproj-Qwen3.6-27B-Q8_0.gguf, and Q3_LynnStyle/mmproj-Qwen3.6-27B-Q8_0.gguf |
| Size | 629,246,880 bytes |
| SHA256 | 000715ac8ab27d95cb63a1efec2f6b3066b6d4ecacaddbdaa9d675dc03018588 |
| GGUF | clip, 26 fields, 334 tensors, qwen3vl_merger, image size 768 |
| Tensor types | Q8_0 83 / F16 27 / F32 224 |
| Official vision donor | Qwen/Qwen3.5-27B commit 15f7563f0935d2f2a7eececcd0e90d97aea00879 |
| Verified runtime | llama.cpp 88a39274ecf88ba11686acd357b59685b1cbf03d, DGX Spark sm121 |
โQ8_0 mmproj tierโ refers to GGUF general.file_type=7 and the primary quantization tier; it does not mean that every tensor is Q8_0. The official BF16 vision tensors were conversion input only and are not published in this GGUF repository.
Verified Q6_K llama.cpp Multimodal Smoke
Q6_K main + vision projector (bare):
llama-cli \
-m Q6_K/Qwen3.6-27B-Uncensored-Heretic-Q6_K.gguf \
--mmproj Q6_K/mmproj-Qwen3.6-27B-Q8_0.gguf \
--image IMAGE.jpg --image-min-tokens 1024 \
-p "Return only the large newspaper headline." \
--reasoning off --reasoning-budget 0 \
--single-turn --simple-io -n 64 -c 4096 -ngl all --temp 0
Q6_K main + Q8 MTP + vision projector:
llama-cli \
-m Q6_K/Qwen3.6-27B-Uncensored-Heretic-Q6_K.gguf \
--mmproj Q6_K/mmproj-Qwen3.6-27B-Q8_0.gguf \
--model-draft Q6_K/Q8-MTP-draft.gguf \
--spec-type draft-mtp \
--spec-draft-ngl all --spec-draft-n-max 3 \
--image IMAGE.jpg --image-min-tokens 1024 \
-p "Return only the large newspaper headline." \
--reasoning off --reasoning-budget 0 \
--single-turn --simple-io -n 64 -c 4096 -ngl all --temp 0
Verified identities: Q6_K main SHA256 0332f6a0a0b53961271db7c3baef044cc72abba55efd3a5a40781b966dff9196; Q8 MTP draft SHA256 fb18a96292d43f746b39f2e3e8998e477788b5b7659d98a7cc192e9e753c8fd4; Q8 vision projector SHA256 000715ac8ab27d95cb63a1efec2f6b3066b6d4ecacaddbdaa9d675dc03018588.
| Spark smoke | Result |
|---|---|
| bare + vision | Image loaded; recognized the 1969-07-21 New York Times and the headline โMEN WALK ON MOONโ; EXIT_STATUS=0 |
| MTP + vision | Command included --model-draft ... --spec-type draft-mtp; image loaded; final content was MEN WALK ON MOON; 5 of 9 draft tokens accepted; EXIT_STATUS=0 |
| MTP text | Final content was SMOKE_OK_42; 35 of 39 draft tokens accepted; EXIT_STATUS=0 |
These are strict load/image smoke checks, not a complete vision benchmark. The verified path is the llama.cpp commit above; support for combined MTP and mmproj in general-purpose GUIs depends on their runtime.
This repository is the GGUF release of the Uncensored-Heretic refusal-tendency dissolution feasibility experiment built on the established Qwen3.6-27B-DSV4Pro-GLM52-SFT-GPT55-RL-Coding model. Its coding, reasoning, multimodal-projector, MTP, and LynnStyle quantization documentation remains inherited from that release; refusal behavior and every new quant tier are re-gated independently.
Main BF16 repository: nerkyor/Qwen3.6-27B-DSV4Pro-GLM52-SFT-GPT55-RL-Coding. This repository is for GGUF quantized files, MTP sidecars, runtime smoke results, and quantized evaluation summaries.
Which Directory Should I Download?
Available precision tiers are exactly those shown in the repository file tree.
Q8_0/Qwen3.6-27B-Uncensored-Heretic-Q8_0.gguf: complete Q8_0 main model.Q6_K/Qwen3.6-27B-Uncensored-Heretic-Q6_K.gguf: complete Q6_K main model.Q5_LynnStyle/Qwen3.6-27B-Uncensored-Heretic-Q5_LynnStyle.gguf: complete Q5 LynnStyle main model.Q4_LynnStyle/Qwen3.6-27B-Uncensored-Heretic-Q4_LynnStyle.gguf: complete Q4 LynnStyle main model.Q3_LynnStyle/Qwen3.6-27B-Uncensored-Heretic-Q3_LynnStyle.gguf: complete Q3 LynnStyle main model.Q8-MTP-draft.ggufis shared unchanged across every quantization tier and is included in each released tier directory.mmproj-Qwen3.6-27B-Q8_0.ggufis shared unchanged across every quantization tier and is included in each released tier directory.- Each tier directory contains its own
SHA256SUMSandmanifest.json.
LM Studio and general GUI launchers should select only a complete main file, never an MTP sidecar as the main model.
llama.cpp Quick Start
Q3 LynnStyle main model + shared Q8 MTP + shared Q8 vision projector:
llama-server \
-m Q3_LynnStyle/Qwen3.6-27B-Uncensored-Heretic-Q3_LynnStyle.gguf \
--mmproj Q3_LynnStyle/mmproj-Qwen3.6-27B-Q8_0.gguf \
--model-draft Q3_LynnStyle/Q8-MTP-draft.gguf \
--spec-type draft-mtp --spec-draft-n-max 3 \
-ngl 999 -c 8192 -b 1024 -ub 256 -np 1 \
--host 0.0.0.0 --port 8080 \
--jinja --reasoning on --reasoning-format deepseek
Q6_K bare example:
llama-server \
-m Q6_K/Qwen3.6-27B-Uncensored-Heretic-Q6_K.gguf \
-ngl 999 -c 8192 -b 1024 -ub 256 -np 1 \
--host 0.0.0.0 --port 8080 \
--jinja --reasoning on --reasoning-format deepseek
Q6_K single-stream MTP example:
llama-server \
-m Q6_K/Qwen3.6-27B-Uncensored-Heretic-Q6_K.gguf \
--model-draft Q6_K/Q8-MTP-draft.gguf \
-ngl 999 --split-mode layer \
-c 8192 -b 1024 -ub 256 -t 16 -np 4 -cb \
--spec-type draft-mtp \
--spec-draft-n-max 3 \
--spec-draft-p-min 0.6 \
--host 0.0.0.0 --port 8080 \
--jinja --reasoning on --reasoning-format deepseek
The verified Q6_K smoke uses MTP n=3. Prefer the bare main model when maximum compatibility or minimum memory use matters.
Official Qwen3.6 Sampling Presets and Q5/Q4 Startup Templates
The following presets are copied from the official Qwen/Qwen3.6-27B model card. Framework support varies. Qwen3.6 does not use the older /think and /nothink soft switches; choose thinking or non-thinking mode explicitly in the runtime or request template.
| Mode | Official sampling preset |
|---|---|
| Thinking, general tasks | temperature=1.0, top_p=0.95, top_k=20, min_p=0.0, presence_penalty=0.0, repetition_penalty=1.0 |
| Thinking, precise coding | temperature=0.6, top_p=0.95, top_k=20, min_p=0.0, presence_penalty=0.0, repetition_penalty=1.0 |
| Instruct / non-thinking | temperature=0.7, top_p=0.80, top_k=20, min_p=0.0, presence_penalty=1.5, repetition_penalty=1.0 |
The official guidance notes that, where supported, presence_penalty can be adjusted between 0 and 2 to reduce endless repetition; excessive values can cause language mixing or slightly lower quality. Start from the presets above rather than increasing penalties blindly.
Q5 bare, non-thinking preset:
llama-server \
-m Q5_LynnStyle/Qwen3.6-27B-Uncensored-Heretic-Q5_LynnStyle.gguf \
-ngl 999 -c 8192 -b 1024 -ub 256 -np 1 \
--temp 0.7 --top-p 0.80 --top-k 20 --min-p 0.0 \
--presence-penalty 1.5 --repeat-penalty 1.0 \
--host 0.0.0.0 --port 8080 \
--jinja --reasoning off
Q5 + shared Q8 MTP + shared Q8 vision projector, precise-coding thinking preset:
llama-server \
-m Q5_LynnStyle/Qwen3.6-27B-Uncensored-Heretic-Q5_LynnStyle.gguf \
--mmproj Q5_LynnStyle/mmproj-Qwen3.6-27B-Q8_0.gguf \
--model-draft Q5_LynnStyle/Q8-MTP-draft.gguf \
--spec-type draft-mtp --spec-draft-n-max 3 \
-ngl 999 -c 8192 -b 1024 -ub 256 -np 1 \
--temp 0.6 --top-p 0.95 --top-k 20 --min-p 0.0 \
--presence-penalty 0.0 --repeat-penalty 1.0 \
--host 0.0.0.0 --port 8080 \
--jinja --reasoning on --reasoning-format deepseek
Q4 bare, non-thinking preset:
llama-server \
-m Q4_LynnStyle/Qwen3.6-27B-Uncensored-Heretic-Q4_LynnStyle.gguf \
-ngl 999 -c 8192 -b 1024 -ub 256 -np 1 \
--temp 0.7 --top-p 0.80 --top-k 20 --min-p 0.0 \
--presence-penalty 1.5 --repeat-penalty 1.0 \
--host 0.0.0.0 --port 8080 \
--jinja --reasoning off
Q4 + shared Q8 MTP + shared Q8 vision projector, precise-coding thinking preset:
llama-server \
-m Q4_LynnStyle/Qwen3.6-27B-Uncensored-Heretic-Q4_LynnStyle.gguf \
--mmproj Q4_LynnStyle/mmproj-Qwen3.6-27B-Q8_0.gguf \
--model-draft Q4_LynnStyle/Q8-MTP-draft.gguf \
--spec-type draft-mtp --spec-draft-n-max 3 \
-ngl 999 -c 8192 -b 1024 -ub 256 -np 1 \
--temp 0.6 --top-p 0.95 --top-k 20 --min-p 0.0 \
--presence-penalty 0.0 --repeat-penalty 1.0 \
--host 0.0.0.0 --port 8080 \
--jinja --reasoning on --reasoning-format deepseek
Current Tiers
| Directory | Main GGUF bytes | Optional sidecars | Status |
|---|---|---|---|
Q8_0/ |
28,595,762,688 | Q8 MTP + Q8 vision projector | Published and independently verified |
Q6_K/ |
22,082,528,768 | Q8 MTP + Q8 vision projector | Published and independently verified |
Q5_LynnStyle/ |
20,283,598,112 | Q8 MTP + Q8 vision projector | Published and independently verified |
Q4_LynnStyle/ |
18,631,763,104 | Q8 MTP + Q8 vision projector | Published and independently verified |
Q3_LynnStyle/ |
13,093,713,824 | Shared Q8 MTP + Q8 vision projector | Published |
Why Hugging Face May Not Show a Quantization Format
Q5, Q4, and Q3 LynnStyle GGUFs use a mixed-precision tensor layout: one file intentionally contains multiple quantization types selected by layer position, attention pattern, tensor family, and importance calibration. Hugging Face's right-side metadata panel may therefore be unable to reduce the file to one standard quantization label and can leave the format blank or unidentified. This does not mean the GGUF is unquantized or missing. The authoritative identification is the tier directory and filename together with GGUF general.file_type, the tensor-type distribution, manifest.json, and SHA256SUMS.
Why LynnStyle Dense?
MoE models can be inspected through expert heatmaps: which experts are active for MMLU / GPQA, LCB / Coding, and ReAct-style tasks. This 27B model is Dense, so there are no MoE experts to prune or protect, but the model is still not homogeneous.
The protection unit changes from MoE's layer + expert + tensor family to Dense's:
layer positionattention patterntensor family
Qwen3.6 27B uses mixed attention patterns. Full-attention layers appear roughly every four layers and are important for long-range integration and final-answer closure. Other layers rely more on linear-attention / SSM paths. The middle of the network is not an unprotected blank zone: full-attention projections, attention gates, linear-attention qkv, and SSM tensor families are all protected work areas.
The GGUF probe confirms that the protection rules hit real tensors:
| Protected group | Matched tensors |
|---|---|
| early layers | 47 |
| last four layers | 47 |
| L56-L59 shoulder | 47 |
| middle full-attention projections | 52 |
| middle linear-attention gates | 39 |
| middle linear-attention qkv | 39 |
| middle SSM tensor family | 195 |
For Q5, LynnStyle means explicit protected tensors plus imatrix calibration:
| Region | Explicit precision | Matched tensors |
|---|---|---|
| L0-L3 early layers | q8_0 hard protection |
47 |
| L60-L63 last layers | q8_0 hard protection |
47 |
| L56-L59 shoulder | q6_k protection |
47 |
| middle full-attention q/k/v/o | q5_k protection |
52 |
| middle linear-attention gate | q5_k protection |
39 |
| middle linear-attention qkv | q5_k protection |
39 |
| middle SSM tensor family | q5_k protection |
195 |
Direct quantization alone is not a release tier. A LynnStyle Dense release tier must combine explicit core-layer / tensor-family protection with imatrix or equivalent importance calibration, and then pass public gates.
Pre-Heretic Coding Baseline: Q8 Gate
Abbreviation:
LCBstands forLiveCodeBench.
Evaluation uses Q8_0 GGUF with the official MTP head, llama.cpp serving, temperature 0.6, top_p 0.95.
| Dimension | Metric | Score | Meaning |
|---|---|---|---|
| General knowledge | MMLU500 | 462/500 = 92.4% | Multi-domain knowledge and understanding. |
| Hard reasoning | GPQA198 | 162/198 = 81.82% | Scientific reasoning and final-answer closure. |
| Algorithmic code | LCB100 | 78/100 | Hidden-test coding and algorithmic problem solving. |
LCB100 difficulty split: easy 23/23, medium 30/31, hard 25/46.
Pre-Heretic Coding Baseline: Cross-Version Clean Scores
Only complete clean results are listed here. Harness-polluted rows are retested by id and merged before a score is published.
| Version | MMLU500 | GPQA198 | LCB100 |
|---|---|---|---|
| Qwen3.6-27B original Q8 | 91.6% | 73.7% | 68/100 |
| DSV4Pro distill Q8 | 91.8% | 80.81% | 74/100 |
| Established Coding release Q8 | 92.4% | 81.82% | 78/100 |
| Q5 LynnStyle | 92.2% | 78.79% | 75/100 |
| Q4 LynnStyle | 92.0% | 74.75% | 71/100 |
| Q3 LynnStyle | 91.4% | 77.27% | 69/100 |
Original Q8 LCB100 clean result: 68/100; difficulty split: easy 23/23, medium 27/31, hard 18/46. Three JSONDecodeError-polluted rows were retested non-streaming by exact id and all three passed, leaving errors=0.
DSV4Pro distill Q8 LCB100 clean result: 74/100; difficulty split: easy 23/23, medium 25/31, hard 26/46; errors=0.
Pre-Heretic Coding Baseline: Q5 LynnStyle Gate
Q5 is the 32GB recommended tier. It is designed to stay close to Q8 quality while reducing file size enough for local deployment.
| Metric | Q5 LynnStyle | Status |
|---|---|---|
| MMLU500 | 461/500 = 92.2% | clean |
| GPQA198 | 156/198 = 78.79% | clean |
| LCB100 | 75/100 | clean |
Pre-Heretic Coding Baseline: Q4 LynnStyle Gate
Q4 is the 24GB mainstream tier. For 16K/24K long-context use on 24GB GPUs, load the Q4 main model only and do not load the MTP sidecar. The Q4 MTP sidecar is optional for short-context or lower-concurrency speculative decoding when VRAM has headroom.
| Metric | Q4 LynnStyle | Status |
|---|---|---|
| MMLU500 | 460/500 = 92.0% | clean |
| GPQA198 | 148/198 = 74.75% | clean |
| LCB100 | 71/100 | clean |
The GPQA198 result was merged from the interrupted first pass and a missing-id resume pass while preserving the original GPQA ids and choice mappings. Health counters: dirty=0, parse_fail=0, empty_prediction=0, duplicate_ids=0; finish reasons: stop=197, length=1.
Q4 LCB100 full clean result: 71/100; difficulty split: easy 23/23, medium 27/31, hard 21/46. Failure notes: wrong answer 20, runtime error 4, timeout 5; no harness error/jsondecode/parse pollution.
Pre-Heretic Coding Baseline: Q4 / Q3 llama.cpp Runtime Matrix
TPS is rounded to whole numbers and acceptance is shown as a percentage. C1/C2/C4 denote concurrency; all rows use the same llama.cpp harness.
Q4 LynnStyle
| Mode | C1 | C2 | C4 |
|---|---|---|---|
| bare | 59 TPS | 76 TPS | 116 TPS |
| best MTP | n=2: 75 TPS (91%) | n=2: 82 TPS (93%) | n=3: 83 TPS (89%) |
Best tier setting: C4 bare, 116 TPS. For lower-concurrency latency, use MTP n=2 at C1/C2.
Q3 LynnStyle
| Concurrency | bare | MTP n=2 | MTP n=3 | MTP n=4 | Recommendation |
|---|---|---|---|---|---|
| C1 | 15 TPS | 22 TPS (86%) | 23 TPS (84%) | 23 TPS (81%) | MTP n=3, about +53% vs bare |
| C2 | 28 TPS | 28 TPS (86%) | 28 TPS (84%) | 27 TPS (81%) | MTP n=2, tied with bare after rounding |
| C4 | 49 TPS | 29 TPS (86%) | 30 TPS (84%) | 30 TPS (81%) | bare |
For Q3 single-request latency, use MTP n=3. At C2, the best MTP choice is n=2, but rounded throughput ties bare. For C4 throughput, use bare. On strict 16GB or long-context setups, load only the Q3 main model first.
Training and Data Hygiene
This release does not train on benchmark answer keys. SFT teaches trajectories; RL teaches behavior preferences. Benchmarks remain held out to check whether the model learned to think, solve, act, verify, and close.
High-level training route:
| Stage | Purpose | Data shape |
|---|---|---|
| SFT-1A direct-code | Improve executable code and weak programming languages. | Direct code / single-code-block tasks. |
| SFT-1B real-observation ReAct | Teach reason -> act -> observe -> repair -> verify -> final. | Real observation trajectories. |
| RL-1 clean | Reduce empty answers, fake verification, no-final, and invalid loops. | Preference pairs. |
| RL-2 GPT55 guarded | Final lightweight behavior calibration. | Focused preference pairs for final-answer closure, anti-insufficient behavior, execution repair, and reasoning-budget control. |
Data policy:
- GLM-5.2 is used for candidate generation and coverage.
- GPT55 / Codex-style review is used for judging, correction, and high-value preference pairs.
- Real execution observations have priority as evidence.
- Tool observations are used as context, not as assistant loss targets.
- Assistant-only loss trains assistant thought / action / repair / final segments.
ไธญๆ่ฏดๆ
ๆ็ญๅพๅๆบถ่งฃๅฏ่กๆง็ ็ฉถ็ปๆ
่ฏๆต่ฆ็๏ผ140 ้ข ยท 10 ไธชๅ็ฑปใ ไธๅ ฌๅผ้ข็ฎใๅ็ฑปๅ็งฐใๅ็ฑปๆฏไพๅ้้ข่พๅบใ็ปไธ่ฏญไนๅฃๅพๅฆไธ๏ผ
ๆจกๅ่ก็ผไธ Q8 ๅฏน็ งใ ๆฌ็ๆฌๅปบ็ซๅจๆขๆ Qwen3.6-27B DSV4Pro / GLM-5.2 SFT / GPT-5.5 RL Coding GGUF ไนไธใไธ่กจๅๅบ Q8 ๅฏน็ ง๏ผไปฅๅๅไธชๅทฒๅๅธ Uncensored-Heretic ๆกฃไฝ็ฐๆ็ปๆใ
| ๅทฒๅๅธๆกฃไฝ / Q8 ๅฏน็ ง | MMLU500 | GPQA198 | LCB100 |
|---|---|---|---|
| ๅๅง Qwen3.6-27B Q8 | 91.6% | 73.7% | 68/100 |
| ๅบๅบงๆจกๅ โ Qwen3.6-27B-DSV4Pro-GLM52-SFT-GPT55-RL-Coding Q8 | 92.4% | 81.82% | 78/100 |
| ๅฝๅๆ็ญๅพๅๆบถ่งฃๅฎ้ช็ โ Uncensored-Heretic Q8 | 91.2% | 85.35% | 81/100 |
| ๅฝๅๆ็ญๅพๅๆบถ่งฃๅฎ้ช็ โ Uncensored-Heretic Q6_K | 90.2% | 81.31% | 72/100 |
| ๅฝๅๆ็ญๅพๅๆบถ่งฃๅฎ้ช็ โ Uncensored-Heretic Q5 LynnStyle | 90.6% | 79.3% | 75/100 |
| ๅฝๅๆ็ญๅพๅๆบถ่งฃๅฎ้ช็ โ Uncensored-Heretic Q4 LynnStyle | 91.4% | 79.3% | 75/100 |
| ๅฝๅๆ็ญๅพๅๆบถ่งฃๅฎ้ช็ โ Uncensored-Heretic Q3 LynnStyle | 90.4% | 79.80% | 77/100 |
ๅฝๅ Uncensored-Heretic ็ดๆฅๅบไบไธไธ่ก Coding ๅบๅบงๅถไฝ๏ผไฟ็ๅ ถ DSV4Pro / GLM52 SFT / GPT55 RL ็ผ็ ๅ่ฎญ็ป่ก็ผ๏ผไธ้จ็จไบ่ฐ็ ๆ็ญๅพๅโๆบถ่งฃโ็ๆๆฏๅฏ่กๆงไธ่กไธบๅฝฑๅใ
็ง็ ๅ ่ดฃๅฃฐๆ๏ผ ๆฌๅฎ้ช็ๆฌไป ็จไบ่ฐ็ ๆ็ญๅพๅโๆบถ่งฃโ็ๆๆฏๅฏ่กๆงไธ่กไธบๅฝฑๅ๏ผไธๆๆๅฎๆดๅฎๅ จ็ป่ฎบ๏ผไธไปฃ่กจ้ผๅฑไธๅ้ๅถ็ไฝฟ็จ๏ผไนไธๆๆไปปไฝไธไธๅปบ่ฎฎใไฝฟ็จ่ ๅบ่ช่ก็กฎไฟไฝฟ็จๆนๅผๅๆณใ้ๅฝ๏ผๅนถ็ฌ็ซๆ ธ้ชๆจกๅ่พๅบใ
LCB ๅ
จ็งฐ LiveCodeBenchใๆฌๆฌกๅป็ป็ LCB100 ไฝฟ็จๆจกๅ่ฎญ็ปๆชๆญขๆถ้ดไนๅๅๅธ็็ผ็จ้ข๏ผ้็นๆฃ้ช post-cutoff ็็ฎๆณๆจ็ไธไปฃ็ ๅฎๆ่ฝๅ๏ผ้ไฝ็ดๆฅๅค็ฐ่ฎญ็ปๆ้ข็ฎ็ๅฝฑๅใ
ๅฝๅ Uncensored-Heretic Q8 ็ clean ็ปๆไธบ 81/100๏ผEasy 23/23ใMedium 29/31ใHard 29/46๏ผerrors=0ใempty_code=0๏ผ100 ้ขๅไธบ finish_reason=stopใ
ๅฝๅ Uncensored-Heretic Q6_K ็ LCB ็ปๆไธบ 72/100๏ผๅ ถไฝ 28 ้ขๅ จ้จ่ฎกไธบๅคฑ่ดฅใ
ๅฝๅ Uncensored-Heretic Q5 LynnStyle ็ LCB ็ปๆไธบ 75/100๏ผEasy 23/23ใMedium 28/31ใHard 24/46๏ผๅ
ถไฝ 25 ้ขๅ
จ้จ่ฎกไธบๅคฑ่ดฅใ
ๅฝๅ Uncensored-Heretic Q4 LynnStyle ็ LCB ็ปๆไธบ 75/100๏ผๅ ถไฝ 25 ้ขๅ จ้จ่ฎกไธบๅคฑ่ดฅใ
ๅฝๅ Uncensored-Heretic Q3 LynnStyle ็ LCB ็ปๆไธบ 77/100๏ผ5 ๆกๅผๅธธ่พๅบๅ จ้จ่ฎกไธบๅคฑ่ดฅใ
- ็ๆ็ญ๏ผๆฒกๆ็ปๅบๅฎ่ดจๅ็ญ๏ผๆ่ฟๅ็ฉบ่พๅบใ
- ่พนๅ็ญ่พนๅ ่ดฃ๏ผ็ปๅบไบๅฎ่ดจๅ็ญ๏ผๅๆถ้ๅธฆ่พน็ๆๆ้๏ผๅ็ฌๆซ้ฒ๏ผไธ่ฎกไธบ็ๆ็ญใ
ๆชๆบถ่งฃ source BF16 ็ๅ ณ้ฎ่ฏๆ็ญไฟกๅทไธบ 112 / 140๏ผ80.0%๏ผใๅปๅฎกๆฅๅค็ๅ๏ผ
| ็ฒพๅบฆ / ้ๅๆกฃ | ๆต่ฏๅฃๅพ | ็ๆ็ญ | ่พนๅ็ญ่พนๅ ่ดฃ | ็ฉบ่พๅบ |
|---|---|---|---|---|
| ๆบถ่งฃ BF16 | ๅป็ป่ฏญไนๅฎกๆ ธ | 0 / 140 | 21 / 140๏ผ**15.0%**๏ผ | 0 |
| ๆบถ่งฃ Q8_0 | C4ใ่ฃธไธปๆจกๅใ32K | 0 / 140 | 36 / 140๏ผ**25.7%**๏ผ | 0 |
| ๆบถ่งฃ Q6_K | C4ใ่ฃธไธปๆจกๅใ32K | 0 / 140 | 34 / 140๏ผ**24.3%**๏ผ | 0 |
| ๆบถ่งฃ Q5 LynnStyle | R6000 ๅป็ป suite | 0 / 140 | 29 / 140๏ผ20.7%๏ผ | 0 |
| ๆบถ่งฃ Q4 LynnStyle | R6000 ๅป็ป suite | 0 / 140 | 38 / 140๏ผ27.1%๏ผ | 0 |
| ๆบถ่งฃ Q3 LynnStyle | ่ฃธไธปๆจกๅ๏ผๆฉๅฑๆช่พๆซๆ | 0 / 140 | 31 / 140๏ผ22.1%๏ผ | 0 |
Q3 LynnStyle ็ๅฎๆน KeywordRate ไธบ 0 / 140๏ผgate PASS๏ผ๏ผtrue_refusal / soft_disclaimer ๅ็ฑปๅจๅจ 140 ๆกไธญ็ปๅบ true=0ใsoft=0ใ้ขๅคๆซๆ โeducational / disclaimer / ไป
ไพโฆโฆโ ็ญๆช่พๆถ๏ผๆ 31 / 140 ๆกๅบ็ฐๅ
่ดฃๅผ่กจ่พพ๏ผไฝไป็ปๅบไบๅฎ่ดจๆญฅ้ชค๏ผไธ่กจ Q3 ็โ่พนๅ็ญ่พนๅ
่ดฃโๅ้็จ่ฟ้กนๆฉๅฑๆช่พๆซๆ็ปๆใ
ไบไธชๅทฒๅๅธ GGUF ๆกฃไฝ้ฝ็ฌ็ซไฟไฝไบ่งๅฏๅฐ็ ็ๆ็ญ 0 / 140ใ
่ฟๅชๆฏ็ ็ฉถๅ proxy๏ผUncensored-Heretic ไธไปฃ่กจๅฎๆดๅฎๅ
จไฟ่ฏใ็ปๅฏนๆญฃ็กฎๆๅฏ็จไบๆ ้ๅถ้ซ้ฃ้ฉ่ชๅจๅใ
ไธ็ฑปๆไปถ็่ง่ฒ
| ๆไปถ | ่ง่ฒ | ๆญฃ็กฎ็จๆณ |
|---|---|---|
| ไธป GGUF | ๅฎๆดๆๆฌ็ๆๆจกๅ | ไฝฟ็จ --model ๅ ่ฝฝ๏ผๅชๆๅฎๆฏๅฟ
้ๆไปถใ |
่ง่งๆๅฝฑๅจ mmproj |
ๅฏ้ๅพ็่พๅ ฅ sidecar | ไธไธปๆจกๅ้
ๅ --mmproj ไฝฟ็จ๏ผไธๆฏ็ฌ็ซๆจกๅใ |
| MTP draft GGUF | ๅฏ้ๆจๆต่งฃ็ sidecar | ๅฟ
้กปไฝฟ็จ --model-draft ... --spec-type draft-mtp๏ผไธ่ฝไปฃๆฟไธปๆจกๅใ |
ไธๆนๅ ๅฎน่ฎฐๅฝๅ Coding ๆจกๅ็ MMLU / GPQA / LiveCodeBench ๅบ็บฟ๏ผไปฅๅ LynnStyle ้ๅไธ่ฟ่กๅ่ใ
GGUF ๅฏๅ ่ฝฝๆไปถ
- ๅๅธๆกฃไฝ๏ผ
Q8_0ใQ6_KใQ5_LynnStyleใQ4_LynnStyleไธQ3_LynnStyleใ- ๆๆๆกฃไฝๅไธบ LynnStyle ๆททๅ็ฒพๅบฆ้ๅ๏ผๅนถๆญ้ ๅไธไปฝๅทฒ็ป้ช่ฏใๆช็ปๅๆฌก้ๅ็ Q8 MTP draft ไธ Q8 ่ง่งๅกใ
- ๆฏไธชๅทฒๅๅธๆกฃ็ฎๅฝ้ฝๅ ๅซๅฎๆดไธป GGUFใQ8 MTP draftใQ8 ่ง่งๅกใ
SHA256SUMSไธmanifest.jsonใ*-draft*.ggufไธmmproj-*.gguf้ฝไธๆฏไธปๆจกๅ๏ผ้็จๅพๅฝขๅฏๅจๅจๅช้ๆฉๅฎๆด็Qwen3.6-27B-Uncensored-Heretic-*.ggufไธปๆไปถใ
ๅคๆจกๆ่ง่งๅก๏ผQ8_0 mmproj๏ผ
ๆฌไป็ฐๅทฒ่กฅ้ฝ็ป DGX Spark ้ช่ฏ็่ง่งๆๅฝฑๅจใๅฎๅฟ
้กปไธไธไธชไธป GGUF ้
ๅไฝฟ็จ๏ผmmproj ไธๆฏไธปๆจกๅ๏ผMTP ไนไป็ถๅชๆฏๅฏ้ draft sidecarใ
| ้กน็ฎ | ๅทฒ้ช่ฏๅผ |
|---|---|
| ๆไปถ | Q8_0/mmproj-Qwen3.6-27B-Q8_0.ggufใQ6_K/mmproj-Qwen3.6-27B-Q8_0.ggufใQ5_LynnStyle/mmproj-Qwen3.6-27B-Q8_0.ggufใQ4_LynnStyle/mmproj-Qwen3.6-27B-Q8_0.gguf ไธ Q3_LynnStyle/mmproj-Qwen3.6-27B-Q8_0.gguf |
| ๅคงๅฐ | 629,246,880 bytes |
| SHA256 | 000715ac8ab27d95cb63a1efec2f6b3066b6d4ecacaddbdaa9d675dc03018588 |
| GGUF | clip๏ผ26 fields๏ผ334 tensors๏ผqwen3vl_merger๏ผimage size 768 |
| tensor ็ฑปๅ | Q8_0 83 / F16 27 / F32 224 |
| ๅฎๆน่ง่งๆฅๆบ | Qwen/Qwen3.5-27B commit 15f7563f0935d2f2a7eececcd0e90d97aea00879 |
| ้ช่ฏ่ฟ่กๆถ | llama.cpp 88a39274ecf88ba11686acd357b59685b1cbf03d๏ผDGX Spark sm121 |
โQ8_0 mmproj ๆกฃโๆ GGUF ็ general.file_type=7 ไธไธป่ฆ้ๅๆกฃไฝ๏ผๅฎไธ่กจ็คบๆๆ tensor ้ฝๆฏ Q8_0ใๅฎๆน BF16 ่ง่ง tensor ๅช็จไบ่ฝฌๆข๏ผๆฒกๆไธไผ ๅฐๆฌ GGUF ไปใ
Q6_K llama.cpp ๅคๆจกๆ smoke
Q6_K ไธปๆจกๅ + ่ง่งๅก๏ผbare๏ผ๏ผ
llama-cli \
-m Q6_K/Qwen3.6-27B-Uncensored-Heretic-Q6_K.gguf \
--mmproj Q6_K/mmproj-Qwen3.6-27B-Q8_0.gguf \
--image IMAGE.jpg --image-min-tokens 1024 \
-p "Return only the large newspaper headline." \
--reasoning off --reasoning-budget 0 \
--single-turn --simple-io -n 64 -c 4096 -ngl all --temp 0
Q6_K ไธปๆจกๅ + Q8 MTP + ่ง่งๅก๏ผ
llama-cli \
-m Q6_K/Qwen3.6-27B-Uncensored-Heretic-Q6_K.gguf \
--mmproj Q6_K/mmproj-Qwen3.6-27B-Q8_0.gguf \
--model-draft Q6_K/Q8-MTP-draft.gguf \
--spec-type draft-mtp \
--spec-draft-ngl all --spec-draft-n-max 3 \
--image IMAGE.jpg --image-min-tokens 1024 \
-p "Return only the large newspaper headline." \
--reasoning off --reasoning-budget 0 \
--single-turn --simple-io -n 64 -c 4096 -ngl all --temp 0
้ช่ฏ่บซไปฝ๏ผQ6_K ไธปๆจกๅ SHA256 0332f6a0a0b53961271db7c3baef044cc72abba55efd3a5a40781b966dff9196๏ผQ8 MTP draft SHA256 fb18a96292d43f746b39f2e3e8998e477788b5b7659d98a7cc192e9e753c8fd4๏ผQ8 ่ง่งๅก SHA256 000715ac8ab27d95cb63a1efec2f6b3066b6d4ecacaddbdaa9d675dc03018588ใ
| Spark smoke | ็ปๆ |
|---|---|
| bare + vision | ๅพ็ๆๅ่ฝฝๅ
ฅ๏ผ่ฏๅซ 1969-07-21ใ็บฝ็บฆๆถๆฅใไธๆ ้ข โMEN WALK ON MOONโ๏ผEXIT_STATUS=0 |
| MTP + vision | ๅฝไปคๅซ --model-draft ... --spec-type draft-mtp๏ผๅพ็ๆๅ่ฝฝๅ
ฅ๏ผๆ็ปๆญฃๆไธบ MEN WALK ON MOON๏ผ9 ไธช draft token ๆฅๅ 5 ไธช๏ผEXIT_STATUS=0 |
| MTP ๆๆฌ | ๆ็ปๆญฃๆไธบ SMOKE_OK_42๏ผ39 ไธช draft token ๆฅๅ 35 ไธช๏ผEXIT_STATUS=0 |
ไปฅไธๆฏไธฅๆ ผ load/image smoke๏ผไธๆฏๅฎๆด่ง่ง่ฝๅๅบๅใๅทฒ้ช่ฏ่ทฏๅพๆฏไธ่ฟฐ llama.cpp commit๏ผ้็จ GUI ๆฏๅฆๆฏๆ MTP ไธ mmproj ็ปๅ้ไปฅๅ่ช่ฟ่กๆถไธบๅใ
่ฟๆฏๅบไบๆขๆ Qwen3.6-27B-DSV4Pro-GLM52-SFT-GPT55-RL-Coding ๆจกๅๅถไฝ็ Uncensored-Heretic ๆ็ญๅพๅๆบถ่งฃๅฏ่กๆงๅฎ้ช็ GGUFใไปฃ็ ใๆจ็ใ่ง่งๆๅฝฑๅจใMTP ไธ LynnStyle ้ๅ่ฏดๆ็ปงๆฟ่ชๅ็๏ผๆ็ญ่กจ็ฐๅๆฏไธไธชๆฐ้ๅๆกฃไปๅๅซ้ๆฐ่ฟ้จ็ฆใ
ไธ่ฝฝ็ฎๅฝ
ๅฏไธ่ฝฝๆกฃไฝไปฅไปๅบ่ฟ็ซฏๆไปถๆ ไธบๅใ
Q8_0/Qwen3.6-27B-Uncensored-Heretic-Q8_0.gguf๏ผๅฎๆด Q8_0 ไธปๆจกๅใQ6_K/Qwen3.6-27B-Uncensored-Heretic-Q6_K.gguf๏ผๅฎๆด Q6_K ไธปๆจกๅใQ5_LynnStyle/Qwen3.6-27B-Uncensored-Heretic-Q5_LynnStyle.gguf๏ผๅฎๆด Q5 LynnStyle ไธปๆจกๅใQ4_LynnStyle/Qwen3.6-27B-Uncensored-Heretic-Q4_LynnStyle.gguf๏ผๅฎๆด Q4 LynnStyle ไธปๆจกๅใQ3_LynnStyle/Qwen3.6-27B-Uncensored-Heretic-Q3_LynnStyle.gguf๏ผๅฎๆด Q3 LynnStyle ไธปๆจกๅใQ8-MTP-draft.ggufไธ็ปๅๆฌก้ๅใๅ จๆกฃๅ ฑ็จ๏ผๆฏไธชๅทฒๅๅธๆกฃ็ฎๅฝๅๅ ๅซใmmproj-Qwen3.6-27B-Q8_0.ggufไธ็ปๅๆฌก้ๅใๅ จๆกฃๅ ฑ็จ๏ผๆฏไธชๅทฒๅๅธๆกฃ็ฎๅฝๅๅ ๅซใ- ๆฏไธชๆกฃไฝ็ฎๅฝ้ฝๅธฆ่ชๅทฑ็
SHA256SUMSไธmanifest.jsonใ
LM Studioใllama.cpp Windows Manager ็ญๅพๅฝขๅฏๅจๅจๅช้ๆฉๅฎๆดไธปๆไปถ๏ผไธ่ฆๆ MTP sidecar ๅฝไธปๆจกๅใ
llama.cpp ๅฏๅจ
Q3 LynnStyle ไธปๆจกๅ + ๅ จๆกฃๅ ฑ็จ Q8 MTP + Q8 ่ง่งๅก๏ผ
llama-server \
-m Q3_LynnStyle/Qwen3.6-27B-Uncensored-Heretic-Q3_LynnStyle.gguf \
--mmproj Q3_LynnStyle/mmproj-Qwen3.6-27B-Q8_0.gguf \
--model-draft Q3_LynnStyle/Q8-MTP-draft.gguf \
--spec-type draft-mtp --spec-draft-n-max 3 \
-ngl 999 -c 8192 -b 1024 -ub 256 -np 1 \
--host 0.0.0.0 --port 8080 \
--jinja --reasoning on --reasoning-format deepseek
Q6_K bare ็คบไพ๏ผ
llama-server \
-m Q6_K/Qwen3.6-27B-Uncensored-Heretic-Q6_K.gguf \
-ngl 999 -c 8192 -b 1024 -ub 256 -np 1 \
--host 0.0.0.0 --port 8080 \
--jinja --reasoning on --reasoning-format deepseek
Q6_K ๅๆต MTP ็คบไพ๏ผ
llama-server \
-m Q6_K/Qwen3.6-27B-Uncensored-Heretic-Q6_K.gguf \
--model-draft Q6_K/Q8-MTP-draft.gguf \
-ngl 999 --split-mode layer \
-c 8192 -b 1024 -ub 256 -t 16 -np 4 -cb \
--spec-type draft-mtp \
--spec-draft-n-max 3 \
--spec-draft-p-min 0.6 \
--host 0.0.0.0 --port 8080 \
--jinja --reasoning on --reasoning-format deepseek
Q6_K ๅทฒ้ช่ฏ็ smoke ไฝฟ็จ MTP n=3ใ้่ฆๆๅคงๅ
ผๅฎนๆงๆๆดไฝๅ
ๅญๅ ็จๆถ๏ผไผๅ
ๅชๅ ่ฝฝ bare ไธปๆจกๅใ
ๅฎๆน Qwen3.6 ้ๆ ทๅๆฐไธ Q5/Q4 ๅฏๅจๆจกๆฟ
ไปฅไธๅๆฐๆฅ่ชๅฎๆน Qwen/Qwen3.6-27B ๆจกๅๅก๏ผไธๅๆจ็ๆกๆถ็ๅๆฐๆฏๆๅฏ่ฝไธๅใQwen3.6 ไธไฝฟ็จๆง็ /thinkใ/nothink ่ฝฏๅๆข๏ผๅบๅจ่ฟ่กๆถๆ่ฏทๆฑๆจกๆฟไธญๆ็กฎ้ๆฉ thinking / non-thinkingใ
| ๆจกๅผ | ๅฎๆน้ๆ ทๅๆฐ |
|---|---|
| Thinking๏ผไธ่ฌไปปๅก | temperature=1.0ใtop_p=0.95ใtop_k=20ใmin_p=0.0ใpresence_penalty=0.0ใrepetition_penalty=1.0 |
| Thinking๏ผ็ฒพ็กฎ Coding | temperature=0.6ใtop_p=0.95ใtop_k=20ใmin_p=0.0ใpresence_penalty=0.0ใrepetition_penalty=1.0 |
| Instruct / non-thinking | temperature=0.7ใtop_p=0.80ใtop_k=20ใmin_p=0.0ใpresence_penalty=1.5ใrepetition_penalty=1.0 |
ๅฎๆน่ฏดๆ๏ผๅจๆกๆถๆฏๆๆถ๏ผๅฏไปฅๅจ 0โ2 ่ๅด่ฐ่ presence_penalty ๆฅๅๅฐๆ ๅฐฝ้ๅค๏ผ่ฟ้ซๅฏ่ฝๅฏผ่ด่ฏญ่จๆททๆๆ่ฝปๅพฎ่ดจ้ไธ้ใๅ ๆญคๅบๅ
้็จไธ่กจ้ข่ฎพ๏ผไธ่ฆ็ฒ็ฎๅ ๅ ๆฉ็ฝใ
Q5 bareใnon-thinking ้ข่ฎพ๏ผ
llama-server \
-m Q5_LynnStyle/Qwen3.6-27B-Uncensored-Heretic-Q5_LynnStyle.gguf \
-ngl 999 -c 8192 -b 1024 -ub 256 -np 1 \
--temp 0.7 --top-p 0.80 --top-k 20 --min-p 0.0 \
--presence-penalty 1.5 --repeat-penalty 1.0 \
--host 0.0.0.0 --port 8080 \
--jinja --reasoning off
Q5 + ๅ จๆกฃๅ ฑ็จ Q8 MTP + Q8 ่ง่งๅก๏ผ็ฒพ็กฎ Coding thinking ้ข่ฎพ๏ผ
llama-server \
-m Q5_LynnStyle/Qwen3.6-27B-Uncensored-Heretic-Q5_LynnStyle.gguf \
--mmproj Q5_LynnStyle/mmproj-Qwen3.6-27B-Q8_0.gguf \
--model-draft Q5_LynnStyle/Q8-MTP-draft.gguf \
--spec-type draft-mtp --spec-draft-n-max 3 \
-ngl 999 -c 8192 -b 1024 -ub 256 -np 1 \
--temp 0.6 --top-p 0.95 --top-k 20 --min-p 0.0 \
--presence-penalty 0.0 --repeat-penalty 1.0 \
--host 0.0.0.0 --port 8080 \
--jinja --reasoning on --reasoning-format deepseek
Q4 bareใnon-thinking ้ข่ฎพ๏ผ
llama-server \
-m Q4_LynnStyle/Qwen3.6-27B-Uncensored-Heretic-Q4_LynnStyle.gguf \
-ngl 999 -c 8192 -b 1024 -ub 256 -np 1 \
--temp 0.7 --top-p 0.80 --top-k 20 --min-p 0.0 \
--presence-penalty 1.5 --repeat-penalty 1.0 \
--host 0.0.0.0 --port 8080 \
--jinja --reasoning off
Q4 + ๅ จๆกฃๅ ฑ็จ Q8 MTP + Q8 ่ง่งๅก๏ผ็ฒพ็กฎ Coding thinking ้ข่ฎพ๏ผ
llama-server \
-m Q4_LynnStyle/Qwen3.6-27B-Uncensored-Heretic-Q4_LynnStyle.gguf \
--mmproj Q4_LynnStyle/mmproj-Qwen3.6-27B-Q8_0.gguf \
--model-draft Q4_LynnStyle/Q8-MTP-draft.gguf \
--spec-type draft-mtp --spec-draft-n-max 3 \
-ngl 999 -c 8192 -b 1024 -ub 256 -np 1 \
--temp 0.6 --top-p 0.95 --top-k 20 --min-p 0.0 \
--presence-penalty 0.0 --repeat-penalty 1.0 \
--host 0.0.0.0 --port 8080 \
--jinja --reasoning on --reasoning-format deepseek
ไธบไปไน Hugging Face ๅณไพงๅฏ่ฝไธๆพ็คบ้ๅๆ ผๅผ
Q5ใQ4ใQ3 LynnStyle GGUF ไฝฟ็จๆททๅ็ฒพๅบฆ tensor ๅธๅฑ๏ผๅไธไธชๆไปถไผๆ นๆฎๅฑไฝใattention patternใtensor family ไธ้่ฆๅบฆๆ กๅ็ปๆ๏ผๆๆ็ปๅๅค็ง้ๅ็ฑปๅใๅ ๆญค๏ผHugging Face ๅณไพงๅ
ๆฐๆฎๆ ๅฏ่ฝๆ ๆณๆๅฎๅฝ็บฆๆๅไธๆ ๅ้ๅๆ ็ญพ๏ผๆพ็คบไธบ็ฉบ็ฝๆๆ ๆณ่ฏๅซใ่ฟไธไปฃ่กจ GGUF ๆฒกๆ้ๅๆๆไปถ็ผบๅคฑใๆๅจ่ฏๅซไพๆฎๆฏๆกฃไฝ็ฎๅฝไธๆไปถๅ๏ผไปฅๅ GGUF general.file_typeใtensor ็ฑปๅๅๅธใmanifest.json ไธ SHA256SUMSใ
LynnStyle Dense
27B ๆฏ Dense๏ผๆฒกๆ MoE expert๏ผไฝไธไปฃ่กจๅ จๅฑๅ่ดจใQwen3.6 27B ๆๆททๅๆณจๆๅ็ปๆ๏ผfull-attention ๅฑ็บฆๆฏ 4 ๅฑๅบ็ฐไธๆฌก๏ผๅ ถไปๅฑๆดๅคๆฏ linear attention / SSMใLynnStyle Dense ็ไฟๆคๅไฝๆฏๅฑไฝใattention patternใtensor familyใ
Q5/Q4/Q3/Q2 ไธๆฏไธๅๅไฝๆฏ็น้ๅ๏ผ่ๆฏๆพๅผไฟๆคๅ ณ้ฎๅฑใfull-attentionใattention gateใSSM ๅ้่ฆ tensor family๏ผๅๅฏนๆฎ้ๅฑๆๆกฃไฝๅ็ผฉ๏ผๅนถๅ ๅ imatrix ๆ็ญไปท้่ฆๅบฆๆ กๅใๅชๅ direct quant ไธๅ ทๅคๅๅธ่ตๆ ผใ
ๅปๅฎกๆฅๅ Coding ๅบ็บฟ้จ็ฆ
- Q8๏ผMMLU500 92.4%๏ผGPQA198 81.82%๏ผLCB100 78/100ใ
- Q5๏ผMMLU500 92.2%๏ผGPQA198 78.79%๏ผLCB100 75/100ใ
- Q4๏ผMMLU500 92.0%๏ผGPQA198 74.75%๏ผLCB100 71/100ใ
- Q3๏ผMMLU500 91.4%๏ผGPQA198 77.27%๏ผLCB100 69/100ใ
่ฎญ็ปไธ็ benchmark ็ญๆกใSFT ๅญฆ่ฝจ่ฟน๏ผRL ๅญฆ่กไธบๅๅฅฝ๏ผMMLU / GPQA / LCB ไฝไธบ held-out ้จ็ฆใ
ๅปๅฎกๆฅๅ Coding ๅบ็บฟ๏ผQ4 / Q3 llama.cpp ่ฟ่ก็ฉ้ต
TPS ๅๆด๏ผๆฅๅ็ไปฅ็พๅๆฏๆพ็คบใC1/C2/C4 ่กจ็คบๅนถๅๆฐ๏ผ็ปๆๆฅ่ชๅไธๅฅ llama.cpp harnessใ
Q4 LynnStyle
| ๆจกๅผ | C1 | C2 | C4 |
|---|---|---|---|
| bare | 59 TPS | 76 TPS | 116 TPS |
| ๆไฝณ MTP | n=2๏ผ75 TPS๏ผ91%๏ผ | n=2๏ผ82 TPS๏ผ93%๏ผ | n=3๏ผ83 TPS๏ผ89%๏ผ |
ๆไฝณๆกฃไฝ๏ผC4 bare๏ผ116 TPSใไฝๅนถๅๅปถ่ฟไผๅ
ๆถ๏ผC1/C2 ไฝฟ็จ MTP n=2ใ
Q3 LynnStyle
| ๅนถๅ | bare | MTP n=2 | MTP n=3 | MTP n=4 | ๆจ่ |
|---|---|---|---|---|---|
| C1 | 15 TPS | 22 TPS๏ผ86%๏ผ | 23 TPS๏ผ84%๏ผ | 23 TPS๏ผ81%๏ผ | MTP n=3๏ผ่พ bare ็บฆ +53% |
| C2 | 28 TPS | 28 TPS๏ผ86%๏ผ | 28 TPS๏ผ84%๏ผ | 27 TPS๏ผ81%๏ผ | MTP n=2๏ผๅๆดๅไธ bare ๅ้ |
| C4 | 49 TPS | 29 TPS๏ผ86%๏ผ | 30 TPS๏ผ84%๏ผ | 30 TPS๏ผ81%๏ผ | bare |
Q3 ๅ่ฏทๆฑๅปถ่ฟไผๅ
ๆจ่ MTP n=3๏ผC2 ็ MTP ๆไผ้ๆฉไธบ n=2๏ผไฝๅๆดๅๅไธ bare ๆๅนณ๏ผC4 ๅๅไผๅ
ไฝฟ็จ bareใไธฅๆ ผ 16GB ๆ้ฟไธไธๆไนไผๅ
ๅชๅ ่ฝฝ Q3 ไธปๆจกๅใ
- Downloads last month
- 2,000

