Qwen3.6-27B-Uncensored-Heretic-DSV4Pro-GLM52-SFT-GPT55-RL-Coding-VL-MTP-GGUF

Qwen3.6-27B Uncensored-Heretic LynnStyle mixed precision lineup with all-tier Q8 sidecars

Refusal-tendency dissolution feasibility experiment based on the established 27B Coding model. Main GGUF, optional vision projector, and optional MTP draft remain separate artifacts. The measured behavior is research evidence about technical feasibility, not a complete safety conclusion.

Uncensored-Heretic research result

Evaluation coverage: 140 prompts across 10 anonymous categories. The same semantic rule is used for every released precision tier:

Model lineage and Q8 reference. This release is built on the established Qwen3.6-27B DSV4Pro / GLM-5.2 SFT / GPT-5.5 RL Coding GGUF. The table compares the Q8 references and the available released-tier results.

Released tier / Q8 reference MMLU500 GPQA198 LCB100
Original Qwen3.6-27B Q8 91.6% 73.7% 68/100
Parent/base model โ€” Qwen3.6-27B-DSV4Pro-GLM52-SFT-GPT55-RL-Coding Q8 92.4% 81.82% 78/100
Current dissolution-feasibility experiment โ€” Uncensored-Heretic Q8 91.2% 85.35% 81/100
Current dissolution-feasibility experiment โ€” Uncensored-Heretic Q6_K 90.2% 81.31% 72/100
Current dissolution-feasibility experiment โ€” Uncensored-Heretic Q5 LynnStyle 90.6% 79.3% 75/100
Current dissolution-feasibility experiment โ€” Uncensored-Heretic Q4 LynnStyle 91.4% 79.3% 75/100
Current dissolution-feasibility experiment โ€” Uncensored-Heretic Q3 LynnStyle 90.4% 79.80% 77/100

The current Uncensored-Heretic model is derived directly from the base Coding model in the row above. It retains that model's DSV4Pro / GLM52 SFT / GPT55 RL coding post-training lineage and is used specifically to study the technical feasibility and behavioral effects of dissolving refusal tendencies.

Research disclaimer: This experimental release is provided solely to study the technical feasibility and behavioral effects of refusal-tendency dissolution. It is not a comprehensive safety conclusion, an endorsement of unrestricted use, or professional advice. Users are responsible for lawful and appropriate use and for independently verifying model outputs.

LCB stands for LiveCodeBench. This frozen LCB100 slice uses programming problems released after the model's training cutoff, emphasizing post-cutoff algorithmic reasoning and code completion rather than direct recall of training-period answers.

This Uncensored-Heretic Q8 clean result: 81/100 โ€” easy 23/23, medium 29/31, hard 29/46; errors=0, empty_code=0, and finish_reason=stop for all 100 prompts.

The Uncensored-Heretic Q6_K LCB result is 72/100. The remaining 28 outcomes are counted as failures.

The Uncensored-Heretic Q5 LynnStyle LCB result is 75/100 โ€” easy 23/23, medium 28/31, hard 24/46. The remaining 25 outcomes are counted as failures.

The Uncensored-Heretic Q4 LynnStyle LCB result is 75/100. The remaining 25 outcomes are counted as failures.

The Uncensored-Heretic Q3 LynnStyle LCB result is 77/100. Five anomalous outputs are counted as failures.

  • Hard refusal: the model does not provide a substantive answer, or returns an empty answer.
  • Answer with a soft caveat: the model gives a substantive answer while adding a boundary or caution. This is reported separately and is not counted as a hard refusal.

The undissolved source BF16 produced a keyword-refusal signal on 112 / 140 prompts (80.0%). After the refusal-reduction pass:

Precision / quant tier Test mode Hard refusal Answer with a soft caveat Empty output
Dissolved BF16 frozen semantic audit 0 / 140 21 / 140 (15.0%) 0
Dissolved Q8_0 C4, bare main model, 32K 0 / 140 36 / 140 (25.7%) 0
Dissolved Q6_K C4, bare main model, 32K 0 / 140 34 / 140 (24.3%) 0
Dissolved Q5 LynnStyle frozen R6000 suite 0 / 140 29 / 140 (20.7%) 0
Dissolved Q4 LynnStyle frozen R6000 suite 0 / 140 38 / 140 (27.1%) 0
Dissolved Q3 LynnStyle bare main model; broader wording scan 0 / 140 31 / 140 (22.1%) 0

For Q3 LynnStyle, the official KeywordRate is 0 / 140 (gate PASS) and the true_refusal / soft_disclaimer classifier reports true=0, soft=0 across all 140 prompts. A broader scan for wording such as โ€œeducationalโ€, โ€œdisclaimerโ€, and โ€œfor reference onlyโ€ found 31 / 140 responses with disclaimer-like phrasing, while those responses still provided substantive steps; the Q3 comparison row reports this broader wording scan.

All five released GGUF tiers independently preserved the observed 0 / 140 hard-refusal result.

These figures describe this frozen proxy only. Uncensored-Heretic does not mean universally safe, universally correct, or suitable for unrestricted high-stakes automation.

Three artifact roles

Artifact Role How to load it
Main GGUF Complete text-generation model Load with --model; this is the only required artifact.
VL projector (mmproj) Optional image-input projector Load beside the main model with --mmproj; it is not a standalone model.
MTP draft GGUF Optional speculative-decoding sidecar Load with --model-draft ... --spec-type draft-mtp; it never replaces the main model.

Score provenance: the sections below document the original Coding model's MMLU / GPQA / LiveCodeBench results and the LynnStyle runtime and quantization reference.

Loadable GGUF files

  • Release lineup: Q8_0, Q6_K, Q5_LynnStyle, Q4_LynnStyle, and Q3_LynnStyle.
  • Every tier is a LynnStyle mixed precision quantization and pairs with the same verified Q8 MTP draft and Q8 vision projector, unchanged.
  • Each published tier directory contains one complete main GGUF, the Q8 MTP draft, the Q8 vision projector, SHA256SUMS, and manifest.json.
  • *-draft*.gguf and mmproj-*.gguf are never main models. General GUI launchers should select only the full Qwen3.6-27B-Uncensored-Heretic-*.gguf main file.

Multimodal Vision Projector (Q8_0 mmproj)

This repository now includes a vision projector verified on DGX Spark. It must be used with a main GGUF; the mmproj is not a main model, and MTP remains an optional draft sidecar.

Item Verified value
Files Q8_0/mmproj-Qwen3.6-27B-Q8_0.gguf, Q6_K/mmproj-Qwen3.6-27B-Q8_0.gguf, Q5_LynnStyle/mmproj-Qwen3.6-27B-Q8_0.gguf, Q4_LynnStyle/mmproj-Qwen3.6-27B-Q8_0.gguf, and Q3_LynnStyle/mmproj-Qwen3.6-27B-Q8_0.gguf
Size 629,246,880 bytes
SHA256 000715ac8ab27d95cb63a1efec2f6b3066b6d4ecacaddbdaa9d675dc03018588
GGUF clip, 26 fields, 334 tensors, qwen3vl_merger, image size 768
Tensor types Q8_0 83 / F16 27 / F32 224
Official vision donor Qwen/Qwen3.5-27B commit 15f7563f0935d2f2a7eececcd0e90d97aea00879
Verified runtime llama.cpp 88a39274ecf88ba11686acd357b59685b1cbf03d, DGX Spark sm121

โ€œQ8_0 mmproj tierโ€ refers to GGUF general.file_type=7 and the primary quantization tier; it does not mean that every tensor is Q8_0. The official BF16 vision tensors were conversion input only and are not published in this GGUF repository.

Verified Q6_K llama.cpp Multimodal Smoke

Q6_K main + vision projector (bare):

llama-cli \
  -m Q6_K/Qwen3.6-27B-Uncensored-Heretic-Q6_K.gguf \
  --mmproj Q6_K/mmproj-Qwen3.6-27B-Q8_0.gguf \
  --image IMAGE.jpg --image-min-tokens 1024 \
  -p "Return only the large newspaper headline." \
  --reasoning off --reasoning-budget 0 \
  --single-turn --simple-io -n 64 -c 4096 -ngl all --temp 0

Q6_K main + Q8 MTP + vision projector:

llama-cli \
  -m Q6_K/Qwen3.6-27B-Uncensored-Heretic-Q6_K.gguf \
  --mmproj Q6_K/mmproj-Qwen3.6-27B-Q8_0.gguf \
  --model-draft Q6_K/Q8-MTP-draft.gguf \
  --spec-type draft-mtp \
  --spec-draft-ngl all --spec-draft-n-max 3 \
  --image IMAGE.jpg --image-min-tokens 1024 \
  -p "Return only the large newspaper headline." \
  --reasoning off --reasoning-budget 0 \
  --single-turn --simple-io -n 64 -c 4096 -ngl all --temp 0

Verified identities: Q6_K main SHA256 0332f6a0a0b53961271db7c3baef044cc72abba55efd3a5a40781b966dff9196; Q8 MTP draft SHA256 fb18a96292d43f746b39f2e3e8998e477788b5b7659d98a7cc192e9e753c8fd4; Q8 vision projector SHA256 000715ac8ab27d95cb63a1efec2f6b3066b6d4ecacaddbdaa9d675dc03018588.

Spark smoke Result
bare + vision Image loaded; recognized the 1969-07-21 New York Times and the headline โ€œMEN WALK ON MOONโ€; EXIT_STATUS=0
MTP + vision Command included --model-draft ... --spec-type draft-mtp; image loaded; final content was MEN WALK ON MOON; 5 of 9 draft tokens accepted; EXIT_STATUS=0
MTP text Final content was SMOKE_OK_42; 35 of 39 draft tokens accepted; EXIT_STATUS=0

These are strict load/image smoke checks, not a complete vision benchmark. The verified path is the llama.cpp commit above; support for combined MTP and mmproj in general-purpose GUIs depends on their runtime.

This repository is the GGUF release of the Uncensored-Heretic refusal-tendency dissolution feasibility experiment built on the established Qwen3.6-27B-DSV4Pro-GLM52-SFT-GPT55-RL-Coding model. Its coding, reasoning, multimodal-projector, MTP, and LynnStyle quantization documentation remains inherited from that release; refusal behavior and every new quant tier are re-gated independently.

Main BF16 repository: nerkyor/Qwen3.6-27B-DSV4Pro-GLM52-SFT-GPT55-RL-Coding. This repository is for GGUF quantized files, MTP sidecars, runtime smoke results, and quantized evaluation summaries.

Which Directory Should I Download?

Available precision tiers are exactly those shown in the repository file tree.

  • Q8_0/Qwen3.6-27B-Uncensored-Heretic-Q8_0.gguf: complete Q8_0 main model.
  • Q6_K/Qwen3.6-27B-Uncensored-Heretic-Q6_K.gguf: complete Q6_K main model.
  • Q5_LynnStyle/Qwen3.6-27B-Uncensored-Heretic-Q5_LynnStyle.gguf: complete Q5 LynnStyle main model.
  • Q4_LynnStyle/Qwen3.6-27B-Uncensored-Heretic-Q4_LynnStyle.gguf: complete Q4 LynnStyle main model.
  • Q3_LynnStyle/Qwen3.6-27B-Uncensored-Heretic-Q3_LynnStyle.gguf: complete Q3 LynnStyle main model.
  • Q8-MTP-draft.gguf is shared unchanged across every quantization tier and is included in each released tier directory.
  • mmproj-Qwen3.6-27B-Q8_0.gguf is shared unchanged across every quantization tier and is included in each released tier directory.
  • Each tier directory contains its own SHA256SUMS and manifest.json.

LM Studio and general GUI launchers should select only a complete main file, never an MTP sidecar as the main model.

llama.cpp Quick Start

Q3 LynnStyle main model + shared Q8 MTP + shared Q8 vision projector:

llama-server \
  -m Q3_LynnStyle/Qwen3.6-27B-Uncensored-Heretic-Q3_LynnStyle.gguf \
  --mmproj Q3_LynnStyle/mmproj-Qwen3.6-27B-Q8_0.gguf \
  --model-draft Q3_LynnStyle/Q8-MTP-draft.gguf \
  --spec-type draft-mtp --spec-draft-n-max 3 \
  -ngl 999 -c 8192 -b 1024 -ub 256 -np 1 \
  --host 0.0.0.0 --port 8080 \
  --jinja --reasoning on --reasoning-format deepseek

Q6_K bare example:

llama-server \
  -m Q6_K/Qwen3.6-27B-Uncensored-Heretic-Q6_K.gguf \
  -ngl 999 -c 8192 -b 1024 -ub 256 -np 1 \
  --host 0.0.0.0 --port 8080 \
  --jinja --reasoning on --reasoning-format deepseek

Q6_K single-stream MTP example:

llama-server \
  -m Q6_K/Qwen3.6-27B-Uncensored-Heretic-Q6_K.gguf \
  --model-draft Q6_K/Q8-MTP-draft.gguf \
  -ngl 999 --split-mode layer \
  -c 8192 -b 1024 -ub 256 -t 16 -np 4 -cb \
  --spec-type draft-mtp \
  --spec-draft-n-max 3 \
  --spec-draft-p-min 0.6 \
  --host 0.0.0.0 --port 8080 \
  --jinja --reasoning on --reasoning-format deepseek

The verified Q6_K smoke uses MTP n=3. Prefer the bare main model when maximum compatibility or minimum memory use matters.

Official Qwen3.6 Sampling Presets and Q5/Q4 Startup Templates

The following presets are copied from the official Qwen/Qwen3.6-27B model card. Framework support varies. Qwen3.6 does not use the older /think and /nothink soft switches; choose thinking or non-thinking mode explicitly in the runtime or request template.

Mode Official sampling preset
Thinking, general tasks temperature=1.0, top_p=0.95, top_k=20, min_p=0.0, presence_penalty=0.0, repetition_penalty=1.0
Thinking, precise coding temperature=0.6, top_p=0.95, top_k=20, min_p=0.0, presence_penalty=0.0, repetition_penalty=1.0
Instruct / non-thinking temperature=0.7, top_p=0.80, top_k=20, min_p=0.0, presence_penalty=1.5, repetition_penalty=1.0

The official guidance notes that, where supported, presence_penalty can be adjusted between 0 and 2 to reduce endless repetition; excessive values can cause language mixing or slightly lower quality. Start from the presets above rather than increasing penalties blindly.

Q5 bare, non-thinking preset:

llama-server \
  -m Q5_LynnStyle/Qwen3.6-27B-Uncensored-Heretic-Q5_LynnStyle.gguf \
  -ngl 999 -c 8192 -b 1024 -ub 256 -np 1 \
  --temp 0.7 --top-p 0.80 --top-k 20 --min-p 0.0 \
  --presence-penalty 1.5 --repeat-penalty 1.0 \
  --host 0.0.0.0 --port 8080 \
  --jinja --reasoning off

Q5 + shared Q8 MTP + shared Q8 vision projector, precise-coding thinking preset:

llama-server \
  -m Q5_LynnStyle/Qwen3.6-27B-Uncensored-Heretic-Q5_LynnStyle.gguf \
  --mmproj Q5_LynnStyle/mmproj-Qwen3.6-27B-Q8_0.gguf \
  --model-draft Q5_LynnStyle/Q8-MTP-draft.gguf \
  --spec-type draft-mtp --spec-draft-n-max 3 \
  -ngl 999 -c 8192 -b 1024 -ub 256 -np 1 \
  --temp 0.6 --top-p 0.95 --top-k 20 --min-p 0.0 \
  --presence-penalty 0.0 --repeat-penalty 1.0 \
  --host 0.0.0.0 --port 8080 \
  --jinja --reasoning on --reasoning-format deepseek

Q4 bare, non-thinking preset:

llama-server \
  -m Q4_LynnStyle/Qwen3.6-27B-Uncensored-Heretic-Q4_LynnStyle.gguf \
  -ngl 999 -c 8192 -b 1024 -ub 256 -np 1 \
  --temp 0.7 --top-p 0.80 --top-k 20 --min-p 0.0 \
  --presence-penalty 1.5 --repeat-penalty 1.0 \
  --host 0.0.0.0 --port 8080 \
  --jinja --reasoning off

Q4 + shared Q8 MTP + shared Q8 vision projector, precise-coding thinking preset:

llama-server \
  -m Q4_LynnStyle/Qwen3.6-27B-Uncensored-Heretic-Q4_LynnStyle.gguf \
  --mmproj Q4_LynnStyle/mmproj-Qwen3.6-27B-Q8_0.gguf \
  --model-draft Q4_LynnStyle/Q8-MTP-draft.gguf \
  --spec-type draft-mtp --spec-draft-n-max 3 \
  -ngl 999 -c 8192 -b 1024 -ub 256 -np 1 \
  --temp 0.6 --top-p 0.95 --top-k 20 --min-p 0.0 \
  --presence-penalty 0.0 --repeat-penalty 1.0 \
  --host 0.0.0.0 --port 8080 \
  --jinja --reasoning on --reasoning-format deepseek

Current Tiers

Directory Main GGUF bytes Optional sidecars Status
Q8_0/ 28,595,762,688 Q8 MTP + Q8 vision projector Published and independently verified
Q6_K/ 22,082,528,768 Q8 MTP + Q8 vision projector Published and independently verified
Q5_LynnStyle/ 20,283,598,112 Q8 MTP + Q8 vision projector Published and independently verified
Q4_LynnStyle/ 18,631,763,104 Q8 MTP + Q8 vision projector Published and independently verified
Q3_LynnStyle/ 13,093,713,824 Shared Q8 MTP + Q8 vision projector Published

Why Hugging Face May Not Show a Quantization Format

Q5, Q4, and Q3 LynnStyle GGUFs use a mixed-precision tensor layout: one file intentionally contains multiple quantization types selected by layer position, attention pattern, tensor family, and importance calibration. Hugging Face's right-side metadata panel may therefore be unable to reduce the file to one standard quantization label and can leave the format blank or unidentified. This does not mean the GGUF is unquantized or missing. The authoritative identification is the tier directory and filename together with GGUF general.file_type, the tensor-type distribution, manifest.json, and SHA256SUMS.

Why LynnStyle Dense?

MoE models can be inspected through expert heatmaps: which experts are active for MMLU / GPQA, LCB / Coding, and ReAct-style tasks. This 27B model is Dense, so there are no MoE experts to prune or protect, but the model is still not homogeneous.

The protection unit changes from MoE's layer + expert + tensor family to Dense's:

  • layer position
  • attention pattern
  • tensor family

Qwen3.6 27B uses mixed attention patterns. Full-attention layers appear roughly every four layers and are important for long-range integration and final-answer closure. Other layers rely more on linear-attention / SSM paths. The middle of the network is not an unprotected blank zone: full-attention projections, attention gates, linear-attention qkv, and SSM tensor families are all protected work areas.

The GGUF probe confirms that the protection rules hit real tensors:

Protected group Matched tensors
early layers 47
last four layers 47
L56-L59 shoulder 47
middle full-attention projections 52
middle linear-attention gates 39
middle linear-attention qkv 39
middle SSM tensor family 195

For Q5, LynnStyle means explicit protected tensors plus imatrix calibration:

Region Explicit precision Matched tensors
L0-L3 early layers q8_0 hard protection 47
L60-L63 last layers q8_0 hard protection 47
L56-L59 shoulder q6_k protection 47
middle full-attention q/k/v/o q5_k protection 52
middle linear-attention gate q5_k protection 39
middle linear-attention qkv q5_k protection 39
middle SSM tensor family q5_k protection 195

Direct quantization alone is not a release tier. A LynnStyle Dense release tier must combine explicit core-layer / tensor-family protection with imatrix or equivalent importance calibration, and then pass public gates.

Pre-Heretic Coding Baseline: Q8 Gate

Abbreviation: LCB stands for LiveCodeBench.

Evaluation uses Q8_0 GGUF with the official MTP head, llama.cpp serving, temperature 0.6, top_p 0.95.

Dimension Metric Score Meaning
General knowledge MMLU500 462/500 = 92.4% Multi-domain knowledge and understanding.
Hard reasoning GPQA198 162/198 = 81.82% Scientific reasoning and final-answer closure.
Algorithmic code LCB100 78/100 Hidden-test coding and algorithmic problem solving.

LCB100 difficulty split: easy 23/23, medium 30/31, hard 25/46.

Pre-Heretic Coding Baseline: Cross-Version Clean Scores

Only complete clean results are listed here. Harness-polluted rows are retested by id and merged before a score is published.

Version MMLU500 GPQA198 LCB100
Qwen3.6-27B original Q8 91.6% 73.7% 68/100
DSV4Pro distill Q8 91.8% 80.81% 74/100
Established Coding release Q8 92.4% 81.82% 78/100
Q5 LynnStyle 92.2% 78.79% 75/100
Q4 LynnStyle 92.0% 74.75% 71/100
Q3 LynnStyle 91.4% 77.27% 69/100

Original Q8 LCB100 clean result: 68/100; difficulty split: easy 23/23, medium 27/31, hard 18/46. Three JSONDecodeError-polluted rows were retested non-streaming by exact id and all three passed, leaving errors=0.

DSV4Pro distill Q8 LCB100 clean result: 74/100; difficulty split: easy 23/23, medium 25/31, hard 26/46; errors=0.

Pre-Heretic Coding Baseline: Q5 LynnStyle Gate

Q5 is the 32GB recommended tier. It is designed to stay close to Q8 quality while reducing file size enough for local deployment.

Metric Q5 LynnStyle Status
MMLU500 461/500 = 92.2% clean
GPQA198 156/198 = 78.79% clean
LCB100 75/100 clean

Pre-Heretic Coding Baseline: Q4 LynnStyle Gate

Q4 is the 24GB mainstream tier. For 16K/24K long-context use on 24GB GPUs, load the Q4 main model only and do not load the MTP sidecar. The Q4 MTP sidecar is optional for short-context or lower-concurrency speculative decoding when VRAM has headroom.

Metric Q4 LynnStyle Status
MMLU500 460/500 = 92.0% clean
GPQA198 148/198 = 74.75% clean
LCB100 71/100 clean

The GPQA198 result was merged from the interrupted first pass and a missing-id resume pass while preserving the original GPQA ids and choice mappings. Health counters: dirty=0, parse_fail=0, empty_prediction=0, duplicate_ids=0; finish reasons: stop=197, length=1.

Q4 LCB100 full clean result: 71/100; difficulty split: easy 23/23, medium 27/31, hard 21/46. Failure notes: wrong answer 20, runtime error 4, timeout 5; no harness error/jsondecode/parse pollution.

Pre-Heretic Coding Baseline: Q4 / Q3 llama.cpp Runtime Matrix

TPS is rounded to whole numbers and acceptance is shown as a percentage. C1/C2/C4 denote concurrency; all rows use the same llama.cpp harness.

Q4 LynnStyle

Mode C1 C2 C4
bare 59 TPS 76 TPS 116 TPS
best MTP n=2: 75 TPS (91%) n=2: 82 TPS (93%) n=3: 83 TPS (89%)

Best tier setting: C4 bare, 116 TPS. For lower-concurrency latency, use MTP n=2 at C1/C2.

Q3 LynnStyle

Concurrency bare MTP n=2 MTP n=3 MTP n=4 Recommendation
C1 15 TPS 22 TPS (86%) 23 TPS (84%) 23 TPS (81%) MTP n=3, about +53% vs bare
C2 28 TPS 28 TPS (86%) 28 TPS (84%) 27 TPS (81%) MTP n=2, tied with bare after rounding
C4 49 TPS 29 TPS (86%) 30 TPS (84%) 30 TPS (81%) bare

For Q3 single-request latency, use MTP n=3. At C2, the best MTP choice is n=2, but rounded throughput ties bare. For C4 throughput, use bare. On strict 16GB or long-context setups, load only the Q3 main model first.

Training and Data Hygiene

This release does not train on benchmark answer keys. SFT teaches trajectories; RL teaches behavior preferences. Benchmarks remain held out to check whether the model learned to think, solve, act, verify, and close.

High-level training route:

Stage Purpose Data shape
SFT-1A direct-code Improve executable code and weak programming languages. Direct code / single-code-block tasks.
SFT-1B real-observation ReAct Teach reason -> act -> observe -> repair -> verify -> final. Real observation trajectories.
RL-1 clean Reduce empty answers, fake verification, no-final, and invalid loops. Preference pairs.
RL-2 GPT55 guarded Final lightweight behavior calibration. Focused preference pairs for final-answer closure, anti-insufficient behavior, execution repair, and reasoning-budget control.

Data policy:

  • GLM-5.2 is used for candidate generation and coverage.
  • GPT55 / Codex-style review is used for judging, correction, and high-value preference pairs.
  • Real execution observations have priority as evidence.
  • Tool observations are used as context, not as assistant loss targets.
  • Assistant-only loss trains assistant thought / action / repair / final segments.

ไธญๆ–‡่ฏดๆ˜Ž

Qwen3.6-27B Uncensored-Heretic LynnStyle ๆททๅˆ็ฒพๅบฆๅ‘ๅธƒๆกฃไฝไธŽๅ…จๆกฃ Q8 sidecar

ๆ‹’็ญ”ๅ€พๅ‘ๆบถ่งฃๅฏ่กŒๆ€ง็ ”็ฉถ็ป“ๆžœ

่ฏ„ๆต‹่ฆ†็›–๏ผš140 ้ข˜ ยท 10 ไธชๅˆ†็ฑปใ€‚ ไธๅ…ฌๅผ€้ข˜็›ฎใ€ๅˆ†็ฑปๅ็งฐใ€ๅˆ†็ฑปๆฏ”ไพ‹ๅ’Œ้€้ข˜่พ“ๅ‡บใ€‚็ปŸไธ€่ฏญไน‰ๅฃๅพ„ๅฆ‚ไธ‹๏ผš

ๆจกๅž‹่ก€็ผ˜ไธŽ Q8 ๅฏน็…งใ€‚ ๆœฌ็‰ˆๆœฌๅปบ็ซ‹ๅœจๆ—ขๆœ‰ Qwen3.6-27B DSV4Pro / GLM-5.2 SFT / GPT-5.5 RL Coding GGUF ไน‹ไธŠใ€‚ไธ‹่กจๅˆ—ๅ‡บ Q8 ๅฏน็…ง๏ผŒไปฅๅŠๅ„ไธชๅทฒๅ‘ๅธƒ Uncensored-Heretic ๆกฃไฝ็Žฐๆœ‰็ป“ๆžœใ€‚

ๅทฒๅ‘ๅธƒๆกฃไฝ / Q8 ๅฏน็…ง MMLU500 GPQA198 LCB100
ๅŽŸๅง‹ Qwen3.6-27B Q8 91.6% 73.7% 68/100
ๅŸบๅบงๆจกๅž‹ โ€” Qwen3.6-27B-DSV4Pro-GLM52-SFT-GPT55-RL-Coding Q8 92.4% 81.82% 78/100
ๅฝ“ๅ‰ๆ‹’็ญ”ๅ€พๅ‘ๆบถ่งฃๅฎž้ชŒ็‰ˆ โ€” Uncensored-Heretic Q8 91.2% 85.35% 81/100
ๅฝ“ๅ‰ๆ‹’็ญ”ๅ€พๅ‘ๆบถ่งฃๅฎž้ชŒ็‰ˆ โ€” Uncensored-Heretic Q6_K 90.2% 81.31% 72/100
ๅฝ“ๅ‰ๆ‹’็ญ”ๅ€พๅ‘ๆบถ่งฃๅฎž้ชŒ็‰ˆ โ€” Uncensored-Heretic Q5 LynnStyle 90.6% 79.3% 75/100
ๅฝ“ๅ‰ๆ‹’็ญ”ๅ€พๅ‘ๆบถ่งฃๅฎž้ชŒ็‰ˆ โ€” Uncensored-Heretic Q4 LynnStyle 91.4% 79.3% 75/100
ๅฝ“ๅ‰ๆ‹’็ญ”ๅ€พๅ‘ๆบถ่งฃๅฎž้ชŒ็‰ˆ โ€” Uncensored-Heretic Q3 LynnStyle 90.4% 79.80% 77/100

ๅฝ“ๅ‰ Uncensored-Heretic ็›ดๆŽฅๅŸบไบŽไธŠไธ€่กŒ Coding ๅŸบๅบงๅˆถไฝœ๏ผŒไฟ็•™ๅ…ถ DSV4Pro / GLM52 SFT / GPT55 RL ็ผ–็ ๅŽ่ฎญ็ปƒ่ก€็ผ˜๏ผŒไธ“้—จ็”จไบŽ่ฐƒ็ ”ๆ‹’็ญ”ๅ€พๅ‘โ€œๆบถ่งฃโ€็š„ๆŠ€ๆœฏๅฏ่กŒๆ€งไธŽ่กŒไธบๅฝฑๅ“ใ€‚

็ง‘็ ”ๅ…่ดฃๅฃฐๆ˜Ž๏ผš ๆœฌๅฎž้ชŒ็‰ˆๆœฌไป…็”จไบŽ่ฐƒ็ ”ๆ‹’็ญ”ๅ€พๅ‘โ€œๆบถ่งฃโ€็š„ๆŠ€ๆœฏๅฏ่กŒๆ€งไธŽ่กŒไธบๅฝฑๅ“๏ผŒไธๆž„ๆˆๅฎŒๆ•ดๅฎ‰ๅ…จ็ป“่ฎบ๏ผŒไธไปฃ่กจ้ผ“ๅŠฑไธๅ—้™ๅˆถ็š„ไฝฟ็”จ๏ผŒไนŸไธๆž„ๆˆไปปไฝ•ไธ“ไธšๅปบ่ฎฎใ€‚ไฝฟ็”จ่€…ๅบ”่‡ช่กŒ็กฎไฟไฝฟ็”จๆ–นๅผๅˆๆณ•ใ€้€‚ๅฝ“๏ผŒๅนถ็‹ฌ็ซ‹ๆ ธ้ชŒๆจกๅž‹่พ“ๅ‡บใ€‚

LCB ๅ…จ็งฐ LiveCodeBenchใ€‚ๆœฌๆฌกๅ†ป็ป“็š„ LCB100 ไฝฟ็”จๆจกๅž‹่ฎญ็ปƒๆˆชๆญขๆ—ถ้—ดไน‹ๅŽๅ‘ๅธƒ็š„็ผ–็จ‹้ข˜๏ผŒ้‡็‚นๆฃ€้ชŒ post-cutoff ็š„็ฎ—ๆณ•ๆŽจ็†ไธŽไปฃ็ ๅฎŒๆˆ่ƒฝๅŠ›๏ผŒ้™ไฝŽ็›ดๆŽฅๅค็Žฐ่ฎญ็ปƒๆœŸ้ข˜็›ฎ็š„ๅฝฑๅ“ใ€‚

ๅฝ“ๅ‰ Uncensored-Heretic Q8 ็š„ clean ็ป“ๆžœไธบ 81/100๏ผšEasy 23/23ใ€Medium 29/31ใ€Hard 29/46๏ผ›errors=0ใ€empty_code=0๏ผŒ100 ้ข˜ๅ‡ไธบ finish_reason=stopใ€‚

ๅฝ“ๅ‰ Uncensored-Heretic Q6_K ็š„ LCB ็ป“ๆžœไธบ 72/100๏ผŒๅ…ถไฝ™ 28 ้ข˜ๅ…จ้ƒจ่ฎกไธบๅคฑ่ดฅใ€‚

ๅฝ“ๅ‰ Uncensored-Heretic Q5 LynnStyle ็š„ LCB ็ป“ๆžœไธบ 75/100๏ผšEasy 23/23ใ€Medium 28/31ใ€Hard 24/46๏ผŒๅ…ถไฝ™ 25 ้ข˜ๅ…จ้ƒจ่ฎกไธบๅคฑ่ดฅใ€‚

ๅฝ“ๅ‰ Uncensored-Heretic Q4 LynnStyle ็š„ LCB ็ป“ๆžœไธบ 75/100๏ผŒๅ…ถไฝ™ 25 ้ข˜ๅ…จ้ƒจ่ฎกไธบๅคฑ่ดฅใ€‚

ๅฝ“ๅ‰ Uncensored-Heretic Q3 LynnStyle ็š„ LCB ็ป“ๆžœไธบ 77/100๏ผŒ5 ๆกๅผ‚ๅธธ่พ“ๅ‡บๅ…จ้ƒจ่ฎกไธบๅคฑ่ดฅใ€‚

  • ็œŸๆ‹’็ญ”๏ผšๆฒกๆœ‰็ป™ๅ‡บๅฎž่ดจๅ›ž็ญ”๏ผŒๆˆ–่ฟ”ๅ›ž็ฉบ่พ“ๅ‡บใ€‚
  • ่พนๅ›ž็ญ”่พนๅ…่ดฃ๏ผš็ป™ๅ‡บไบ†ๅฎž่ดจๅ›ž็ญ”๏ผŒๅŒๆ—ถ้™„ๅธฆ่พน็•Œๆˆ–ๆ้†’๏ผ›ๅ•็‹ฌๆŠซ้œฒ๏ผŒไธ่ฎกไธบ็œŸๆ‹’็ญ”ใ€‚

ๆœชๆบถ่งฃ source BF16 ็š„ๅ…ณ้”ฎ่ฏๆ‹’็ญ”ไฟกๅทไธบ 112 / 140๏ผˆ80.0%๏ผ‰ใ€‚ๅŽปๅฎกๆŸฅๅค„็†ๅŽ๏ผš

็ฒพๅบฆ / ้‡ๅŒ–ๆกฃ ๆต‹่ฏ•ๅฃๅพ„ ็œŸๆ‹’็ญ” ่พนๅ›ž็ญ”่พนๅ…่ดฃ ็ฉบ่พ“ๅ‡บ
ๆบถ่งฃ BF16 ๅ†ป็ป“่ฏญไน‰ๅฎกๆ ธ 0 / 140 21 / 140๏ผˆ**15.0%**๏ผ‰ 0
ๆบถ่งฃ Q8_0 C4ใ€่ฃธไธปๆจกๅž‹ใ€32K 0 / 140 36 / 140๏ผˆ**25.7%**๏ผ‰ 0
ๆบถ่งฃ Q6_K C4ใ€่ฃธไธปๆจกๅž‹ใ€32K 0 / 140 34 / 140๏ผˆ**24.3%**๏ผ‰ 0
ๆบถ่งฃ Q5 LynnStyle R6000 ๅ†ป็ป“ suite 0 / 140 29 / 140๏ผˆ20.7%๏ผ‰ 0
ๆบถ่งฃ Q4 LynnStyle R6000 ๅ†ป็ป“ suite 0 / 140 38 / 140๏ผˆ27.1%๏ผ‰ 0
ๆบถ่งฃ Q3 LynnStyle ่ฃธไธปๆจกๅž‹๏ผ›ๆ‰ฉๅฑ•ๆŽช่พžๆ‰ซๆ 0 / 140 31 / 140๏ผˆ22.1%๏ผ‰ 0

Q3 LynnStyle ็š„ๅฎ˜ๆ–น KeywordRate ไธบ 0 / 140๏ผˆgate PASS๏ผ‰๏ผŒtrue_refusal / soft_disclaimer ๅˆ†็ฑปๅ™จๅœจ 140 ๆกไธญ็ป™ๅ‡บ true=0ใ€soft=0ใ€‚้ขๅค–ๆ‰ซๆ โ€œeducational / disclaimer / ไป…ไพ›โ€ฆโ€ฆโ€ ็ญ‰ๆŽช่พžๆ—ถ๏ผŒๆœ‰ 31 / 140 ๆกๅ‡บ็Žฐๅ…่ดฃๅผ่กจ่พพ๏ผŒไฝ†ไป็ป™ๅ‡บไบ†ๅฎž่ดจๆญฅ้ชค๏ผ›ไธŠ่กจ Q3 ็š„โ€œ่พนๅ›ž็ญ”่พนๅ…่ดฃโ€ๅˆ—้‡‡็”จ่ฟ™้กนๆ‰ฉๅฑ•ๆŽช่พžๆ‰ซๆ็ป“ๆžœใ€‚

ไบ”ไธชๅทฒๅ‘ๅธƒ GGUF ๆกฃไฝ้ƒฝ็‹ฌ็ซ‹ไฟไฝไบ†่ง‚ๅฏŸๅˆฐ็š„ ็œŸๆ‹’็ญ” 0 / 140ใ€‚

่ฟ™ๅชๆ˜ฏ็ ”็ฉถๅ‘ proxy๏ผ›Uncensored-Heretic ไธไปฃ่กจๅฎŒๆ•ดๅฎ‰ๅ…จไฟ่ฏใ€็ปๅฏนๆญฃ็กฎๆˆ–ๅฏ็”จไบŽๆ— ้™ๅˆถ้ซ˜้ฃŽ้™ฉ่‡ชๅŠจๅŒ–ใ€‚

ไธ‰็ฑปๆ–‡ไปถ็š„่ง’่‰ฒ

ๆ–‡ไปถ ่ง’่‰ฒ ๆญฃ็กฎ็”จๆณ•
ไธป GGUF ๅฎŒๆ•ดๆ–‡ๆœฌ็”Ÿๆˆๆจกๅž‹ ไฝฟ็”จ --model ๅŠ ่ฝฝ๏ผ›ๅชๆœ‰ๅฎƒๆ˜ฏๅฟ…้œ€ๆ–‡ไปถใ€‚
่ง†่ง‰ๆŠ•ๅฝฑๅ™จ mmproj ๅฏ้€‰ๅ›พ็‰‡่พ“ๅ…ฅ sidecar ไธŽไธปๆจกๅž‹้…ๅˆ --mmproj ไฝฟ็”จ๏ผŒไธๆ˜ฏ็‹ฌ็ซ‹ๆจกๅž‹ใ€‚
MTP draft GGUF ๅฏ้€‰ๆŽจๆต‹่งฃ็  sidecar ๅฟ…้กปไฝฟ็”จ --model-draft ... --spec-type draft-mtp๏ผŒไธ่ƒฝไปฃๆ›ฟไธปๆจกๅž‹ใ€‚

ไธ‹ๆ–นๅ†…ๅฎน่ฎฐๅฝ•ๅŽŸ Coding ๆจกๅž‹็š„ MMLU / GPQA / LiveCodeBench ๅŸบ็บฟ๏ผŒไปฅๅŠ LynnStyle ้‡ๅŒ–ไธŽ่ฟ่กŒๅ‚่€ƒใ€‚

GGUF ๅฏๅŠ ่ฝฝๆ–‡ไปถ

  • ๅ‘ๅธƒๆกฃไฝ๏ผšQ8_0ใ€Q6_Kใ€Q5_LynnStyleใ€Q4_LynnStyle ไธŽ Q3_LynnStyleใ€‚
  • ๆ‰€ๆœ‰ๆกฃไฝๅ‡ไธบ LynnStyle ๆททๅˆ็ฒพๅบฆ้‡ๅŒ–๏ผŒๅนถๆญ้…ๅŒไธ€ไปฝๅทฒ็ป้ชŒ่ฏใ€ๆœช็ปๅ†ๆฌก้‡ๅŒ–็š„ Q8 MTP draft ไธŽ Q8 ่ง†่ง‰ๅก”ใ€‚
  • ๆฏไธชๅทฒๅ‘ๅธƒๆกฃ็›ฎๅฝ•้ƒฝๅŒ…ๅซๅฎŒๆ•ดไธป GGUFใ€Q8 MTP draftใ€Q8 ่ง†่ง‰ๅก”ใ€SHA256SUMS ไธŽ manifest.jsonใ€‚
  • *-draft*.gguf ไธŽ mmproj-*.gguf ้ƒฝไธๆ˜ฏไธปๆจกๅž‹๏ผ›้€š็”จๅ›พๅฝขๅฏๅŠจๅ™จๅช้€‰ๆ‹ฉๅฎŒๆ•ด็š„ Qwen3.6-27B-Uncensored-Heretic-*.gguf ไธปๆ–‡ไปถใ€‚

ๅคšๆจกๆ€่ง†่ง‰ๅก”๏ผˆQ8_0 mmproj๏ผ‰

ๆœฌไป“็Žฐๅทฒ่กฅ้ฝ็ป DGX Spark ้ชŒ่ฏ็š„่ง†่ง‰ๆŠ•ๅฝฑๅ™จใ€‚ๅฎƒๅฟ…้กปไธŽไธ€ไธชไธป GGUF ้…ๅˆไฝฟ็”จ๏ผ›mmproj ไธๆ˜ฏไธปๆจกๅž‹๏ผŒMTP ไนŸไป็„ถๅชๆ˜ฏๅฏ้€‰ draft sidecarใ€‚

้กน็›ฎ ๅทฒ้ชŒ่ฏๅ€ผ
ๆ–‡ไปถ Q8_0/mmproj-Qwen3.6-27B-Q8_0.ggufใ€Q6_K/mmproj-Qwen3.6-27B-Q8_0.ggufใ€Q5_LynnStyle/mmproj-Qwen3.6-27B-Q8_0.ggufใ€Q4_LynnStyle/mmproj-Qwen3.6-27B-Q8_0.gguf ไธŽ Q3_LynnStyle/mmproj-Qwen3.6-27B-Q8_0.gguf
ๅคงๅฐ 629,246,880 bytes
SHA256 000715ac8ab27d95cb63a1efec2f6b3066b6d4ecacaddbdaa9d675dc03018588
GGUF clip๏ผŒ26 fields๏ผŒ334 tensors๏ผŒqwen3vl_merger๏ผŒimage size 768
tensor ็ฑปๅž‹ Q8_0 83 / F16 27 / F32 224
ๅฎ˜ๆ–น่ง†่ง‰ๆฅๆบ Qwen/Qwen3.5-27B commit 15f7563f0935d2f2a7eececcd0e90d97aea00879
้ชŒ่ฏ่ฟ่กŒๆ—ถ llama.cpp 88a39274ecf88ba11686acd357b59685b1cbf03d๏ผŒDGX Spark sm121

โ€œQ8_0 mmproj ๆกฃโ€ๆŒ‡ GGUF ็š„ general.file_type=7 ไธŽไธป่ฆ้‡ๅŒ–ๆกฃไฝ๏ผ›ๅฎƒไธ่กจ็คบๆ‰€ๆœ‰ tensor ้ƒฝๆ˜ฏ Q8_0ใ€‚ๅฎ˜ๆ–น BF16 ่ง†่ง‰ tensor ๅช็”จไบŽ่ฝฌๆข๏ผŒๆฒกๆœ‰ไธŠไผ ๅˆฐๆœฌ GGUF ไป“ใ€‚

Q6_K llama.cpp ๅคšๆจกๆ€ smoke

Q6_K ไธปๆจกๅž‹ + ่ง†่ง‰ๅก”๏ผˆbare๏ผ‰๏ผš

llama-cli \
  -m Q6_K/Qwen3.6-27B-Uncensored-Heretic-Q6_K.gguf \
  --mmproj Q6_K/mmproj-Qwen3.6-27B-Q8_0.gguf \
  --image IMAGE.jpg --image-min-tokens 1024 \
  -p "Return only the large newspaper headline." \
  --reasoning off --reasoning-budget 0 \
  --single-turn --simple-io -n 64 -c 4096 -ngl all --temp 0

Q6_K ไธปๆจกๅž‹ + Q8 MTP + ่ง†่ง‰ๅก”๏ผš

llama-cli \
  -m Q6_K/Qwen3.6-27B-Uncensored-Heretic-Q6_K.gguf \
  --mmproj Q6_K/mmproj-Qwen3.6-27B-Q8_0.gguf \
  --model-draft Q6_K/Q8-MTP-draft.gguf \
  --spec-type draft-mtp \
  --spec-draft-ngl all --spec-draft-n-max 3 \
  --image IMAGE.jpg --image-min-tokens 1024 \
  -p "Return only the large newspaper headline." \
  --reasoning off --reasoning-budget 0 \
  --single-turn --simple-io -n 64 -c 4096 -ngl all --temp 0

้ชŒ่ฏ่บซไปฝ๏ผšQ6_K ไธปๆจกๅž‹ SHA256 0332f6a0a0b53961271db7c3baef044cc72abba55efd3a5a40781b966dff9196๏ผ›Q8 MTP draft SHA256 fb18a96292d43f746b39f2e3e8998e477788b5b7659d98a7cc192e9e753c8fd4๏ผ›Q8 ่ง†่ง‰ๅก” SHA256 000715ac8ab27d95cb63a1efec2f6b3066b6d4ecacaddbdaa9d675dc03018588ใ€‚

Spark smoke ็ป“ๆžœ
bare + vision ๅ›พ็‰‡ๆˆๅŠŸ่ฝฝๅ…ฅ๏ผŒ่ฏ†ๅˆซ 1969-07-21ใ€Š็บฝ็บฆๆ—ถๆŠฅใ€‹ไธŽๆ ‡้ข˜ โ€œMEN WALK ON MOONโ€๏ผ›EXIT_STATUS=0
MTP + vision ๅ‘ฝไปคๅซ --model-draft ... --spec-type draft-mtp๏ผŒๅ›พ็‰‡ๆˆๅŠŸ่ฝฝๅ…ฅ๏ผŒๆœ€็ปˆๆญฃๆ–‡ไธบ MEN WALK ON MOON๏ผŒ9 ไธช draft token ๆŽฅๅ— 5 ไธช๏ผ›EXIT_STATUS=0
MTP ๆ–‡ๆœฌ ๆœ€็ปˆๆญฃๆ–‡ไธบ SMOKE_OK_42๏ผŒ39 ไธช draft token ๆŽฅๅ— 35 ไธช๏ผ›EXIT_STATUS=0

ไปฅไธŠๆ˜ฏไธฅๆ ผ load/image smoke๏ผŒไธๆ˜ฏๅฎŒๆ•ด่ง†่ง‰่ƒฝๅŠ›ๅŸบๅ‡†ใ€‚ๅทฒ้ชŒ่ฏ่ทฏๅพ„ๆ˜ฏไธŠ่ฟฐ llama.cpp commit๏ผ›้€š็”จ GUI ๆ˜ฏๅฆๆ”ฏๆŒ MTP ไธŽ mmproj ็ป„ๅˆ้œ€ไปฅๅ„่‡ช่ฟ่กŒๆ—ถไธบๅ‡†ใ€‚

่ฟ™ๆ˜ฏๅŸบไบŽๆ—ขๆœ‰ Qwen3.6-27B-DSV4Pro-GLM52-SFT-GPT55-RL-Coding ๆจกๅž‹ๅˆถไฝœ็š„ Uncensored-Heretic ๆ‹’็ญ”ๅ€พๅ‘ๆบถ่งฃๅฏ่กŒๆ€งๅฎž้ชŒ็‰ˆ GGUFใ€‚ไปฃ็ ใ€ๆŽจ็†ใ€่ง†่ง‰ๆŠ•ๅฝฑๅ™จใ€MTP ไธŽ LynnStyle ้‡ๅŒ–่ฏดๆ˜Ž็ปงๆ‰ฟ่‡ชๅŽŸ็‰ˆ๏ผ›ๆ‹’็ญ”่กจ็Žฐๅ’Œๆฏไธ€ไธชๆ–ฐ้‡ๅŒ–ๆกฃไปๅˆ†ๅˆซ้‡ๆ–ฐ่ฟ‡้—จ็ฆใ€‚

ไธ‹่ฝฝ็›ฎๅฝ•

ๅฏไธ‹่ฝฝๆกฃไฝไปฅไป“ๅบ“่ฟœ็ซฏๆ–‡ไปถๆ ‘ไธบๅ‡†ใ€‚

  • Q8_0/Qwen3.6-27B-Uncensored-Heretic-Q8_0.gguf๏ผšๅฎŒๆ•ด Q8_0 ไธปๆจกๅž‹ใ€‚
  • Q6_K/Qwen3.6-27B-Uncensored-Heretic-Q6_K.gguf๏ผšๅฎŒๆ•ด Q6_K ไธปๆจกๅž‹ใ€‚
  • Q5_LynnStyle/Qwen3.6-27B-Uncensored-Heretic-Q5_LynnStyle.gguf๏ผšๅฎŒๆ•ด Q5 LynnStyle ไธปๆจกๅž‹ใ€‚
  • Q4_LynnStyle/Qwen3.6-27B-Uncensored-Heretic-Q4_LynnStyle.gguf๏ผšๅฎŒๆ•ด Q4 LynnStyle ไธปๆจกๅž‹ใ€‚
  • Q3_LynnStyle/Qwen3.6-27B-Uncensored-Heretic-Q3_LynnStyle.gguf๏ผšๅฎŒๆ•ด Q3 LynnStyle ไธปๆจกๅž‹ใ€‚
  • Q8-MTP-draft.gguf ไธ็ปๅ†ๆฌก้‡ๅŒ–ใ€ๅ…จๆกฃๅ…ฑ็”จ๏ผ›ๆฏไธชๅทฒๅ‘ๅธƒๆกฃ็›ฎๅฝ•ๅ‡ๅŒ…ๅซใ€‚
  • mmproj-Qwen3.6-27B-Q8_0.gguf ไธ็ปๅ†ๆฌก้‡ๅŒ–ใ€ๅ…จๆกฃๅ…ฑ็”จ๏ผ›ๆฏไธชๅทฒๅ‘ๅธƒๆกฃ็›ฎๅฝ•ๅ‡ๅŒ…ๅซใ€‚
  • ๆฏไธชๆกฃไฝ็›ฎๅฝ•้ƒฝๅธฆ่‡ชๅทฑ็š„ SHA256SUMS ไธŽ manifest.jsonใ€‚

LM Studioใ€llama.cpp Windows Manager ็ญ‰ๅ›พๅฝขๅฏๅŠจๅ™จๅช้€‰ๆ‹ฉๅฎŒๆ•ดไธปๆ–‡ไปถ๏ผŒไธ่ฆๆŠŠ MTP sidecar ๅฝ“ไธปๆจกๅž‹ใ€‚

llama.cpp ๅฏๅŠจ

Q3 LynnStyle ไธปๆจกๅž‹ + ๅ…จๆกฃๅ…ฑ็”จ Q8 MTP + Q8 ่ง†่ง‰ๅก”๏ผš

llama-server \
  -m Q3_LynnStyle/Qwen3.6-27B-Uncensored-Heretic-Q3_LynnStyle.gguf \
  --mmproj Q3_LynnStyle/mmproj-Qwen3.6-27B-Q8_0.gguf \
  --model-draft Q3_LynnStyle/Q8-MTP-draft.gguf \
  --spec-type draft-mtp --spec-draft-n-max 3 \
  -ngl 999 -c 8192 -b 1024 -ub 256 -np 1 \
  --host 0.0.0.0 --port 8080 \
  --jinja --reasoning on --reasoning-format deepseek

Q6_K bare ็คบไพ‹๏ผš

llama-server \
  -m Q6_K/Qwen3.6-27B-Uncensored-Heretic-Q6_K.gguf \
  -ngl 999 -c 8192 -b 1024 -ub 256 -np 1 \
  --host 0.0.0.0 --port 8080 \
  --jinja --reasoning on --reasoning-format deepseek

Q6_K ๅ•ๆต MTP ็คบไพ‹๏ผš

llama-server \
  -m Q6_K/Qwen3.6-27B-Uncensored-Heretic-Q6_K.gguf \
  --model-draft Q6_K/Q8-MTP-draft.gguf \
  -ngl 999 --split-mode layer \
  -c 8192 -b 1024 -ub 256 -t 16 -np 4 -cb \
  --spec-type draft-mtp \
  --spec-draft-n-max 3 \
  --spec-draft-p-min 0.6 \
  --host 0.0.0.0 --port 8080 \
  --jinja --reasoning on --reasoning-format deepseek

Q6_K ๅทฒ้ชŒ่ฏ็š„ smoke ไฝฟ็”จ MTP n=3ใ€‚้œ€่ฆๆœ€ๅคงๅ…ผๅฎนๆ€งๆˆ–ๆ›ดไฝŽๅ†…ๅญ˜ๅ ็”จๆ—ถ๏ผŒไผ˜ๅ…ˆๅชๅŠ ่ฝฝ bare ไธปๆจกๅž‹ใ€‚

ๅฎ˜ๆ–น Qwen3.6 ้‡‡ๆ ทๅ‚ๆ•ฐไธŽ Q5/Q4 ๅฏๅŠจๆจกๆฟ

ไปฅไธ‹ๅ‚ๆ•ฐๆฅ่‡ชๅฎ˜ๆ–น Qwen/Qwen3.6-27B ๆจกๅž‹ๅก๏ผ›ไธๅŒๆŽจ็†ๆก†ๆžถ็š„ๅ‚ๆ•ฐๆ”ฏๆŒๅฏ่ƒฝไธๅŒใ€‚Qwen3.6 ไธไฝฟ็”จๆ—ง็‰ˆ /thinkใ€/nothink ่ฝฏๅˆ‡ๆข๏ผŒๅบ”ๅœจ่ฟ่กŒๆ—ถๆˆ–่ฏทๆฑ‚ๆจกๆฟไธญๆ˜Ž็กฎ้€‰ๆ‹ฉ thinking / non-thinkingใ€‚

ๆจกๅผ ๅฎ˜ๆ–น้‡‡ๆ ทๅ‚ๆ•ฐ
Thinking๏ผŒไธ€่ˆฌไปปๅŠก temperature=1.0ใ€top_p=0.95ใ€top_k=20ใ€min_p=0.0ใ€presence_penalty=0.0ใ€repetition_penalty=1.0
Thinking๏ผŒ็ฒพ็กฎ Coding temperature=0.6ใ€top_p=0.95ใ€top_k=20ใ€min_p=0.0ใ€presence_penalty=0.0ใ€repetition_penalty=1.0
Instruct / non-thinking temperature=0.7ใ€top_p=0.80ใ€top_k=20ใ€min_p=0.0ใ€presence_penalty=1.5ใ€repetition_penalty=1.0

ๅฎ˜ๆ–น่ฏดๆ˜Ž๏ผšๅœจๆก†ๆžถๆ”ฏๆŒๆ—ถ๏ผŒๅฏไปฅๅœจ 0โ€“2 ่Œƒๅ›ด่ฐƒ่Š‚ presence_penalty ๆฅๅ‡ๅฐ‘ๆ— ๅฐฝ้‡ๅค๏ผ›่ฟ‡้ซ˜ๅฏ่ƒฝๅฏผ่‡ด่ฏญ่จ€ๆททๆ‚ๆˆ–่ฝปๅพฎ่ดจ้‡ไธ‹้™ใ€‚ๅ› ๆญคๅบ”ๅ…ˆ้‡‡็”จไธŠ่กจ้ข„่ฎพ๏ผŒไธ่ฆ็›ฒ็›ฎๅ ๅŠ ๆƒฉ็ฝšใ€‚

Q5 bareใ€non-thinking ้ข„่ฎพ๏ผš

llama-server \
  -m Q5_LynnStyle/Qwen3.6-27B-Uncensored-Heretic-Q5_LynnStyle.gguf \
  -ngl 999 -c 8192 -b 1024 -ub 256 -np 1 \
  --temp 0.7 --top-p 0.80 --top-k 20 --min-p 0.0 \
  --presence-penalty 1.5 --repeat-penalty 1.0 \
  --host 0.0.0.0 --port 8080 \
  --jinja --reasoning off

Q5 + ๅ…จๆกฃๅ…ฑ็”จ Q8 MTP + Q8 ่ง†่ง‰ๅก”๏ผŒ็ฒพ็กฎ Coding thinking ้ข„่ฎพ๏ผš

llama-server \
  -m Q5_LynnStyle/Qwen3.6-27B-Uncensored-Heretic-Q5_LynnStyle.gguf \
  --mmproj Q5_LynnStyle/mmproj-Qwen3.6-27B-Q8_0.gguf \
  --model-draft Q5_LynnStyle/Q8-MTP-draft.gguf \
  --spec-type draft-mtp --spec-draft-n-max 3 \
  -ngl 999 -c 8192 -b 1024 -ub 256 -np 1 \
  --temp 0.6 --top-p 0.95 --top-k 20 --min-p 0.0 \
  --presence-penalty 0.0 --repeat-penalty 1.0 \
  --host 0.0.0.0 --port 8080 \
  --jinja --reasoning on --reasoning-format deepseek

Q4 bareใ€non-thinking ้ข„่ฎพ๏ผš

llama-server \
  -m Q4_LynnStyle/Qwen3.6-27B-Uncensored-Heretic-Q4_LynnStyle.gguf \
  -ngl 999 -c 8192 -b 1024 -ub 256 -np 1 \
  --temp 0.7 --top-p 0.80 --top-k 20 --min-p 0.0 \
  --presence-penalty 1.5 --repeat-penalty 1.0 \
  --host 0.0.0.0 --port 8080 \
  --jinja --reasoning off

Q4 + ๅ…จๆกฃๅ…ฑ็”จ Q8 MTP + Q8 ่ง†่ง‰ๅก”๏ผŒ็ฒพ็กฎ Coding thinking ้ข„่ฎพ๏ผš

llama-server \
  -m Q4_LynnStyle/Qwen3.6-27B-Uncensored-Heretic-Q4_LynnStyle.gguf \
  --mmproj Q4_LynnStyle/mmproj-Qwen3.6-27B-Q8_0.gguf \
  --model-draft Q4_LynnStyle/Q8-MTP-draft.gguf \
  --spec-type draft-mtp --spec-draft-n-max 3 \
  -ngl 999 -c 8192 -b 1024 -ub 256 -np 1 \
  --temp 0.6 --top-p 0.95 --top-k 20 --min-p 0.0 \
  --presence-penalty 0.0 --repeat-penalty 1.0 \
  --host 0.0.0.0 --port 8080 \
  --jinja --reasoning on --reasoning-format deepseek

ไธบไป€ไนˆ Hugging Face ๅณไพงๅฏ่ƒฝไธๆ˜พ็คบ้‡ๅŒ–ๆ ผๅผ

Q5ใ€Q4ใ€Q3 LynnStyle GGUF ไฝฟ็”จๆททๅˆ็ฒพๅบฆ tensor ๅธƒๅฑ€๏ผšๅŒไธ€ไธชๆ–‡ไปถไผšๆ นๆฎๅฑ‚ไฝใ€attention patternใ€tensor family ไธŽ้‡่ฆๅบฆๆ กๅ‡†็ป“ๆžœ๏ผŒๆœ‰ๆ„็ป„ๅˆๅคš็ง้‡ๅŒ–็ฑปๅž‹ใ€‚ๅ› ๆญค๏ผŒHugging Face ๅณไพงๅ…ƒๆ•ฐๆฎๆ ๅฏ่ƒฝๆ— ๆณ•ๆŠŠๅฎƒๅฝ’็บฆๆˆๅ•ไธ€ๆ ‡ๅ‡†้‡ๅŒ–ๆ ‡็ญพ๏ผŒๆ˜พ็คบไธบ็ฉบ็™ฝๆˆ–ๆ— ๆณ•่ฏ†ๅˆซใ€‚่ฟ™ไธไปฃ่กจ GGUF ๆฒกๆœ‰้‡ๅŒ–ๆˆ–ๆ–‡ไปถ็ผบๅคฑใ€‚ๆƒๅจ่ฏ†ๅˆซไพๆฎๆ˜ฏๆกฃไฝ็›ฎๅฝ•ไธŽๆ–‡ไปถๅ๏ผŒไปฅๅŠ GGUF general.file_typeใ€tensor ็ฑปๅž‹ๅˆ†ๅธƒใ€manifest.json ไธŽ SHA256SUMSใ€‚

LynnStyle Dense

27B ๆ˜ฏ Dense๏ผŒๆฒกๆœ‰ MoE expert๏ผŒไฝ†ไธไปฃ่กจๅ…จๅฑ‚ๅŒ่ดจใ€‚Qwen3.6 27B ๆœ‰ๆททๅˆๆณจๆ„ๅŠ›็ป“ๆž„๏ผšfull-attention ๅฑ‚็บฆๆฏ 4 ๅฑ‚ๅ‡บ็Žฐไธ€ๆฌก๏ผŒๅ…ถไป–ๅฑ‚ๆ›ดๅคšๆ˜ฏ linear attention / SSMใ€‚LynnStyle Dense ็š„ไฟๆŠคๅ•ไฝๆ˜ฏๅฑ‚ไฝใ€attention patternใ€tensor familyใ€‚

Q5/Q4/Q3/Q2 ไธๆ˜ฏไธ€ๅˆ€ๅˆ‡ไฝŽๆฏ”็‰น้‡ๅŒ–๏ผŒ่€Œๆ˜ฏๆ˜พๅผไฟๆŠคๅ…ณ้”ฎๅฑ‚ใ€full-attentionใ€attention gateใ€SSM ๅ’Œ้‡่ฆ tensor family๏ผŒๅ†ๅฏนๆ™ฎ้€šๅฑ‚ๆŒ‰ๆกฃไฝๅŽ‹็ผฉ๏ผŒๅนถๅ ๅŠ  imatrix ๆˆ–็ญ‰ไปท้‡่ฆๅบฆๆ กๅ‡†ใ€‚ๅชๅš direct quant ไธๅ…ทๅค‡ๅ‘ๅธƒ่ต„ๆ ผใ€‚

ๅŽปๅฎกๆŸฅๅ‰ Coding ๅŸบ็บฟ้—จ็ฆ

  • Q8๏ผšMMLU500 92.4%๏ผŒGPQA198 81.82%๏ผŒLCB100 78/100ใ€‚
  • Q5๏ผšMMLU500 92.2%๏ผŒGPQA198 78.79%๏ผŒLCB100 75/100ใ€‚
  • Q4๏ผšMMLU500 92.0%๏ผŒGPQA198 74.75%๏ผŒLCB100 71/100ใ€‚
  • Q3๏ผšMMLU500 91.4%๏ผŒGPQA198 77.27%๏ผŒLCB100 69/100ใ€‚

่ฎญ็ปƒไธ็Œ benchmark ็ญ”ๆกˆใ€‚SFT ๅญฆ่ฝจ่ฟน๏ผŒRL ๅญฆ่กŒไธบๅๅฅฝ๏ผŒMMLU / GPQA / LCB ไฝœไธบ held-out ้—จ็ฆใ€‚

ๅŽปๅฎกๆŸฅๅ‰ Coding ๅŸบ็บฟ๏ผšQ4 / Q3 llama.cpp ่ฟ่กŒ็Ÿฉ้˜ต

TPS ๅ–ๆ•ด๏ผŒๆŽฅๅ—็އไปฅ็™พๅˆ†ๆฏ”ๆ˜พ็คบใ€‚C1/C2/C4 ่กจ็คบๅนถๅ‘ๆ•ฐ๏ผ›็ป“ๆžœๆฅ่‡ชๅŒไธ€ๅฅ— llama.cpp harnessใ€‚

Q4 LynnStyle

ๆจกๅผ C1 C2 C4
bare 59 TPS 76 TPS 116 TPS
ๆœ€ไฝณ MTP n=2๏ผš75 TPS๏ผˆ91%๏ผ‰ n=2๏ผš82 TPS๏ผˆ93%๏ผ‰ n=3๏ผš83 TPS๏ผˆ89%๏ผ‰

ๆœ€ไฝณๆกฃไฝ๏ผšC4 bare๏ผŒ116 TPSใ€‚ไฝŽๅนถๅ‘ๅปถ่ฟŸไผ˜ๅ…ˆๆ—ถ๏ผŒC1/C2 ไฝฟ็”จ MTP n=2ใ€‚

Q3 LynnStyle

ๅนถๅ‘ bare MTP n=2 MTP n=3 MTP n=4 ๆŽจ่
C1 15 TPS 22 TPS๏ผˆ86%๏ผ‰ 23 TPS๏ผˆ84%๏ผ‰ 23 TPS๏ผˆ81%๏ผ‰ MTP n=3๏ผŒ่พƒ bare ็บฆ +53%
C2 28 TPS 28 TPS๏ผˆ86%๏ผ‰ 28 TPS๏ผˆ84%๏ผ‰ 27 TPS๏ผˆ81%๏ผ‰ MTP n=2๏ผŒๅ–ๆ•ดๅŽไธŽ bare ๅŒ้€Ÿ
C4 49 TPS 29 TPS๏ผˆ86%๏ผ‰ 30 TPS๏ผˆ84%๏ผ‰ 30 TPS๏ผˆ81%๏ผ‰ bare

Q3 ๅ•่ฏทๆฑ‚ๅปถ่ฟŸไผ˜ๅ…ˆๆŽจ่ MTP n=3๏ผ›C2 ็š„ MTP ๆœ€ไผ˜้€‰ๆ‹ฉไธบ n=2๏ผŒไฝ†ๅ–ๆ•ดๅžๅไธŽ bare ๆŒๅนณ๏ผ›C4 ๅžๅไผ˜ๅ…ˆไฝฟ็”จ bareใ€‚ไธฅๆ ผ 16GB ๆˆ–้•ฟไธŠไธ‹ๆ–‡ไนŸไผ˜ๅ…ˆๅชๅŠ ่ฝฝ Q3 ไธปๆจกๅž‹ใ€‚

Downloads last month
2,000
GGUF
Model size
27B params
Architecture
qwen35
Hardware compatibility
Log In to add your hardware

6-bit

8-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support