---
license: apache-2.0
base_model: Qwen/Qwen3.6-35B-A3B
language:
  - en
pipeline_tag: text-generation
tags:
  - gguf
  - llama.cpp
  - speculative-decoding
  - multi-token-prediction
  - mixture-of-experts
  - agentic
  - reasoning
  - tess
---

# Tess-4-35B-A3B — GGUF with native MTP self-speculation

**Reasoning that scales with the problem — now with decode speed that scales with your hardware.** Tess-4-35B-A3B is the Mixture-of-Experts member of the Tess-4 family: an agentic, thinking-native model that deliberates harder exactly when it matters, on an architecture light enough to run interactively on a single consumer machine.

These are not ordinary GGUFs. Each file carries the model's **multi-token-prediction (MTP) block spliced back in**, enabling native self-speculative decoding in llama.cpp — **+23–50% decode throughput, with the trunk weights byte-identical to a standard conversion**. Every speculative proposal is verified by the full model before acceptance; reduced precision exists only on the proposal path.

## The Tess-4 family

Tess-4 models are trained by [Migel Tissera](https://huggingface.co/migtissera) on **64K-token long-context agentic traces — real engineering work done with Fable-5** — not synthetic single-turn data. Reasoning is distilled from a **three-model teacher ensemble (Opus-4.8, GPT-5.5, and GLM-5.2)** into coherent deliberation patterns, using explicit `<think>...</think>` blocks for private reasoning before the visible answer. The same dataset and recipe produced [Tess-4-27B](https://huggingface.co/migtissera/Tess-4-27B), which ranks **first on BenchLocal at 81% (122/150), ahead of larger models including Qwen3.6-35B and Gemma-4-31B**.

Tess-4-35B-A3B applies that recipe to **Qwen3.6-35B-A3B**: a 40-block hybrid MoE — gated-delta-net linear attention with full GQA every fourth layer, 256 experts with 8 active (~3B active parameters), 262,144-token trained context. The sparse activation is what makes it special on local hardware: you get 35B-class breadth at ~3B-class decode cost, and with the MTP morph below, over 100 tokens/second on an Apple M4 Max.

## What's in this repo

| File | Size | Use |
|---|---:|---|
| `Tess-4-35B-A3B-Q8_0-MTP-Q4D-Q4H-PV64K.gguf` | 37.7 GB | Reference quality; cleanest determinism record |
| `Tess-4-35B-A3B-Q6_K-MTP-Q4D-Q4H-PV64K.gguf` | 29.4 GB | Mid-size |
| `Tess-4-35B-A3B-Q4_K_M-MTP-Q4D-Q4H-PV64K.gguf` | 22.0 GB | Smallest, fastest trunk |
| `mtp-Qwen3.6-35B-A3B-Q4_0.gguf` | 1.2 GB | Donor: MTP block (rebuilds only — not needed to run) |
| `Qwen3.6-35B-A3B-base.gguf` | 3.7 GB | Donor: MTP block, BF16 master (rebuilds only) |
| `tess-draft-head-Q4_0.gguf` | 0.3 GB | Donor: Tess draft head (rebuilds only) |

Naming: `MTP` = embedded nextn block, `Q4D`/`Q4H` = Q4_0 draft block/head, `PV64K` = pruned 65,536-token draft vocabulary (the draft's LM-head matvec drops ~75% of its bytes; the full head remains in-file as a fallback via `LLAMA_MTP_PRUNED_HEAD=0`).

## Requirements

Built and validated on the `opt-minimax` branch of the trinity-cloud llama.cpp fork, which provides the `--spec-type draft-mtp` runtime and the pruned-vocab draft path. On llama.cpp builds without MTP support these files are untested — the extra `blk.40.*` tensors may be rejected by stricter loaders. Single-stream serving (`-np 1`) is required for speculation.

## Quickstart

```bash
llama-server -m Tess-4-35B-A3B-Q8_0-MTP-Q4D-Q4H-PV64K.gguf \
  -c 32768 -ub 2048 -np 1 -ngl 99 -fa on --jinja --metrics \
  --spec-type draft-mtp \
  --spec-draft-n-max 5 --spec-draft-n-min 0 --spec-draft-p-min 0.70 \
  --host 127.0.0.1 --port 8787
```

No separate draft model (`-md`) — the MTP block lives inside the GGUF. The chat template (Qwen3.5-family, `<think>` blocks) is embedded; `--jinja` enables it, and tool calling works through it.

```
<|im_start|>user
prompt<|im_end|>
<|im_start|>assistant
<think>reasoning</think>
answer<|im_end|>
```

Context is cheap on this architecture (only ~10/40 layers carry KV): raising `-c` toward the trained 262,144 costs roughly 20 KB/token. Keep the server context at or above your client's ceiling, with client-side compaction triggering a margin below it.

## Measured performance

Apple M4 Max (128 GiB), llama.cpp fork `opt-minimax`, greedy decoding, production speculation config (`n_max=5, p_min=0.70`), 32K context:

| Variant | Workload | No speculation | MTP | Draft acceptance |
|---|---|---:|---:|---:|
| Q8_0 | code, 1536 tok | 74.8 t/s | **109.9 t/s** (+47%) | 88.9% |
| Q8_0 | prose, 768 tok | 80.0 t/s | **92.4 t/s** (+15%) | 84.9% |
| Q8_0 | mixed sweep, 1024 tok | 70.7 t/s | **102.6 t/s** (+45%) | 85.9% |
| Q4_K_M | code, 1536 tok | 98.6 t/s | **120.9 t/s** (+23%) | 90.9% |
| Q4_K_M | prose, 768 tok | 99.1 t/s | **103.0 t/s** (+4%) | 82.4% |
| Q6_K | prose, 768 tok | 80.9 t/s | **106.8 t/s** (+32%) | 84.5% |
| Q6_K | code, 1536 tok | 80.8 t/s | ~128 t/s¹ | 91.9% |

¹ Divergent token stream (see determinism notes); ballpark only.

Speculation never degrades output quality: the trunk verifies every drafted token, and rejected proposals cost only the draft step. Warm long-form code generation on Q8_0 reached 116.8 t/s.

## Important: run it gated

The llama.cpp binary default for `--spec-draft-p-min` is **0.0 (ungated)**. If you launch with `--spec-type draft-mtp` and forget the flag, deep low-confidence drafting silently degrades both throughput at depth (acceptance measured collapsing from 85.9% to 54.5%) **and** greedy reproducibility versus non-speculative decode. Always pass `--spec-draft-p-min 0.70`. Per-request override of speculation parameters is disabled in the fork's server, so a correctly launched server cannot be un-gated by clients.

## Determinism notes (read before building eval harnesses)

With the production config, greedy MTP output was verified token-identical to non-speculative decode on multi-prompt probes: Q8_0 to 1536 tokens plus a 1024-token sweep, Q4_K_M to 1536 tokens. Serving a PV64K file *without* speculation flags is verified byte-identical to a plain trunk conversion — so these files fully substitute for the originals in non-speculative use.

One open, characterized deviation: on very long generations, speculative and non-speculative greedy decode can diverge at a rare near-tie token (different kernel batch widths → last-ulp numeric differences → an occasional argmax flip between near-equal candidates). Both branches are coherent greedy continuations; this is a reproducibility deviation, not a quality loss. Measured incidence scales with trunk quant coarseness:

- **Q8_0:** ~1 flip per few thousand tokens on some prompts (first seen at ~token 110 of a 4096-token probe).
- **Q4_K_M:** clean at all tested lengths (≤1536).
- **Q6_K:** flips readily on code prompts (as early as ~token 10).

If you need strict spec-vs-nospec determinism (evals, regression gates, reproducible pipelines), serve without the speculation flags. If you validate identity yourself, use ≥4096-token probes across several prompts — short probes provably miss this class.

## How these files were made

The fine-tune's GGUF conversions are trunk-only (fine-tuning does not train the MTP block). The morph restores speculation ability with three parts:

1. **MTP block from the base model** — Qwen3.6-35B-A3B's trained `mtp.*` tensors, converted directly from the HF checkpoint and quantized to Q4_0. Draft-side precision affects only acceptance rate, never correctness.
2. **A Tess-native draft head** — this fine-tune's own `output.weight` (F16 source) quantized to Q4_0, so drafts rank tokens in Tess's distribution. Result: 82–92% acceptance, the same range the base model achieves drafting itself.
3. **A pruned 64K draft vocabulary** — bit-identical row copies of the Q4_0 head (non-normal tokens ∪ code-corpus tokens ∪ frequency-ordered BPE fill), scattered back to full-vocab logits at −inf, leaving sampling and confidence gating untouched.

The trunk tensors are byte-for-byte the standard conversion. The three donor files in this repo are sufficient to rebuild the morph onto any future trunk requantization.

## Best practices

- Production speculation config: **`n_max=5, n_min=0, p_min=0.70`**. Deeper ungated drafting is strictly worse on throughput *and* reproducibility.
- Pick Q8_0 for maximum fidelity, Q4_K_M for speed/memory (its faster trunk narrows the relative MTP gain — the draft step is a fixed cost), Q6_K in between (weakest spec-determinism record of the three).
- Greedy evals: serve without speculation flags.
- Apple Silicon benchmarking: the M4 Max throttles hard under sustained load; only back-to-back A/Bs in one thermal state are comparable.

## License

Apache 2.0, inherited from Qwen3.6-35B-A3B.

## Citation

```bibtex
@misc{tess4-35b-a3b,
  title   = {Tess-4-35B-A3B: an agentic, thinking-native Mixture-of-Experts
             model with native MTP self-speculation},
  author  = {Tissera, Migel},
  year    = {2026},
  url     = {https://huggingface.co/migtissera}
}
```