--- license: apache-2.0 base_model: - Qwen/Qwen3.6-27B library_name: transformers tags: - speculative-decoding - dspark - draft-model - vllm - qwen3 datasets: - nvidia/Open-SWE-Traces - greghavens/gpt-5.6-sol-coding-and-debugging-traces - greghavens/fable-5-coding-and-debugging-traces - greghavens/glm-5.2-coding-and-debugging-traces - Roman1111111/claude-sonnet-4.6-120000x - qsardor/Claude-Sonnet-Opus - ianncity/GLM-5.2-Conversation --- # Qwen3.6-27B-DSpark A DSpark draft head for **Qwen3.6-27B**, trained on a mixed agentic + general-chat corpus. Drops into stock vLLM as a `--speculative-config`. **2.62× faster decoding than no speculation**, **1.48× faster than the upstream DSpark head**, and **1.27× faster than Qwen3.6's built-in MTP**, at **4.67 tokens accepted per target pass**. Measured over 96 held-out prompts spanning agentic tool-use, coding and general chat, with generation length and sampling fixed so wall-clock is directly comparable. > **Two builds of this head.** This is the **full-vocabulary** build (8.20 GiB) — > prefer it for long-context agentic work. A **vocabulary-truncated** build of the > same trained weights is at > [`abstract-extraordinary/Qwen3.6-27B-DSpark-49k`](https://huggingface.co/abstract-extraordinary/Qwen3.6-27B-DSpark-49k) > (6.20 GiB, faster on short prompts, no throughput advantage on long ones). See > [Vocabulary variants](#vocabulary-variants). ## Usage ```bash vllm serve unsloth/Qwen3.6-27B-NVFP4 \ --speculative-config '{"method":"dspark", "model":"abstract-extraordinary/Qwen3.6-27B-DSpark", "num_speculative_tokens":8}' \ --trust-remote-code ``` Use `num_speculative_tokens: 8`. The head drafts blocks of exactly 8 tokens (`block_size=8`), so a smaller value leaves trained capacity unused and a larger one asks it to predict positions it has never seen. ## Benchmarks **96 held-out prompts across 8 workload buckets.** All five configurations ran in a single session against the same prompts, identical but for `--speculative-config`, each emitting exactly 300 greedy tokens (`temperature=0`, `min_tokens=max_tokens`, `ignore_eos`) so wall-clock is directly comparable. Target `unsloth/Qwen3.6-27B-NVFP4`, single GB10, concurrency 1. Every figure is a draft-weighted pool over the whole run, not a mean over prompts. | config | tok/s | tok/step | accept | vs no drafter | |---|---|---|---|---| | no drafter | 10.05 | — | — | 1.00× | | [`satgeze/Qwen3.6-27B-DSpark`](https://huggingface.co/satgeze/Qwen3.6-27B-DSpark) (upstream, untrained) | 17.74 | 2.86 | 23.4% | 1.77× | | Qwen3.6 built-in MTP (3 tokens) | 20.76 | 3.25 | 75.1% | 2.07× | | **this model** | **26.28** | **4.67** | 46.2% | **2.62×** | | [same head, 49k vocab](https://huggingface.co/abstract-extraordinary/Qwen3.6-27B-DSpark-49k) | 28.18 | 4.66 | 46.0% | 2.80× | The upstream head was run at **both** `num_speculative_tokens: 8` and `15` (its own `block_size`); 8 was faster (17.74 vs 17.07 tok/s) and is quoted here, so the baseline is shown at its stronger setting. MTP shows a much higher acceptance *percentage* while being slower. The two are not in tension: MTP drafts 3 tokens per pass and this head drafts 8, so a higher hit rate on fewer guesses still yields fewer accepted tokens per step (3.25 vs 4.67). Tokens per step is the quantity that converts into wall-clock. By workload: | bucket | mean prompt tok | no drafter | this model | speedup | |---|---|---|---|---| | general chat (ianncity) | 125 | 11.10 | 34.51 | 3.11× | | agentic (kimi-k3) | 1,000 | 10.92 | 33.11 | 3.03× | | agentic (fable-5) | 1,446 | 10.84 | 32.62 | 3.01× | | general chat (qsardor) | 68 | 11.11 | 32.13 | 2.89× | | general/code/math (sonnet-4.6) | 93 | 11.11 | 30.95 | 2.79× | | agentic (glm-5.2) | 5,770 | 9.78 | 23.48 | 2.40× | | agentic (gpt-5.6-sol) | 9,752 | 8.92 | 20.59 | 2.31× | | agentic SWE (nvidia) | 15,841 | 7.82 | 16.67 | 2.13× | Absolute tok/s tracks prompt length rather than head quality — long prompts spend most of their wall-clock in prefill, which speculation does not accelerate. The speedup column is the like-for-like comparison. The kimi-k3 bucket was excluded from training entirely and is a clean generalisation probe. ### Acceptance by draft position ![Cumulative acceptance by draft position](acceptance-by-position.png) Acceptance decays *multiplicatively*: surviving to position 3 means winning four consecutive rejection tests. That compounding is why a few points at position 0 turn into a large difference in tokens per step, and it is what separates this head from the untrained upstream one. Upstream starts at a respectable 62.4% at position 0 but halves roughly every two positions to 5.8% by position 7; this head starts at 82.7% and still retains 22.8%. ## Vocabulary variants | | this model | [`-49k`](https://huggingface.co/abstract-extraordinary/Qwen3.6-27B-DSpark-49k) | |---|---|---| | draft vocabulary | 248,320 (full) | 49,167 | | size on disk | 8.20 GiB | 6.20 GiB | | tok/s (96 prompts) | 26.28 | 28.18 | | tok/step | 4.67 | 4.66 | | best for | long-context agentic | short-prompt / chat-shaped | Both are the **same trained weights**. The `-49k` build row-gathers `lm_head` and `markov_w2` to the tokens the target actually emits — a post-hoc transform, not a retrain. It is faster per decode step because the draft head streams its whole weight set every step and a smaller head streams less. That saving is large on short prompts and **disappears on long ones**, where prefill dominates wall-clock, while a small acceptance cost remains. **If your prompts run to thousands of tokens, use this full-vocab build.** ## Training Assistant turns **regenerated by the target model itself** (temperature 0.7, top-p 0.95), so the head learns the distribution it will actually have to predict — not the distribution of whatever model produced the original traces. 14,707 conversations from seven sources: fresh `nvidia/Open-SWE-Traces` windows, three agentic trace sets, and three general-chat sets. 44.93M supervised tokens at 46.6% loss density (mean 3,055 per sample). 600 steps in 6.3 h on one node, lr 6e-4, 384 anchors, `block_size=8`; final loss 0.73 from ~1.05. Held-out offline accepted length was monotone across all 20 checkpoints and plateaued at steps 540–600, so the run converged rather than being truncated. Held-out prompts were excluded from training by content fingerprint before harvesting, train/val shard offsets verified disjoint, and the publishers' own splits respected where they exist. All evaluation above uses only held-out samples. ## Limitations - **Correctness is unverified — not failed.** Speculative decoding is distribution-preserving under rejection sampling, but we could not run a token-identity check: greedy decoding is not reproducible on this stack run-to-run (a target-only run matches *itself* on 3 of 12 prompts, because prefix caching and batch composition change the numerical path). This head adds no measurable divergence beyond that baseline, which is weaker evidence than a proof. - **No live-traffic measurement.** Every number here is a held-out benchmark at concurrency 1. Held-out and live figures have disagreed before for this line of heads. - **Single hardware.** Benchmarked only on one GB10 with `unsloth/Qwen3.6-27B-NVFP4`. Speculative-decoding economics are bandwidth-dependent; expect different ratios elsewhere. - Acceptance is far from uniform across workloads (40.5–60.4% by bucket). Long agentic prompts draft best per step but gain least in wall-clock, because prefill dominates there. - Run-to-run spread on this benchmark is about 1% on pooled tok/s (measured by repeating the no-drafter and MTP configurations on two vLLM builds). Differences smaller than that should not be read as real. ## Credits Initialised from [`satgeze/Qwen3.6-27B-DSpark`](https://huggingface.co/satgeze/Qwen3.6-27B-DSpark). DSpark architecture from *DSpark: Confidence-Scheduled Speculative Decoding with Semi-Autoregressive Generation*. Training data from NVIDIA and the community dataset authors listed above.