DCore-DPace-Qwen3-8B-Budget16

Draft model for speculative decoding with Qwen/Qwen3-8B as the target model, trained with a draft budget of 16 tokens per block.

D-Pace is the training recipe of DCore. This checkpoint applies D-Pace to the original DFlash architecture (recipe ablation).

This is a draft model: it is conditioned on hidden states of the target model and is not meant to be used as a standalone language model.

Paper: arXiv:2610.24993

Model Details

Item Value
Target model Qwen/Qwen3-8B (36 layers)
Draft architecture DFlashDraftModel
Draft layers 5
Hidden size / heads / KV heads 4096 / 32 / 8
Block size 16
Target layers used as context features 1, 9, 17, 25, 33
Vocabulary 151936 (Qwen3 tokenizer)
Precision bfloat16

Files

  • README.md
  • config.json
  • dflash.py
  • model.safetensors

Usage

from transformers import AutoModel

draft = AutoModel.from_pretrained(
    "ernie-research/DCore-DPace-Qwen3-8B-Budget16",
    trust_remote_code=True,
    dtype="bfloat16",
)

The draft model must be paired with Qwen/Qwen3-8B as the target model in a speculative decoding framework that supports DFlash-style block drafting.

Collection

Part of the DCore collection by ERNIE Research.

Downloads last month
81
Safetensors
Model size
1B params
Tensor type
BF16
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for ernie-research/DCore-DPace-Qwen3-8B-Budget16

Finetuned
Qwen/Qwen3-8B
Finetuned
(2,364)
this model

Collection including ernie-research/DCore-DPace-Qwen3-8B-Budget16

Paper for ernie-research/DCore-DPace-Qwen3-8B-Budget16