lapalaba/Qwen3.8-27B-DFlash2-ru

Qwen3.8-27B DFlash2 — Draft Head

Draft head only — not a standalone language model. This checkpoint is a DFlash2 speculator for Qwen3.8-27B. It proposes token blocks for speculative decoding; the verifier checks them and produces the final output. Do not load this checkpoint as a chat model.

Training

We started from incoai/Qwen3.8-27B-DFlash2 and continued training on a curated pool of 271,148 examples. The data covers Russian-domain agent interactions, production traces, and RAG. The RAG subset mixes native Russian examples with translated examples derived from Russian and English sources, so the dataset is not exclusively Russian-language or production data.

We excluded test and continuation-probe groups before training capture, then bucketed sequences by rendered token length under the pinned model template. Training used a 24,576-token context with thinking mode disabled.

Rendered length	Examples	Share
≤2,047 tokens	150,764	55.60%
2,048–4,095 tokens	91,065	33.58%
4,096–5,188 tokens	15,547	5.73%
5,189–8,191 tokens	11,722	4.32%
≥8,192 tokens	2,050	0.76%
Total	271,148	100.00%

We selected checkpoint step 542,298 from the training run. It had the best observed training-eval accuracy (0.66958) and loss (2.03607).

Model architecture

The export uses DFlash2DraftModel in bfloat16: five draft layers, hidden size 5,120, intermediate size 17,408, vocabulary size 248,320, DFlash2 block size 8, mask token ID 248,070, convolution kernel 2, convolution group size 16, selector rank 256, selector top-k 16, and verifier layer IDs [5, 19, 33, 47, 61].

Evaluation

We compared this checkpoint against z-lab/Qwen3.8-27B-DFlash2 (Baseline). Both were evaluated in SGLang on one NVIDIA H200 with the Qwen3.8-27B-FP8 verifier, FlashAttention 3, DFlash2 block size 8, 24,576-token context, and a 4,096-token output cap. We used temperature=0 and prompts formatted with z-lab/dflash.

Each concurrency setting has three repeats. Concurrency 64 is effectively unlimited (64-worker client cap). Baseline and this checkpoint were measured on different dates; RPS and output tok/s in the detailed results include mean ± standard deviation.

Focused throughput view

Output throughput at concurrency 32 (tok/s):

Benchmark	Baseline (tok/s)	This checkpoint (tok/s)	Δ vs baseline	Baseline τ	This checkpoint τ
HumanEval	1,049	1,117	+6.4%	6.90	6.46
Retail	402	405	+0.6%	4.97	5.05
MBPP	1,137	975	-14.2%	5.87	5.01

The checkpoint is tuned for Russian-language, retail-domain workloads. The observed pattern is consistent with a draft/verifier match that improved for retail evaluation and at higher concurrency on HumanEval, while moving away from the predominantly English, general-purpose distribution used by MBPP and the broader academic benchmarks. This is a specialization trade-off: better serving behavior on the target workload, with lower throughput on some out-of-domain tasks.

Positive configurations at practical concurrency

The view below intentionally contains only runs whose measured output throughput is above baseline: retail at 16/32/64 concurrency and HumanEval at 32/64. It is not a full benchmark summary; the complete, including negative, results remain in the detailed evaluation.

Tau is accepted output tokens per verifier step (higher is better). Throughput and tau measure serving/speculation behavior, not answer accuracy. Full results, including TTFT, TPOT, E2E, RPS, Tau, and failures: Detailed evaluation.

Usage

Serve the verifier, attaching this checkpoint as its draft head. Use versions of SGLang or vLLM that support Qwen3.8 and DFlash2. Authenticate with Hugging Face if either repository is private; for offline serving, replace repository IDs with local paths.

SGLang

python -m sglang.launch_server \
  --model-path Qwen/Qwen3.8-27B \
  --served-model-name qwen38-dflash2 \
  --host 0.0.0.0 \
  --port 8000 \
  --trust-remote-code \
  --language-only \
  --attention-backend fa3 \
  --mm-attention-backend fa3 \
  --linear-attn-prefill-backend flashinfer \
  --linear-attn-decode-backend flashinfer \
  --mamba-radix-cache-strategy extra_buffer \
  --tp-size 1 \
  --max-running-requests 32 \
  --dtype bfloat16 \
  --context-length 24576 \
  --mem-fraction-static 0.80 \
  --speculative-algorithm DFLASH \
  --speculative-draft-model-path lapalaba/Qwen3.8-27B-DFlash2-ru \
  --speculative-dflash-block-size 8

vLLM

Requires vLLM 0.28.0 or newer. DFlash2 uses method: dflash; a block size of 8 corresponds to 7 speculative tokens in the vLLM config.

vllm serve Qwen/Qwen3.8-27B \
  --served-model-name qwen38-dflash2 \
  --host 0.0.0.0 \
  --port 8000 \
  --tensor-parallel-size 1 \
  --max-model-len 24576 \
  --max-num-seqs 32 \
  --gpu-memory-utilization 0.80 \
  --speculative-config '{"method":"dflash","model":"lapalaba/Qwen3.8-27B-DFlash2-ru","num_speculative_tokens":7}'

The vLLM example uses the non-FP8 verifier, following the published DFlash2 serving recipe. The evaluation above used the FP8 verifier in SGLang; these are not vLLM benchmarks.

API smoke test

curl --fail http://localhost:8000/health
curl --fail http://localhost:8000/v1/models
curl --fail http://localhost:8000/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{"model":"qwen38-dflash2","messages":[{"role":"user","content":"Привет! Кратко представься."}],"max_tokens":128,"temperature":0}'