Blockway

/

Agens-Volundr-32B-Preview

Like 153

Follow

Blockway 16

Scores first

Same benchmark harness (OpenAI-compatible requests, same judge model glm-5.3-flash-321b for every scored row, same request shapes), single stream, medians of 3 runs. Both models served with their DFlash2 drafter enabled.

Decode / prefill speed (tokens/s), by prompt context:

(TTFT at 160k: 113.6 s vs 50.5 s.)

• Qwen is ~2.7x faster at decode and ~1.5-2x at prefill — mostly the bytes/token ratio (61 GB bf16 Volundr vs 52 GB bf16 Qwen with fp8 KV cache).
• Volundr's headline claim holds on our card: decode is flat with context — it actually rises from 31.9 (20k) to 38.5 (160k) t/s, while Qwen dips (95.7 -> 74.8).

LLM-judged quality (1-10 rubric, fixed judge for every row):

• Code is the upset: Volundr posted the best code score we measured all day, beating bf16 Qwen with and without a drafter. This matches the published card (LiveCodeBench +4.2, HumanEval +4.3 over Qwen3.8-27B).
• Prose/chat go to Qwen by a hair. One caveat on Volundr's variance: in one chat turn it burned the entire 49k-token thinking budget emitting an empty response, and several turns carried repetition flags — the card's admitted long-session weakness is real and visible in single-turn form too.
• Effort disclosure: Volundr ran at its template default (xhigh thinking); Qwen3.8-27B at reasoning_effort=medium. Speed rows are effort-neutral.

Code pack, item-level

32 items per model (12 hard "h" + 20 "k"; Python executed against hidden tests, Rust compiled and judge-scored). Averages: Volundr 8.16 (Python 8.45, Rust 7.17) vs Qwen 7.74 (Python 8.26 over 19 scored rows, Rust 6.92).

• Volundr's edge is functional correctness: 95% all-tests-pass on Python, and every Rust program that compiled scored 8-10.
• The same item (h10) sank both models differently: Volundr burned the full 49k-token budget emitting no usable code (score 2, length); Qwen shipped code that failed 2/4 tests (score 5).
• Volundr's outputs run 3-20x larger on the hard items (46.6k vs 6.3k tokens on h01). The judge mostly tolerated the length (7-9 scores) — but it is exactly what made the code pack take 3 h 41 min instead of 20 min.
• Qwen's deficit is judged implementation quality, not function: it passed 100% of tests on h03/h04/h12 yet scored 6/6/5 where Volundr's equivalents scored 8/9/5.
• (Qwen's h09 lost its judge row (NULL) despite passing 6/6, hence the 31-row average.)

Wall-clock to complete the full sweep

~4.6x end-to-end. The gap is almost entirely the code pack (3 h 41 min vs 20 min): Volundr's ~3x slower decode, compounded by spending the full 48k thinking budget on weak items and its attempt-2 repair rounds. Caveat: the two runs overlapped in wall-clock and shared the same warm judge, so each total carries roughly symmetric contention; the per-request t/s rows above are unaffected.

Weights and hardware

• Server: single RTX PRO 6000 Blackwell Workstation 96 GB (sm120), tensor-parallel 1. The published INT4/W4A16 is single-GPU-only anyway, so tp=1 is the supported shape.
• Volundr bf16 checkpoint: 60.9 GiB (72 layers: 54 KDA + 17 BCSA + 1 dense; by bytes it's 59.7% MLP, 19.3% KDA projections, 6.6% Engram tables in fp32, 5.9% attention). DFlash2 drafter: 3.6 GiB.
• Qwen3.8-27B bf16 reference: 51.8 GiB, served on vLLM 0.29.0 with the W4A16 DFlash2 drafter (~3.8 GB) and fp8 KV. Volundr ran with bf16 KV. Different engines — Blockway's sglang build for Volundr (stock vLLM can't load it), stock-ish vLLM for Qwen — that's not apples-to-apples engine-wise, but it's each model's natural serving path.
• Measured KV pool for Volundr: 166,270 tokens on one 96 GB card with the drafter resident (the 2x48 GB README config is roughly the same total memory; at tp=1 on 96 GB you get a bigger pool than the tp2 layout suggests).

Serving recipe (sm120, single 96 GB card, bf16 + DFlash2)

The published preview-sm89 image works on Blackwell as-is — the CUDA kernels come from the base image (lmsysorg/sglang:v0.5.16-cu129-runtime), the patch adds only Python + Rust modules, so the sm89/sm90 label is not an arch gate. (The "24 GB cards not supported" note is a memory statement: 61 + 3.6 GiB of weights + pools don't leave a 24 GB card alive.)

docker run -d --name agens-volundr-bf16 \
  --runtime=nvidia --network host --ipc=host --shm-size 32g \
  -e NVIDIA_VISIBLE_DEVICES=<GPU-UUID> \
  -e PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True \
  -e VOLUNDR_FI_WORKSPACE_MB=256 \
  -v /path/to/Agens-Volundr-32B-Preview:/models/Agens-Volundr-32B-Preview:ro \
  -v /path/to/Agens-Volundr-32B-Preview-DFlash2:/models/Agens-Volundr-32B-Preview-DFlash2:ro \
  ghcr.io/blockwayz/agens-sglang:preview-sm89 \
  python3 -m sglang.launch_server \
  --model-path /models/Agens-Volundr-32B-Preview \
  --speculative-algorithm DFLASH \
  --speculative-draft-model-path /models/Agens-Volundr-32B-Preview-DFlash2 \
  --speculative-num-draft-tokens 8 \
  --speculative-draft-model-quantization unquant \
  --tp-size 1 \
  --attention-backend flashinfer \
  --page-size 1 \
  --disable-radix-cache \
  --disable-prefill-cuda-graph \
  --mem-fraction-static 0.92 \
  --context-length 165000 \
  --chunked-prefill-size 2048 \
  --max-mamba-cache-size 8 \
  --max-running-requests 8 \
  --cuda-graph-max-bs 8 \
  --reasoning-parser agens \
  --tool-call-parser agens \
  --served-model-name agens-volundr-32b \
  --host 0.0.0.0 --port 30000

The one gotcha that will bite you on first start: with the DFlash2 chain-verify CUDA graphs (bs >= 4, 8 draft tokens), the BCSA wrappers' default 128 MB FlashInfer workspace overflows during graph capture — aligned_alloc ... 147849216 ... but only 134217728 bytes available in AlignedAllocator. It's env-tunable: VOLUNDR_FI_WORKSPACE_MB=256 (it's read at import time by the model class, hence the -e). Everything else follows the README's DFlash2 recipe; we sized --context-length 165000 and the slot counts for single-user long-context serving.

Flags worth keeping: --attention-backend flashinfer is required for spec decode (the BCSA verify kernel lives there); --disable-radix-cache because Engram needs the request's raw token ids (a prefix-cache hit would resume ids it never saw); --disable-prefill-cuda-graph because Engram/BCSA prefill isn't graph-safe; --page-size 1 because BCSA reads pooled far-field K/V straight out of the token-indexed cache. First start compiles Triton kernels for your GPU — a few minutes, normal.

Notes: reasoning comes back in reasoning_content with the agens parser; the template accepts reasoning_effort of xhigh/medium/low only (anything else is a template error — mind proxies that pass "high"); DFlash2 is a single-user speedup — measured ~3x on this setup's sibling runs, and it's the wrong tool at high concurrency per the README.