cometkim

/

Qwen3.8-27B-nvfp4full-NInfer

Like 9

My published nvfp4full snapshot quantizes nearly every Text linear of Qwen3.8-27B to NVFP4: attention and GDN projections use a locally encoded max-abs profile (with a few BF16 exceptions), while MLP codes come word-for-word from the unsloth community export for layers 0–55 and from my local encoder for layers 56–63.

Now there is NVIDIA's official NVFP4 quantization checkpoint: https://huggingface.co/nvidia/Qwen3.8-27B-NVFP4

Therefore, I did several experiments taking into account a new snapshot of this artifact.

Experiment 1: weight-space fidelity

Relative Frobenius error versus the BF16 reference, 192 MLP matrices (64 layers × gate/up/down):

NVIDIA codes win on 191 of the 192 tensors in the published payload, and NVIDIA's worst tensor is still below unsloth's best. Two side observations: my local max-abs encoder is noise-bounded and self-consistent, while unsloth's words have statistically identical block scales but a non-self-consistent code assignment (only ~50% agreement with an RNE reconstruction on the stored grid), which explains the much wider error spread. And loss-aware QAT training lands in the same error band as NVIDIA PTQ — though for a co-adapted checkpoint this metric cannot rank quality.

For lm_head, NVIDIA keeps it in FP4 (0.0848), a 15× weight-error increase over my int8 head (0.0055) in exchange for roughly 0.6 GiB. I have not adopted that yet.

Experiment 2: the candidate snapshot

I built a candidate with all 128 MLP parents (layers 0–63) swapped to NVIDIA words. Codes and E4M3 block scales are copied byte-exactly; only the two global scalars per site need conversion — ModelOpt stores multipliers (amax/2688), the reciprocal of the positive divisors my engine stores, so each site scalar is inverted with a single fp32 rounding. Every other object of the published snapshot is byte-identical in the new build (1,059 objects compared), so only the MLP payloads change. Notes for anyone else consuming this export: packed codes have the low nibble at even k, weight_scale is a natural [N, K/16] E4M3 layout, and the attention sites are per-tensor FP8 with scalar scales.

Experiment 3: end-to-end gates

Same binary, same workloads, deterministic metrics (greedy speculative acceptance, fixed-corpus perplexity):

Three conclusions follow:

1. The language-model function improves. Perplexity drops by 1.35%, with three of four domains better (Chinese 6.44 → 6.36, English reference 6.60 → 6.32, code 1.689 → 1.687). The better weight fidelity carries through to the function level.
2. But MTP acceptance drops by 7.2 points (roughly −9% MTP decode). This is independent of the activation scales (the variant shows an identical acceptance). The improved target function has moved away from the MTP drafter's agreement surface. It is not a quality regression — perplexity confirms that — but it is a real speculative-throughput cost.
3. Codes and activation scales are co-calibrated. Pairing the NVIDIA codes with the previous d_x scales collapses DFlash2 acceptance to 8.61%. The site system — codes, block scales, global scales — has to be imported whole, never partially mixed.

Status

I have not adopted it yet. The swap buys a clear perplexity improvement for free, at a throughput cost on whichever lane benefits from speculation. Benchmark scores (GPQA, LongBench) are the next check; this post deliberately sticks to deterministic metrics. Thanks to the NVIDIA team for preparing this checkpoint — measured here, it is clearly better than the community export.