Lernex/Metis-1.6-Praxis-NVFP4

Technical report  ·  Metis-1.6  ·  Lernex  ·  X

Praxis in BF16  ·  FP8  ·  NVFP4

Metis-1.6 Praxis NVFP4

Metis-1.6 Praxis in NVFP4 for Dromos, Lernex's inference engine: a third of the size at 2.01 GB, with 99.0% of BF16's benchmark average, and it decodes 27% faster than BF16 for one stream.

Benchmarks

Across seven zero-shot tasks, NVFP4 keeps 99.0% of BF16's average score, and 98.2% with --fast-math, the faster maths explained after Run it.

Task	Metric	BF16	NVFP4	NVFP4 --fast-math
MMLU	acc	37.61	36.00	35.71
HellaSwag	acc_norm	56.09	55.66	55.24
ARC-Easy	acc_norm	54.38	54.12	53.66
ARC-Challenge	acc_norm	34.98	34.22	33.79
OpenBookQA	acc_norm	38.00	37.60	37.60
PIQA	acc_norm	73.23	72.42	72.47
WinoGrande	acc	58.48	59.35	58.09
Average		50.40	49.91	49.51

Each task is scored by log-likelihood over every item in lm-evaluation-harness 0.4.13. The NVFP4 columns decode this repository's weights exactly and run them in the reference implementation with Dromos's activations: BF16 by default, and for --fast-math each input rounded the way Dromos rounds it on an RTX 50 series GPU, to FP4 for the experts and to FP8 per token for the other projections. The BF16 model's card scores generated answers instead, so compare these columns with each other, not with that card.

Run it

curl -fsSL https://github.com/lernex/dromos/releases/latest/download/install.sh | sh   # Linux, macOS
dromos run praxis:nvfp4

On Windows, install with irm https://github.com/lernex/dromos/releases/latest/download/install.ps1 | iex; pip install dromos works everywhere. praxis:nvfp4 is Dromos's short name for this repository, which it downloads the first time, and dromos serve praxis:nvfp4 serves the OpenAI API on http://127.0.0.1:8080/v1.

The model's thinking comes back in reasoning_content and tool calls are parsed into tool_calls. Thinking is on by default; send "chat_template_kwargs": {"enable_thinking": false} for direct answers, or run dromos run praxis:nvfp4 --think off. Dromos applies the recommended sampling from generation_config.json by itself: temperature 0.8, top-p 0.95, top-k 20 and repetition penalty 1.05 when thinking, and temperature 0.3 with no penalty for direct answers. For code in thinking mode, use temperature 1.0 with no penalty.

It runs wherever Dromos does: NVIDIA GPUs through CUDA 12 and 13, Apple silicon through Metal, and CPUs. A 6 GB GPU holds it entirely, with room for 18 sequences of 8,192 tokens.

vLLM, SGLang and Transformers do not read this format. Use Praxis FP8 with vLLM or SGLang, and Praxis with Transformers.

What --fast-math does

Dromos can multiply this model's weights in two ways.

• By default it expands the stored weights to 16 bits as it multiplies and does the maths in BF16, as the original model does. The weights are the only thing rounded.
• With --fast-math it multiplies on the GPU's 4-bit and 8-bit tensor cores instead, which means rounding what goes into each multiplication too. That is faster on the GPUs that have those instructions, and a little less accurate: it keeps 98.2% of BF16's benchmark average, against 99.0% by default.

On an RTX 5070 Ti, Dromos 0.1.0 reads a 4,096-token prompt at 20,103 tokens per second with --fast-math, against 16,339 by default: 23% faster.

It only changes the steps where the GPU handles many tokens at once: reading the prompt, and generating for more than 8 conversations together. A single conversation writes its answer one token at a time with the default maths either way, because that step's speed is set by how fast the GPU reads the weights, not by the multiplying. The answer can still come out slightly different, since it builds on the prompt as --fast-math read it.

Your GPU	What --fast-math changes
RTX 50 series, RTX PRO Blackwell	The 4-bit experts multiply in FP4 and the 8-bit projections in FP8, as measured in Benchmarks
RTX 40 series, H100	The 8-bit projections multiply in FP8; the 4-bit experts stay in BF16
RTX 30 series, A100, Apple silicon, CPUs	Nothing: without FP8 or FP4 tensor cores, Dromos keeps the default

Turn it on when reading the prompt is the slow part: long documents, a large codebase in context, or serving many people at once. For everyday chat, leave it off; the default stays closer to the original model.

dromos serve praxis:nvfp4 --fast-math

Once the model is downloaded, dromos doctor praxis:nvfp4 shows what it changes on your GPU.

Speed

Decode speed on an RTX 5070 Ti (16 GB) in tokens per second with the CUDA 13 build of Dromos 0.1.0, with 512 prompt tokens and 256 generated per stream, greedy decoding without speculation, counting the tokens generated once every stream is decoding:

	1 stream	32 streams	On disk
BF16, Metis-1.6-Praxis	353.1	2,386	6.16 GB
NVFP4, this repository	447.5	3,053	2.01 GB

How close it stays to BF16

Mean top-20 KL divergence from the native model's FP32 logits, and how often the top token agrees, over the whole prompt, on the release's three reference prompts: a thinking chat, a chat with a tool call and a prose passage. None of them is in the calibration set. For comparison, Praxis FP8 measures 0.036 to 0.040 on the tool prompt and 0.0066 to 0.0077 on prose in vLLM and SGLang. Lower is closer; 0 would be the original exactly.

Maths	Thinking chat	Chat with tools	Prose
BF16 (default)	0.014, 100.0%	0.056, 88.6%	0.0092, 98.0%
--fast-math	0.044, 100.0%	0.11, 83.5%	0.020, 95.7%

What is in NVFP4

The routed experts, most of the model's parameters, are NVFP4: 4-bit E2M1 values with one FP8 (E4M3) scale per 16 inputs and an FP32 scale per expert. The attention and Mamba-2 input projections, the projection into the experts' latent space, the shared expert, the depth memory and the tied embedding are FP8 with one scale per output row. What this architecture is most sensitive to stays as trained: every Mamba-2 output projection and every projection out of the latent space stay BF16, and the routers, norms and Mamba-2's A, D and dt bias keep their native precision. Every 8-bit weight and all but 7 of the 3,072 expert weight matrices were rounded with GPTQ (Hessian error compensation in activation order); the other 7, experts routed too rarely in calibration to estimate their Hessian, use round-to-nearest. The calibration set is 352 windows of 2,048 tokens, and none of it is evaluation data: WikiText-103, Magicoder OSS-Instruct code, GSM8K and MATH training problems, Aya conversations in 24 languages, UltraChat conversations, long-form essays, and Praxis FP8's own tool-call transcripts. Every document was checked against the 46 benchmark datasets on our evaluation machine and the three reference prompts above, and any that shared a 13-token run with one was left out.

quantization_config.json lists the recipe tensor by tensor, calibration_mix.json the calibration set, and quantization_receipt.json the checksums and per-module errors of the conversion.

Citation

If Metis helps your work, please cite the technical report, and the model if you used it directly:

@misc{vollmer2026thinktwice,
  title         = {Think Twice: The Metis-1.6 Technical Report},
  author        = {Vollmer, Giulianno},
  year          = {2026},
  eprint        = {2610.37903},
  archivePrefix = {arXiv},
  primaryClass  = {cs.LG},
  url           = {https://arxiv.org/abs/2610.37903}
}

@misc{vollmer2026praxis,
  title     = {{Metis-1.6 Praxis}},
  author    = {Vollmer, Giulianno},
  year      = {2026},
  publisher = {Hugging Face},
  url       = {https://huggingface.co/Lernex/Metis-1.6-Praxis}
}

License

Apache 2.0, the same as the original model.