Victoria

44% of the experts cut (512 to 288 per layer), retrained at 4-bit, and still 70.0% on Terminal-Bench 2.1 (avg@3).

Victoria is an open-weight coding and agent model from Farpoint Technologies, a Canadian company. We took a larger open-weight model, removed 44% of its experts (512 down to 288 per layer), and retrained the result at 4-bit to restore quality. It runs on a single GPU while keeping most of the original model's coding and agent ability. The original model and its licence are listed under Licence and attribution.

See also: Maple, a Canadian-first model built on Victoria.

Highlights

  • Fast decode. 5.9B active parameters per token, unchanged by the expert cut. With the included draft head on, the NVFP4 build decodes 279.6 tok/s on one stream on one NVIDIA B300, 2.08x its 134.7 tok/s with the head off.
  • 70.0% on Terminal-Bench 2.1, avg@3. Three full runs of all 89 tasks at 8 hours per task scored 75.3%, 68.5% and 66.3%. The previous NVFP4 build scored 62.5% on the identical protocol.
  • Size. 48.0 GiB of weights including the draft head, plus the model's 95.4 GiB n-gram lookup table. The GGUF build holds 49.17 GiB of weights on llama.cpp.
  • Fewer wasted tokens. The new build used 35% fewer output tokens than the previous NVFP4 build (69.4M against 107.3M across three runs).
  • Two runtimes. NVFP4 for vLLM on NVIDIA Blackwell GPUs, and GGUF Q4_K_M for llama.cpp.

Builds

The two builds are different checkpoints: the NVFP4 build is the latest checkpoint, and the GGUF build is the checkpoint from the first release. Each build's scores were measured on that build only.

Build Runtime Download Resident weights Terminal-Bench 2.1, 8 h per task HumanEval
nvfp4/ vLLM, NVIDIA B300 or B200 154.0 GB (143.4 GiB) 48.0 GiB of weights plus the 95.4 GiB lookup table; vLLM holds both on the GPU 70.0% avg@3 (75.3 / 68.5 / 66.3) 97.0% (159/164), one sample per problem
gguf/ Q4_K_M llama.cpp 107.20 GB (smaller download, same model) or 155.20 GB (full-precision lookup table) 52.79 GB (49.17 GiB) 75.28% (67/89), one run 93.2% avg@5

Quick start

Download:

hf download rmonsurate/Victoria --include "nvfp4/*" --local-dir .

# full GGUF set, 155.20 GB
hf download rmonsurate/Victoria --include "gguf/victoria-s410-bitexact-0*" --local-dir .

# smaller download, 107.20 GB: the model's lookup table is stored at 8-bit, GPU memory use is the same
hf download rmonsurate/Victoria --include "gguf/victoria-s410-bitexact-tbl8-*" --local-dir .

nvfp4/SHA256SUMS lists the sha256 of every file in the NVFP4 build.

NVFP4 on vLLM, one GPU, draft head on. This is the command the decode speeds below were measured with:

vllm serve /path/to/nvfp4 \
  --served-model-name victoria \
  --tensor-parallel-size 1 \
  --max-model-len 20480 \
  --max-num-seqs 16 \
  --max-num-batched-tokens 16384 \
  --gpu-memory-utilization 0.80 \
  --enable-prefix-caching \
  --compilation-config '{"inductor_compile_config":{"triton.autotune_at_compile_time":false}}' \
  --kernel-config '{"enable_flashinfer_autotune":false}' \
  --trust-remote-code \
  --enable-auto-tool-choice \
  --tool-call-parser qwen3_coder \
  --reasoning-parser qwen3 \
  --speculative-config '{"method":"mtp","num_speculative_tokens":3}'
  • The --compilation-config line is required: compile-time Triton autotuning on this model either hangs or blows out startup time.
  • For long agent sessions, the Terminal-Bench runs used --max-model-len 262144 --max-num-seqs 64 with everything else the same, minus --speculative-config.
  • We ran vLLM from the upstream vLLM image for the original model, with two fixes backported for a prefix-caching crash on this architecture (vllm#53798 and vllm#54076, open upstream when we built it). On a vLLM without those fixes, leave out --enable-prefix-caching.

GGUF build: use our llama.cpp build

The GGUF files include the model's draft head, stored as 32 extra tensors. Mainline llama.cpp does not know about this head yet, so it refuses to load the files with an error like wrong number of tensors; expected 1256, got 1224. Your download is fine. The files are complete.

The easiest fix is a prebuilt llama.cpp from rmonsurate/llama.cpp-victoria. It is stock llama.cpp b11276 plus one patch for the draft head, with zips for Windows (CUDA and CPU), Linux (CUDA and CPU) and macOS. The same files are on the GitHub release. We are working on getting the patch into mainline llama.cpp.

Run it with the draft head on:

llama-server -m gguf/victoria-s410-bitexact-tbl8-00001-of-00003.gguf -ngl 99 -fa on -c 8192 --spec-type draft-mtp --spec-draft-n-max 3

Or with the draft head off:

llama-server -m gguf/victoria-s410-bitexact-tbl8-00001-of-00003.gguf -ngl 99 -fa on -c 8192

On an M3 Max with 128 GB, one request at a time, the head took generation from 26.8 to 27.8 tok/s (off) to 34.3 to 38.0 tok/s (on), with 70.4% of drafted tokens accepted. We have not measured it on CUDA yet. For the scores on this card we ran 8 slots at 16384 context each across two GPUs, with speculative decoding off.

If you would rather build it yourself, clone the branch:

git clone --branch qwen4exp-draft-mtp https://github.com/rmonsurate/llama.cpp
cd llama.cpp

# NVIDIA GPU (CUDA)
cmake -B build -DGGML_CUDA=ON
cmake --build build --config Release -j

# or CPU only, or Metal on a Mac
# cmake -B build
# cmake --build build --config Release -j

The server binary lands in build/bin/ (on Windows, build\bin\Release\). You can also apply victoria-draft-mtp-b11276.patch from the repo above to your own checkout with git am.

Evaluation

NVFP4 build

Model Terminal-Bench 2.1, avg@3 Run 1 Run 2 Run 3
Victoria NVFP4 (this release) 70.04% 67/89 61/89 59/89
Victoria NVFP4 (previous build) 62.54% 57/89 54/89 56/89

Counting passes out of three runs per task, this release does better than the previous build on 18 tasks and worse on 6, and ties on the rest (two-sided exact sign test, p = 0.023). Harbor 0.22.0 with the Terminus 2 agent, 8 hours per task, 262144 context, all 89 tasks in every denominator; a trial with no verdict counts as a fail.

HumanEval on the NVFP4 weights is 97.0% (159/164): one sample per problem, thinking on, temperature 0.6, top_p 0.95, top_k 20. That is a single run, not an average over five runs like the GGUF figure, so the two numbers are not directly comparable.

Decode speed and the draft head

Setup Output tok/s, mean of 3 runs (each run) Draft acceptance Tokens per forward pass of the main model
Draft head off 134.7 (133.7, 133.6, 136.9) n/a 1
Draft head as pruned, never retrained 269.3 (270.4, 269.8, 267.6) 64.1% 2.92
Trained draft head, shipped in nvfp4/ 279.6 (284.0, 278.5, 276.2) 67.6% 3.03

One NVIDIA B300, one stream, the vLLM command above, 50 fixed prompts (20 coding, 30 agentic), temperature 0.6, top_p 0.95, thinking off, up to 3 draft tokens per step, a fresh server for each run. Acceptance is accepted draft tokens over proposed draft tokens. The draft head only proposes tokens and the main model verifies every one, so it changes speed, not answers.

GGUF build

Model Terminal-Bench 2.1 Kept vs original
Victoria GGUF Q4_K_M 75.28% (67/89) 84.8%
Original full-size model, before the expert cut (see Licence and attribution) 88.76% (79/89) baseline

Setup: Harbor 0.22.0 with the Terminus 2 agent, 8 hours per task, avg@1, all 89 tasks in the denominator, 262144 context. One avg@1 run over 89 tasks carries a standard error of roughly 5 points. HumanEval on the same weights is 93.2% avg@5, sd 1.3, at n=164. We evaluated text only.

Training curves

Drawn from the training logs of the NVFP4 build (steps 510 to 810) and of the draft head.

Training loss Training loss.

Held-out divergence from the full model Held-out divergence from the full model, agentic and coding slices. Lower is closer to the original model.

Early-stop tendency while thinking How much the model leans toward ending its turn while still thinking. Lower means fewer empty agent turns.

Draft head training Draft head training: divergence from the main model and draft acceptance rate.

Acknowledgements

This work was made possible by the support of Dell, who provided lab access to the 8x NVIDIA B300 system it was built and evaluated on, and of Mario at Buzz (NASDAQ: HIVE). Thanks to the Qwen team for the original model, and to the maintainers of llama.cpp, vLLM, harbor and Terminal-Bench.

Licence and attribution

Victoria is a modified version of Qwen3.8-Flash-Next, an open-weight model released by Qwen. Farpoint Technologies removed 44% of its experts, retrained it at 4-bit and trained its draft head. Victoria is distributed under the same licence as the original, the Qwen Community License 1.0. The full text, including Qwen's copyright notice, ships in this repo as LICENSE and must be kept with every copy.

Two further conditions in that licence pass on to anyone who uses Victoria:

  • A licensee running a Model as a Service or AI Work Assistant business must obtain a separate license from Qwen before any commercial use.
  • A commercial product or service with more than 100,000,000 monthly active users or US$20,000,000 monthly revenue must display the model name prominently in its user interface.

Read the licence text, and talk to Qwen if either condition describes what you are building. Attribution goes to Qwen for Qwen3.8-Flash-Next and to Farpoint Technologies for Victoria.

Downloads last month
996
GGUF
Model size
126B params
Architecture
qwen4exp
Hardware compatibility
Log In to add your hardware

We're not able to determine the quantization variants.

Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support