F2LLM-v2-1.7B — GGUF (Q8_0)

A GGUF conversion of codefuse-ai/F2LLM-v2-1.7B for use with llama.cpp on device. No changes to the weights other than the conversion and Q8_0 quantisation. All credit for the model belongs to the CodeFuse authors; it is redistributed here under the Apache-2.0 licence of the original.

Why this exists

The upstream repository publishes safetensors only. This is the same model in GGUF form so it can be loaded by llama.cpp on iOS.

Conversion

# llama.cpp b10192 (SHA 9ebfc3a8cf4c1c6983258c4d603274b2b3d3dd65)
python convert_hf_to_gguf.py --outtype f16 --outfile f2llm-f16.gguf <src>
llama-quantize f2llm-f16.gguf F2LLM-v2-1.7B-Q8_0.gguf Q8_0


Architecture	`qwen3` (Qwen3Model), 28 layers
Embedding dimension	2048
Pooling	last-token (`qwen3.pooling_type = 3`)
Normalisation	L2 (apply in your client)
`add_eos_token`	true
Size	1749 MiB
sha256	`4ba9abad93b1d46d162650da80ca4f1ebc29022fed51cae59aa9e2d5aa024ad3`

Usage notes

Query and document sides are asymmetric. Prefix queries with the instruction form and encode documents raw:

Instruct: Given a question, retrieve passages that can help answer the question.
Query: <your query>

Verified on device (iPhone 16e, Metal) against a hard multilingual retrieval set covering de/fr/ja/ko/zh — correct ranking in all five, with the widest gold-vs-distractor margins in ja/ko/zh.