kernelpool/GLM-5.3-5bit-UVMAX

Mixed-precision (UVMAX) quantization of zai-org/GLM-5.3-BF16, converted with mlx-lm from the bf16 release.

What is UVMAX?

UVMAX assigns bit widths per tensor class instead of quantizing uniformly.

tensor class	precision	parameters	size	share
Expert FFN gate/up	4-bit, group 128	483B	239.1 GiB	59.1%
Expert FFN down	5-bit, group 128	242B	147.7 GiB	36.5%
Attention + DSA indexer + dense MLP	8-bit, group 64	13.7B	13.6 GiB	3.4%
Shared experts	8-bit, group 64	2.8B	2.8 GiB	0.7%
Embeddings, LM head	4-bit, group 64	1.9B	1.0 GiB	0.2%
Routers	bf16	0.1B	0.2 GiB	<0.1%
Norms	bf16	—	—	<0.1%
total	4.67 bits/weight	743B	404 GiB

Quality

Teacher-forced against the bf16 release on identical tokens, 48 windows of 1025 tokens. KLD is KL(bf16 ‖ quant) over the full output distribution and is corpus-specific.

UVMAX (4.67 bits/weight)

corpus	ppl bf16	ppl UVMAX	ratio	mean KLD	median KLD	top-1 agreement
Linux kernel C	1.434	1.459	1.02×	0.040	0.0002	96.8%
XNU kernel C	2.435	2.472	1.02×	0.056	0.0027	93.4%
JavaScriptCore C++	1.757	1.791	1.02×	0.055	0.0007	95.1%
English prose	2.687	2.750	1.02×	0.063	0.0076	92.4%
all	2.015	2.053	1.02×	0.053	0.0012	94.4%

Uniform 4-bit (4.50 bits/weight)

corpus	ppl bf16	ppl 4-bit	ratio	mean KLD	median KLD	top-1 agreement
Linux kernel C	1.434	1.492	1.04×	0.076	0.0004	95.3%
XNU kernel C	2.435	2.522	1.04×	0.109	0.0063	91.0%
JavaScriptCore C++	1.757	1.841	1.05×	0.102	0.0014	93.4%
English prose	2.687	2.909	1.08×	0.143	0.0209	88.6%
all	2.015	2.119	1.05×	0.107	0.0028	92.1%

Use with mlx

Requires a recent mlx-lm with GLM-5.3 (glm_moe_dsa) support.

The model fits on a single 512 GB machine:

mlx_lm.server --model kernelpool/GLM-5.3-5bit-UVMAX

Sampling follows the base model: temperature 1.0, top-p 0.95.