Qwen2.5-Math-1.5B-RoPE-300k

Weights are identical to Qwen/Qwen2.5-Math-1.5B. Only config.json is changed.

Mirrors what open-r1/Qwen2.5-Math-7B-RoPE-300k did for the 7B variant.

Config changes vs base

Field	Base (Qwen2.5-Math-1.5B)	This model
rope_theta	10000	300000.0
max_position_embeddings	4096	32768
sliding_window	4096	null
use_mrope	false	removed
rope_scaling	—	null

Why

Increasing rope_theta 10k → 300k compresses RoPE frequencies so that the angular range seen during training (positions 0–4096) maps to positions 0–32768 at inference. No fine-tuning required — the model has seen all the relevant rotation angles; they are just stretched across a wider position range.

The 1.5B model has head_dim=128 (hidden_size=1536 / 12 heads). The compression ratio for the slowest frequency dimension is ~28×, placing rope300k position 32768 at an effective base-training position of ~1152 — well within the original training distribution.