Dominic/smollm135_fairy2i_tinystories

SmolLM-135M model with Fairy2i packed weights (QAT), fine-tuned on TinyStories. Trained for 12k steps on 200k train stories with Fairy2i QAT (phase quantization and packing) and eval on the published validation split (~7.9 perplexity for the packed checkpoint).

See https://huggingface.co/Dominic/smollm135_fullprec_tinystories for comparable full precision (BF16) model.

Minimal generation steps:

GIT_LFS_SKIP_SMUDGE=1 git clone --depth 1 https://huggingface.co/Dominic/smollm135_fairy2i_tinystories
cd smollm135_fairy2i_tinystories
git clone https://github.com/PKULab1806/Fairy2i-W2.git ~/Fairy2i-W2
python generate_tinystories_fairy2i.py

The clone pulls scripts and small files only (large LFS blobs stay as pointers). The first run downloads smollm135_fairy2i_tinystories.safetensors into the Hugging Face cache (HF_HOME).

This model is only intended for generating toy story examples and comparing quantization techniques.