ArkAiLab-Adl/LFM2.5-1.2B-Instruct-Quantized

LFM2.5-1.2B-Instruct (Quantized)

This repository contains quantized variants of the LiquidAI/LFM2.5-1.2B-Instruct model.

The weights provided here are quantized only for improved inference efficiency.
No additional training, fine-tuning, or architectural modifications have been performed.

---

Model Overview

• Base model: LiquidAI/LFM2.5-1.2B-Instruct
• Parameters: ~1.2B
• Type: Instruction-tuned causal language model
• Quantization performed by: ArkAiLab

---

Available Variants

The following variants are provided in this repository:

fp16/ → FP16 baseline weights

int8/ → 8-bit quantization using bitsandbytes

nf4/ → 4-bit NF4 quantization using bitsandbytes (double quant)

Each variant is stored in its own directory and includes the required configuration and tokenizer files.

---

Usage

Example loading a specific variant with Hugging Face Transformers:

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

model_path = "ArkAiLab-Adl/LFM2.5-1.2B-Instruct-Quantized/nf4"

tokenizer = AutoTokenizer.from_pretrained(
    model_path,
    trust_remote_code=True
)

model = AutoModelForCausalLM.from_pretrained(
    model_path,
    device_map="auto",
    trust_remote_code=True
)

---

Quantization Details

• INT8 and NF4 variants use the bitsandbytes backend.

• NF4 is recommended for the best trade-off between memory usage and output quality.

• All variants have been validated with basic text generation tests.

License

This repository follows the same license as the original model:

• LiquidAI/LFM2.5-1.2B-Instruct Refer to the original model repository for full license terms.

Attribution

Original model:

• LiquidAI/LFM2.5-1.2B-Instruct Quantized variants:
• Provided by ArkAiLab

Disclaimer

These weights are provided as-is for research and development purposes. Performance and output quality may vary depending on hardware, software versions, and inference configuration.