README.md

metadata

license: mit
library_name: transformers
base_model:
  - deepseek-ai/DeepSeek-V3.2-Speciale

'Make knowledge free for everyone'

EXPERIMENTAL!

Channel wise INT8 quant. Requires CPU with AMX support (Xeon 5 and above) and 700GB-1TB ram. Not tested due to hardware.

SGlang sould support it. SGlang CPU

Made with https://huggingface.co/meituan/DeepSeek-R1-Channel-INT8/blob/main/inference/bf16_cast_channel_int8.py