Instructions to use koute/GLM-4.7-Flash-Derestricted with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use koute/GLM-4.7-Flash-Derestricted with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="koute/GLM-4.7-Flash-Derestricted")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("koute/GLM-4.7-Flash-Derestricted")
model = AutoModelForCausalLM.from_pretrained("koute/GLM-4.7-Flash-Derestricted")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use koute/GLM-4.7-Flash-Derestricted with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "koute/GLM-4.7-Flash-Derestricted"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "koute/GLM-4.7-Flash-Derestricted",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/koute/GLM-4.7-Flash-Derestricted

SGLang

How to use koute/GLM-4.7-Flash-Derestricted with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "koute/GLM-4.7-Flash-Derestricted" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "koute/GLM-4.7-Flash-Derestricted",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "koute/GLM-4.7-Flash-Derestricted" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "koute/GLM-4.7-Flash-Derestricted",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use koute/GLM-4.7-Flash-Derestricted with Docker Model Runner:
```
docker model run hf.co/koute/GLM-4.7-Flash-Derestricted
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

GLM-4.7-Flash-Derestricted

This is a GLM-4.7-Flash model which has been uncensored using the Norm-Preserving Biprojected Abliteration methodology, similar to other models from the 'derestricted' family.

Benchmark results

All benchmarks were measured using a local vLLM instance and inspect_evals.

MMLU-Pro (subset of 200 samples picked at random)

GLM-4.7-Flash: 0.715, 694606 output tokens
GLM-4.7-Flash-Derestricted: 0.755, 632992 output tokens

Measured with:

LOCAL_API_KEY="dummy" LOCAL_BASE_URL="http://127.0.0.1:9001/v1" uv run inspect eval inspect_evals/mmlu_pro --model "openai-api/local/glm-4.7-flash-derestricted" --seed 123456 --reasoning-history all --log-dir eval-logs-glm-4.7-flash-derestricted-mmlu-pro --frequency-penalty 0 --presence-penalty 0 --temperature 0.7 --top-p 0.95 --max-tokens 8192 --max-connections 200 --sample-shuffle 6375934876 --limit 200

Downloads last month: 77

Model tree for koute/GLM-4.7-Flash-Derestricted

Base model

zai-org/GLM-4.7-Flash

Finetuned

(66)

this model

Quantizations

3 models