Instructions to use ZeusLabs/Chronos-Platinum-72B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use ZeusLabs/Chronos-Platinum-72B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="ZeusLabs/Chronos-Platinum-72B")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("ZeusLabs/Chronos-Platinum-72B")
model = AutoModelForCausalLM.from_pretrained("ZeusLabs/Chronos-Platinum-72B", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Inference
Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use ZeusLabs/Chronos-Platinum-72B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "ZeusLabs/Chronos-Platinum-72B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ZeusLabs/Chronos-Platinum-72B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/ZeusLabs/Chronos-Platinum-72B

SGLang

How to use ZeusLabs/Chronos-Platinum-72B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "ZeusLabs/Chronos-Platinum-72B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ZeusLabs/Chronos-Platinum-72B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "ZeusLabs/Chronos-Platinum-72B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ZeusLabs/Chronos-Platinum-72B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use ZeusLabs/Chronos-Platinum-72B with Docker Model Runner:
```
docker model run hf.co/ZeusLabs/Chronos-Platinum-72B
```

Chronos-Platinum-72B / README.md

elinas

Update README.md

96a3bdc verified almost 2 years ago

preview code

Raw

History Blame Contribute Delete

2.85 kB

	---
	base_model:
	- Qwen/Qwen2.5-72B
	tags:
	- roleplay
	- storywriting
	- qwen2.5
	- finetune
	- transformers
	- pytorch
	---

	# Zeus Labs ~ Chronos-Platinum-72B

	![image/png](https://huggingface.co/static-proxy/cdn-uploads.huggingface.co/production/uploads/630417380907b9a115c6aa9f/G05mAhqcp4S_WBfE2vBLl.png)

	Qwen 2.5 72B base model, trained for two epochs on the Chronos Divergence dataset using ChatML. It works well for roleplaying and storywriting as well as general assistant tasks.

	## Instruct Template

	This model uses `ChatML` - below is an example. It is a preset in many frontends.

	```
	<\|im_start\|>system
	You are a helpful assistant<\|im_end\|>
	<\|im_start\|>user
	Hello there!<\|im_end\|>
	<\|im_start\|>assistant
	Hi! I'm an AI assistant, designed to help people like you with all sorts of tasks. Is there anything you need help with?<\|im_end\|>
	<\|im_start\|>user
	I was wondering how transformers work?<\|im_end\|>
	<\|im_start\|>assistant
	```

	## Quantizations
	Please note that we tested this model with a 5.0bpw EXL2 quant. Results are not expected to be the same when going below this quanitzation. Thanks to our model quanters!

	#### LlamaCPP (GGUF)
	[bartowski](https://huggingface.co/bartowski/Chronos-Platinum-72B-GGUF)

	[mradermacher](https://huggingface.co/mradermacher/Chronos-Platinum-72B-i1-GGUF)

	#### Exllama2
	[bartowski](https://huggingface.co/bartowski/Chronos-Platinum-72B-exl2)

	## Sampling Settings
	Here are some settings that work well with this model:
	```
	Temp -> 0.7 - 1.2
	Min P -> 0.025 - 0.05 [temp in order, not last]
	Presence Penalty -> 1.0
	Repetition Penalty range -> 4000
	```
	Higher temp gives more uniqueness and less repetition. Please do not take these settings as the "best" - your system prompt matters significantly, and if you're roleplaying
	use the Basic system prompt in SillyTavern. You can also try other samplers like Top P.

	Note that Presence Penalty works with Repetition Penalty Range.

	## Credit
	Thank you to my team consisting of [@ToastyPigeon](https://huggingface.co/ToastyPigeon), [@Fizzarolli](https://huggingface.co/Fizzarolli), and myself [@elinas](https://huggingface.co/elinas).

	Additional thanks to [@AlpinDale](https://huggingface.co/AlpinDale) and the rest of the PygmalionAI team for graciously providing the compute to finetune this model!
	Thank you to [anthracite-org](https://huggingface.co/anthracite-org) as well for sponsoring this model.

	## Additional Details

	We used a combination of provided logs and WizardLM evol both cleaned up and de-slopped.

	Thanks to Anthropic and OpenAI for the models used to generate synthetic and partially synthetic data to train this model.

	Thanks Elon Musk for being based enough to train AI that compares to the top models.

	If you have any questions or concerns, please post in the community tab.

	DISCLAIMER: Outputs generated by the model are not reflective of our views.