Instructions to use bottlecapai/ThinkingCap-Qwen3.8-27B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use bottlecapai/ThinkingCap-Qwen3.8-27B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="bottlecapai/ThinkingCap-Qwen3.8-27B")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("bottlecapai/ThinkingCap-Qwen3.8-27B")
model = AutoModelForMultimodalLM.from_pretrained("bottlecapai/ThinkingCap-Qwen3.8-27B", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use bottlecapai/ThinkingCap-Qwen3.8-27B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "bottlecapai/ThinkingCap-Qwen3.8-27B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "bottlecapai/ThinkingCap-Qwen3.8-27B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/bottlecapai/ThinkingCap-Qwen3.8-27B

SGLang

How to use bottlecapai/ThinkingCap-Qwen3.8-27B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "bottlecapai/ThinkingCap-Qwen3.8-27B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "bottlecapai/ThinkingCap-Qwen3.8-27B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "bottlecapai/ThinkingCap-Qwen3.8-27B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "bottlecapai/ThinkingCap-Qwen3.8-27B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use bottlecapai/ThinkingCap-Qwen3.8-27B with Docker Model Runner:
```
docker model run hf.co/bottlecapai/ThinkingCap-Qwen3.8-27B
```

use with DFLASH2 drafter possible?

by ghit72 - opened Sep 23

Discussion

ghit72

Sep 23

hi,
can this
z-lab/Qwen3.8-27B-DFlash2
be used with your finetune?

Adamiros

BottleCapAI org Sep 23

Hi! Based on our Qwen 3.6 experience it should work well. But we have not tested it for Qwen 3.8 yet.

vcerny

Sep 23

tried that. my average draft acceptance for 7 positions is around 30% which is borderline useful.. I think MTP is way to go currently, unless somebody trains a new speculator.

Adamiros

BottleCapAI org Sep 23

Good to know, thanks!

ghit72 changed discussion status to closed Sep 23

Adamiros

BottleCapAI org Sep 24

In my measurements, ThinkingCap's acceptance rate is consistent with the stock Qwen 3.8. Where it varies is individual task domains.

Benchmark	ThinkingCap accept len	ThinkingCap accept rate	Base accept len	Base accept rate
AIME 2026	3.20	31.4%	3.19	31.3%
GPQA Diamond	3.09	29.9%	3.12	30.3%
LiveCodeBench	2.92	27.5%	2.93	27.6%
GSM8K	4.19	45.5%	4.00	42.9%
τ²-Bench	3.24	32.0%	3.43	34.7%

@vcerny Did you perceive better values when using the stock model?

Adamiros changed discussion status to open Sep 24

bfkxnr

29 days ago

I have not seen any noticeable degradation compared to the base model, and DFlash2 is ~17% faster than MTP in the worst case scenario: prose, temperature=1. I grid-searched the best parameters for both drafters before comparing. I did not test it on long context lengths (60k or higher). That being said, there is always room for improvement and I hope somebody releases a ThinkingCap-calibrated DFlash2 model to squeeze more acceptance out of it.

vcerny

29 days ago

ok, my tests were very limited, so I am happy to see that vanilla DFlash2 actually makes sense. Thanks!

Adamiros

BottleCapAI org 27 days ago

Thanks for sharing your experience. I'll close this with the fact DFlash performs comparably to the stock Qwen, and will let you know if we decide to release an improved drafter.

Adamiros changed discussion status to closed 27 days ago

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment