Instructions to use Xkev/Llama-3.2V-11B-cot with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Xkev/Llama-3.2V-11B-cot with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="Xkev/Llama-3.2V-11B-cot") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# Load model directly from transformers import AutoProcessor, AutoModelForImageTextToText processor = AutoProcessor.from_pretrained("Xkev/Llama-3.2V-11B-cot") model = AutoModelForImageTextToText.from_pretrained("Xkev/Llama-3.2V-11B-cot") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] inputs = processor.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=40) print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use Xkev/Llama-3.2V-11B-cot with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "Xkev/Llama-3.2V-11B-cot" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Xkev/Llama-3.2V-11B-cot", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/Xkev/Llama-3.2V-11B-cot
- SGLang
How to use Xkev/Llama-3.2V-11B-cot with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "Xkev/Llama-3.2V-11B-cot" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Xkev/Llama-3.2V-11B-cot", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "Xkev/Llama-3.2V-11B-cot" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Xkev/Llama-3.2V-11B-cot", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }' - Docker Model Runner
How to use Xkev/Llama-3.2V-11B-cot with Docker Model Runner:
docker model run hf.co/Xkev/Llama-3.2V-11B-cot
License compatibility
Hi, I'd like to report a License Conflict in Xkev/Llama-3.2V-11B-cot. I noticed that this model appears to be fine-tuned from meta-llama/Llama-3.2-11B-Vision-Instruct, while being published under the Apache-2.0 license. Given the terms outlined in the LLaMA 3.2 Community License, especially regarding redistribution, attribution, and naming, this combination of licenses could potentially lead to legal or usage misunderstandings.
⚠️ Key violations of LLaMA 3.2 license terms:
Clause 1.b.i – Releasing derivative models must:
• Include a copy of the original license
• Display “Built with Llama”
Clause 1.b.iii – Must retain the following notice in the release:
“Llama 3.2 is licensed under the Llama 3.2 Community License, Copyright © Meta Platforms, Inc.”
🔹Suggestion:
1. It’s a good idea to explicitly include all key clauses of the LLaMA 3.2 Community License, such as:
- A copy of the LLaMA 3.2 license in the repository or model card
- A visible “Built with LLaMA” label on the model card, documentation, or UI
- Including the following notice in a “NOTICE” file or documentation:
> “Llama 3.2 is licensed under the LLaMA 3.2 Community License, Copyright © Meta Platforms, Inc.”
- A statement clarifying that use of the model must comply with Meta’s Acceptable Use Policy
2. Maybe we can just drop the Apache-2.0 tag and going with the LLaMA 3.2 Community License. This approach may help reduce potential confusion about redistribution rights and downstream usage conditions.
Thanks for your attention!
Looking forward to your response!
Thank you very much for submitting the issue! Sorry I just saw it now. I’ve already made the changes according to your suggestion. Thanks again!