Instructions to use quantumaikr/quantum-dpo-v0.1 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use quantumaikr/quantum-dpo-v0.1 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="quantumaikr/quantum-dpo-v0.1")# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("quantumaikr/quantum-dpo-v0.1") model = AutoModelForCausalLM.from_pretrained("quantumaikr/quantum-dpo-v0.1", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use quantumaikr/quantum-dpo-v0.1 with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "quantumaikr/quantum-dpo-v0.1" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "quantumaikr/quantum-dpo-v0.1", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/quantumaikr/quantum-dpo-v0.1
- SGLang
How to use quantumaikr/quantum-dpo-v0.1 with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "quantumaikr/quantum-dpo-v0.1" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "quantumaikr/quantum-dpo-v0.1", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "quantumaikr/quantum-dpo-v0.1" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "quantumaikr/quantum-dpo-v0.1", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use quantumaikr/quantum-dpo-v0.1 with Docker Model Runner:
docker model run hf.co/quantumaikr/quantum-dpo-v0.1
Best 7B model for my use case
I have already checked many 7B models, but this one works best for my local llm tool usage project.
The idea in it is to first use the tools one by one based on the question and previous observations obtained, until the second LLM call can fully answer the user's question.
Other models either misused tools, misspelled tools inputs, misinterpreted the obtained observations, or used the same tools repeatedly for some unknown reason.
This model uses only the tools that are currently in demand, quickly and efficiently.
Here an example of an answer to the question "Who is Leo DiCaprio's current girlfriend? What is her age raised to the 0.43 power?":
Thoughts: We need to find Leo DiCaprio's current girlfriend first. For that, we can use the Weather function as it is not relevant to this question. Then, we need to know her age to calculate it to the power of 0.43. For that, we can use the WebSearch function to find her age.
External tool: WebSearch
Inputs: ["Leo DiCaprio's current girlfriend age"]
Observation: Celebs Leonardo DiCaprio's Full Dating History Is鈥 True Journey Alexa, play "What's My Age Again?" By Mehera Bonner Published: Oct 30, 2023 3:02 PM EST Save Article Leonardo DiCaprio... | Special Series Main Character of the Day Leo DiCaprio's dating history is part of our obsession with staying young forever February 9, 20234:58 PM ET Manuela L贸pez Restrepo Enlarge this image... | We're told the 48-year-old A-lister is officially dating 25-year-old model Vittoria Ceretti. The couple has been spotted on several dates in the US and abroad in recent months, sparking... | Leonardo DiCaprio was accompanied by new girlfriend Vittoria Ceretti as he celebrated his 49th birthday with a star-studded soir茅e in Beverly Hills, Calif., Saturday night. The "Titanic" star... | The actor made headlines last year for breaking up with his 25-year-old girlfriend Camila Morrone, further fuelling the theory that he does not date any women over that age. But now, the Hollywood star reportedly appears to be reaching even younger, having been linked to a 19-year-old Israeli model, Eden Polani. | In August 2022, Morrone and DiCaprio split up, which naturally revived the theory that Leo only dates women under the age of 25. Of course, that's a little convoluted considering the current rumours around the actor and 27-year-old Hadid.
Thoughts: We have found that Leo DiCaprio's current girlfriend is Vittoria Ceretti and her age is 25. Now, we need to calculate 25 raised to the power of 0.43.
External tool: Calculator
Inputs: ['25^0.43', '25: Age of Vittoria Ceretti']
def solution():
"""25^0.43 | Additional Explanation: 25: Age of Vittoria Ceretti"""
base = 25
exponent = 0.43
result = base ** exponent
return result
Observation: Result: 3.991298452658078
Thoughts: We have calculated 25 raised to the power of 0.43 and the result is 3.991298452658078. This is the final observation needed to answer the user's question.
External tool: End
Second LLM call to answer the question (final answer):
Leo DiCaprio's current girlfriend is reportedly 25-year-old model Vittoria Ceretti. When you raise 25 to the power of 0.43, the result is approximately 3.9913.