Instructions to use ajibawa-2023/SlimOrca-13B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use ajibawa-2023/SlimOrca-13B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="ajibawa-2023/SlimOrca-13B")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("ajibawa-2023/SlimOrca-13B")
model = AutoModelForCausalLM.from_pretrained("ajibawa-2023/SlimOrca-13B")

Inference
Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use ajibawa-2023/SlimOrca-13B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "ajibawa-2023/SlimOrca-13B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ajibawa-2023/SlimOrca-13B",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/ajibawa-2023/SlimOrca-13B

SGLang

How to use ajibawa-2023/SlimOrca-13B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "ajibawa-2023/SlimOrca-13B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ajibawa-2023/SlimOrca-13B",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "ajibawa-2023/SlimOrca-13B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ajibawa-2023/SlimOrca-13B",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use ajibawa-2023/SlimOrca-13B with Docker Model Runner:
```
docker model run hf.co/ajibawa-2023/SlimOrca-13B
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

SlimOrca-13B: A General Purpose Intelligent Model

This Model is trained on refined version of SlimOrca made available by Open-Orca team. The idea was to check how this Model will perform in the absence of "system" prompt/instruction. This Model is very good in various types of General Purpose content generation such as Q&A (including multiple choice), Articles from Summary, Sentiment Analysis, Context & Hypothesis, Reviews, Erotic story generation etc. It can also generate Uncensored content. Kindly be careful while generating Uncensored content as you will be responsible for what you generate.

It is trained on 517981 set of conversations. Each set having 2 conversations. I have shared this data.

All the credit goes to the Open-Orca team for releasing SlimOrca dataset.

Training: Entire dataset was trained on Azure 4 x A100 80GB. For 3 epoch, training took almost 11 Days. DeepSpeed codebase was used for training purpose. Entire data is trained on Llama-2 by Meta.

This is a full fine tuned model. Links for quantized models are given below.

GPTQ GGML & AWQ

GPTQ: Link

GGUF: Link

AWQ: Link

Special Thanks to TheBloke for making these models available.

Example Prompt:

This is a conversation with your Assistant. It is a computer program designed to help you with various tasks such as answering questions, providing recommendations, and helping with decision making. You can ask it anything you want and it will do its best to give you accurate and relevant information.

Context
You are a helpful AI assistant.

USER: <prompt>
ASSISTANT:

You can modify above Prompt as per your requirement. I have used ShareGPT/Vicuna format v1.1 .

I want to say special Thanks to the Open Source community for helping & guiding me to better understand the AI/Model development.

Thank you for your love & support.

Example Output

Example 1

Example 2

Open LLM Leaderboard Evaluation Results

Detailed results can be found here

Metric	Value
Avg.	60.39
AI2 Reasoning Challenge (25-Shot)	60.15
HellaSwag (10-Shot)	81.40
MMLU (5-Shot)	57.04
TruthfulQA (0-shot)	49.37
Winogrande (5-shot)	74.43
GSM8k (5-shot)	39.95

Downloads last month: 869

Model tree for ajibawa-2023/SlimOrca-13B

Quantizations

6 models

Datasets used to train ajibawa-2023/SlimOrca-13B

Evaluation results

normalized accuracy on AI2 Reasoning Challenge (25-Shot)
test set Open LLM Leaderboard

60.150
normalized accuracy on HellaSwag (10-Shot)
validation set Open LLM Leaderboard

81.400
accuracy on MMLU (5-Shot)
test set Open LLM Leaderboard

57.040
mc2 on TruthfulQA (0-shot)
validation set Open LLM Leaderboard

49.370
accuracy on Winogrande (5-shot)
validation set Open LLM Leaderboard

74.430
accuracy on GSM8k (5-shot)
test set Open LLM Leaderboard

39.950