Instructions to use BeaverAI/Behemoth-123B-v2b-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use BeaverAI/Behemoth-123B-v2b-GGUF with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf BeaverAI/Behemoth-123B-v2b-GGUF:Q4_K_M # Run inference directly in the terminal: llama cli -hf BeaverAI/Behemoth-123B-v2b-GGUF:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf BeaverAI/Behemoth-123B-v2b-GGUF:Q4_K_M # Run inference directly in the terminal: llama cli -hf BeaverAI/Behemoth-123B-v2b-GGUF:Q4_K_M
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf BeaverAI/Behemoth-123B-v2b-GGUF:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama cli -hf BeaverAI/Behemoth-123B-v2b-GGUF:Q4_K_M
Use Docker
docker model run hf.co/BeaverAI/Behemoth-123B-v2b-GGUF:Q4_K_M
- Llama new
- LM Studio
- Jan
- Ollama
How to use BeaverAI/Behemoth-123B-v2b-GGUF with Ollama:
ollama run hf.co/BeaverAI/Behemoth-123B-v2b-GGUF:Q4_K_M
- Unsloth Desktop
- Docker Model Runner
How to use BeaverAI/Behemoth-123B-v2b-GGUF with Docker Model Runner:
docker model run hf.co/BeaverAI/Behemoth-123B-v2b-GGUF:Q4_K_M
- Lemonade
How to use BeaverAI/Behemoth-123B-v2b-GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull BeaverAI/Behemoth-123B-v2b-GGUF:Q4_K_M
Run and chat with the model
lemonade run user.Behemoth-123B-v2b-GGUF-Q4_K_M
List all available models
lemonade list
- Atomic Chat
Upload folder using huggingface_hub
Browse files- .gitattributes +18 -0
- Behemoth-123B-v2b-BF16-00001-of-00006.gguf +3 -0
- Behemoth-123B-v2b-BF16-00002-of-00006.gguf +3 -0
- Behemoth-123B-v2b-BF16-00003-of-00006.gguf +3 -0
- Behemoth-123B-v2b-BF16-00004-of-00006.gguf +3 -0
- Behemoth-123B-v2b-BF16-00005-of-00006.gguf +3 -0
- Behemoth-123B-v2b-BF16-00006-of-00006.gguf +3 -0
- Behemoth-123B-v2b-Q2_K-00001-of-00002.gguf +3 -0
- Behemoth-123B-v2b-Q2_K-00002-of-00002.gguf +3 -0
- Behemoth-123B-v2b-Q3_K_M-00001-of-00002.gguf +3 -0
- Behemoth-123B-v2b-Q3_K_M-00002-of-00002.gguf +3 -0
- Behemoth-123B-v2b-Q5_K_M-00001-of-00002.gguf +3 -0
- Behemoth-123B-v2b-Q5_K_M-00002-of-00002.gguf +3 -0
- Behemoth-123B-v2b-Q6_K-00001-of-00003.gguf +3 -0
- Behemoth-123B-v2b-Q6_K-00002-of-00003.gguf +3 -0
- Behemoth-123B-v2b-Q6_K-00003-of-00003.gguf +3 -0
- Behemoth-123B-v2b-Q8_0-00001-of-00003.gguf +3 -0
- Behemoth-123B-v2b-Q8_0-00002-of-00003.gguf +3 -0
- Behemoth-123B-v2b-Q8_0-00003-of-00003.gguf +3 -0
.gitattributes
CHANGED
|
@@ -35,3 +35,21 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
| 36 |
Behemoth-123B-v2b-Q4_K_M-00001-of-00002.gguf filter=lfs diff=lfs merge=lfs -text
|
| 37 |
Behemoth-123B-v2b-Q4_K_M-00002-of-00002.gguf filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
| 36 |
Behemoth-123B-v2b-Q4_K_M-00001-of-00002.gguf filter=lfs diff=lfs merge=lfs -text
|
| 37 |
Behemoth-123B-v2b-Q4_K_M-00002-of-00002.gguf filter=lfs diff=lfs merge=lfs -text
|
| 38 |
+
Behemoth-123B-v2b-BF16-00001-of-00006.gguf filter=lfs diff=lfs merge=lfs -text
|
| 39 |
+
Behemoth-123B-v2b-BF16-00002-of-00006.gguf filter=lfs diff=lfs merge=lfs -text
|
| 40 |
+
Behemoth-123B-v2b-BF16-00003-of-00006.gguf filter=lfs diff=lfs merge=lfs -text
|
| 41 |
+
Behemoth-123B-v2b-BF16-00004-of-00006.gguf filter=lfs diff=lfs merge=lfs -text
|
| 42 |
+
Behemoth-123B-v2b-BF16-00005-of-00006.gguf filter=lfs diff=lfs merge=lfs -text
|
| 43 |
+
Behemoth-123B-v2b-BF16-00006-of-00006.gguf filter=lfs diff=lfs merge=lfs -text
|
| 44 |
+
Behemoth-123B-v2b-Q2_K-00001-of-00002.gguf filter=lfs diff=lfs merge=lfs -text
|
| 45 |
+
Behemoth-123B-v2b-Q2_K-00002-of-00002.gguf filter=lfs diff=lfs merge=lfs -text
|
| 46 |
+
Behemoth-123B-v2b-Q3_K_M-00001-of-00002.gguf filter=lfs diff=lfs merge=lfs -text
|
| 47 |
+
Behemoth-123B-v2b-Q3_K_M-00002-of-00002.gguf filter=lfs diff=lfs merge=lfs -text
|
| 48 |
+
Behemoth-123B-v2b-Q5_K_M-00001-of-00002.gguf filter=lfs diff=lfs merge=lfs -text
|
| 49 |
+
Behemoth-123B-v2b-Q5_K_M-00002-of-00002.gguf filter=lfs diff=lfs merge=lfs -text
|
| 50 |
+
Behemoth-123B-v2b-Q6_K-00001-of-00003.gguf filter=lfs diff=lfs merge=lfs -text
|
| 51 |
+
Behemoth-123B-v2b-Q6_K-00002-of-00003.gguf filter=lfs diff=lfs merge=lfs -text
|
| 52 |
+
Behemoth-123B-v2b-Q6_K-00003-of-00003.gguf filter=lfs diff=lfs merge=lfs -text
|
| 53 |
+
Behemoth-123B-v2b-Q8_0-00001-of-00003.gguf filter=lfs diff=lfs merge=lfs -text
|
| 54 |
+
Behemoth-123B-v2b-Q8_0-00002-of-00003.gguf filter=lfs diff=lfs merge=lfs -text
|
| 55 |
+
Behemoth-123B-v2b-Q8_0-00003-of-00003.gguf filter=lfs diff=lfs merge=lfs -text
|
Behemoth-123B-v2b-BF16-00001-of-00006.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:c37f45295dcd5776f97ab79e1dc709a5e938e4b3832f6207412e37f4ec2eb516
|
| 3 |
+
size 44394786208
|
Behemoth-123B-v2b-BF16-00002-of-00006.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:dbce730c74eb0def9a4b7de6d380ea2285f9bc19862399c504d534193098b1ad
|
| 3 |
+
size 44998173376
|
Behemoth-123B-v2b-BF16-00003-of-00006.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:5fcaf3b379b88a26fa0d02660d00926fc00c24d3bfd09983ea3b8bc000b46275
|
| 3 |
+
size 44998025760
|
Behemoth-123B-v2b-BF16-00004-of-00006.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:8d92418360a484b5f40c178a57e36f486f1c2750daa005511e509692bccdcf50
|
| 3 |
+
size 44947792704
|
Behemoth-123B-v2b-BF16-00005-of-00006.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:982088903eafc4c199dcb61c4c6cc8febdcf344f9d96a028862e92f43bf8641e
|
| 3 |
+
size 44998074976
|
Behemoth-123B-v2b-BF16-00006-of-00006.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:923a965f886403bcd01f6a8bdc6673490b81f4f0c114fd28004c127deea72d3e
|
| 3 |
+
size 20888424448
|
Behemoth-123B-v2b-Q2_K-00001-of-00002.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:a818a95bd5c7d07d79bdb530c6517699b54edca390fe6dac5043b3d71f23ea62
|
| 3 |
+
size 44964988128
|
Behemoth-123B-v2b-Q2_K-00002-of-00002.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:3ccf6a43f0acd6b3836069119a7336f31bc266a72499004f007777902c0ed1b4
|
| 3 |
+
size 231309664
|
Behemoth-123B-v2b-Q3_K_M-00001-of-00002.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:3056ecac32b20fbadbe0bf46e5cde123a6d0e1677dd8e3a91ef7d6374b84c101
|
| 3 |
+
size 44841998720
|
Behemoth-123B-v2b-Q3_K_M-00002-of-00002.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:a2262a7a35a7a3f1fbd9fc7a50d169419d5216f1b7de3c5fca607c930cabf851
|
| 3 |
+
size 14260776128
|
Behemoth-123B-v2b-Q5_K_M-00001-of-00002.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:8732524219473273b43067f8765a959a908c52ee7cda33671afd1e43b7551f66
|
| 3 |
+
size 44878851232
|
Behemoth-123B-v2b-Q5_K_M-00002-of-00002.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:891430d11422774bd53e15cdf15d420c616d22f2105288edb1303086d622ffe6
|
| 3 |
+
size 41609451936
|
Behemoth-123B-v2b-Q6_K-00001-of-00003.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:fa93f6edc536718529e98b6c4405b5274e47f20fe99c84fe06434e26f5a7d1eb
|
| 3 |
+
size 44904996576
|
Behemoth-123B-v2b-Q6_K-00002-of-00003.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:c1f10bd576687faf477f3917bf5bac9dfe930a157e82f5d8ea5231fb27960368
|
| 3 |
+
size 44842275744
|
Behemoth-123B-v2b-Q6_K-00003-of-00003.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:bf9b02eaad2ca53fac1cf52c232d4525a44af045362d68fa8ef459b31883a44c
|
| 3 |
+
size 10839004224
|
Behemoth-123B-v2b-Q8_0-00001-of-00003.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:497743ddcda4c0becce115fa9547a322f4ee491bfb123864f601715990559991
|
| 3 |
+
size 44924750816
|
Behemoth-123B-v2b-Q8_0-00002-of-00003.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:13dc9123b388ddd550ffb74faaba45ae6096a1ba39beaf837138355cf7d485b5
|
| 3 |
+
size 44883902592
|
Behemoth-123B-v2b-Q8_0-00003-of-00003.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:4f5b2aff78ce665214908085ca9cd98fc930f543214b401fc5f81b5940917448
|
| 3 |
+
size 40471722592
|