# Issues when trying to build llama.cpp **URL:** \ **Category:** Beginners\ **Created:** [April 2, 2025, 7:37pm UTC](https://discuss.huggingface.co/t/issues-when-trying-to-build-llama-cpp/148603 "2025-04-02T19:37:46Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![jonACE](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/jonace/32/44664_2.png) [@jonACE](https://discuss.huggingface.co/u/jonACE)\ **Post date:** [April 2, 2025, 7:37pm UTC](https://discuss.huggingface.co/t/issues-when-trying-to-build-llama-cpp/148603/1 "2025-04-02T19:37:46Z")
I’m doing some training and wanted to save a .GGUF for OLLAMA but failed with the following errors: make: Entering directory ‘/home/user/app/llama.cpp’ make: Leaving directory ‘/home/user/app/llama.cpp’ Makefile:2: \*\*\* The Makefile build is deprecated. Use the CMake build instead. For more details, see [llama.cpp/docs/build.md at master · ggml-org/llama.cpp · GitHub](https://github.com/ggml-org/llama.cpp/blob/master/docs/build.md). Stop. – The C compiler identification is GNU 11.4.0 – The CXX compiler identification is GNU 11.4.0 – Detecting C compiler ABI info – Detecting C compiler ABI info - done – Check for working C compiler: /usr/bin/cc - skipped – Detecting C compile features – Detecting C compile features - done – Detecting CXX compiler ABI info – Detecting CXX compiler ABI info - done – Check for working CXX compiler: /usr/bin/c++ - skipped – Detecting CXX compile features – Detecting CXX compile features - done – Found Git: /usr/bin/git (found version “2.34.1”) – Looking for pthread.h – Looking for pthread.h - found – Performing Test CMAKE\_HAVE\_LIBC\_PTHREAD – Performing Test CMAKE\_HAVE\_LIBC\_PTHREAD - Success – Found Threads: TRUE – Warning: ccache not found - consider installing it for faster compilation or disable this warning with GGML\_CCACHE=OFF – CMAKE\_SYSTEM\_PROCESSOR: x86\_64 – Including CPU backend – Found OpenMP\_C: -fopenmp (found version “4.5”) – Found OpenMP\_CXX: -fopenmp (found version “4.5”) – Found OpenMP: TRUE (found version “4.5”) – x86 detected – Adding CPU backend variant ggml-cpu: -march=native CMake Error at /usr/share/cmake-3.22/Modules/FindPackageHandleStandardArgs.cmake:230 (message): Could NOT find CURL (missing: CURL\_LIBRARY CURL\_INCLUDE\_DIR) Call Stack (most recent call first): /usr/share/cmake-3.22/Modules/FindPackageHandleStandardArgs.cmake:594 (\_FPHSA\_FAILURE\_MESSAGE) /usr/share/cmake-3.22/Modules/FindCURL.cmake:181 (find\_package\_handle\_standard\_args) common/CMakeLists.txt:88 (find\_package) – Configuring incomplete, errors occurred! See also “/home/user/app/llama.cpp/build/CMakeFiles/CMakeOutput.log”. ────────────────────────── Traceback (most recent call last) ─────────────────────────── /home/user/.pyenv/versions/3.10.16/lib/python3.10/site-packages/streamlit/runtime/sc riptrunner/exec\_code.py:121 in exec\_func\_with\_error\_handling /home/user/.pyenv/versions/3.10.16/lib/python3.10/site-packages/streamlit/runtime/sc riptrunner/script\_runner.py:640 in code\_to\_exec /home/user/app/app.py:107 in ``` 104 tokenizer.push_to_hub("jonACE/llama-2-7b-chat_fine_tuned", token=hf_token) 105 106 # save GGUF versions ``` ❱ 107 model.save\_pretrained\_gguf(“./llama-2-7b-chat\_fine\_tuned”, tokenizer,) 108 model.push\_to\_hub\_gguf(“jonACE/llama-2-7b-chat\_fine\_tuned”, tokenizer) 109 110 model.save\_pretrained\_gguf(“./llama-2-7b-chat\_fine\_tuned”, tokenizer, quantiza /home/user/.pyenv/versions/3.10.16/lib/python3.10/site-packages/unsloth/save.py:1805 in unsloth\_save\_pretrained\_gguf ``` 1802 │ │ │ git_clone = install_llama_cpp_clone_non_blocking() 1803 │ │ │ python_install = install_python_non_blocking(["gguf", "protobuf"] 1804 │ │ │ git_clone.wait() ``` ❱ 1805 │ │ │ makefile = install\_llama\_cpp\_make\_non\_blocking() 1806 │ │ │ new\_save\_directory, old\_username = unsloth\_save\_model(\*\*arguments 1807 │ │ │ python\_install.wait() 1808 │ │ pass /home/user/.pyenv/versions/3.10.16/lib/python3.10/site-packages/unsloth/save.py:785 in install\_llama\_cpp\_make\_non\_blocking ``` 782 │ │ n_jobs = max(int(psutil.cpu_count()), 1) # Use less CPUs since 1.5x f 783 │ │ check = os.system("cmake llama.cpp -B llama.cpp/build -DBUILD_SHARED_ 784 │ │ if check != 0: ``` ❱ 785 │ │ │ raise RuntimeError(f"\*\*\* Unsloth: Failed compiling llama.cpp usin 786 │ │ pass 787 │ │ # f"cmake --build llama.cpp/build --config Release -j{psutil.cpu\_coun 788 │ │ full\_command = [ ──────────────────────────────────────────────────────────────────────────────────────── RuntimeError: \*\*\* Unsloth: Failed compiling llama.cpp using os.system(…) with error 256. Please report this ASAP! Stopping… Is this a known issue? How can I fix this? ---
**Author:** ![John6666](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/john6666/32/27664_2.png) [@John6666](https://discuss.huggingface.co/u/John6666)\ **Post date:** [April 3, 2025, 1:59am UTC](https://discuss.huggingface.co/t/issues-when-trying-to-build-llama-cpp/148603/2 "2025-04-03T01:59:16Z")
It seems that the command for building Lllama.cpp has changed. Please refer to the following github description. > > > Please give instructions. There is nothing in README but it says that it support…s it > > **[GitHub - abetlen/llama-cpp-python: Python bindings for llama.cpp](https://github.com/abetlen/llama-cpp-python)** > > Python bindings for llama.cpp ---
**Author:** ![jonACE](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/jonace/32/44664_2.png) [@jonACE](https://discuss.huggingface.co/u/jonACE)\ **Post date:** [April 3, 2025, 5:13am UTC](https://discuss.huggingface.co/t/issues-when-trying-to-build-llama-cpp/148603/3 "2025-04-03T05:13:41Z")
Hi, Thanks for the reply. I’m not sure I’m able to control the building of llama.cpp as I’m running a python script for the training and after the training, I did the saving and push to HF the built models: ```auto ..... perform_training() model.save_pretrained("./llama-2-7b-chat_fine_tuned") tokenizer.save_pretrained("./llama-2-7b-chat_fine_tuned") model.push_to_hub("jonACE/llama-2-7b-chat_fine_tuned", token=hf_token) tokenizer.push_to_hub("jonACE/llama-2-7b-chat_fine_tuned", token=hf_token) # save GGUF versions model.save_pretrained_gguf("./llama-2-7b-chat_fine_tuned", tokenizer,) model.push_to_hub_gguf("jonACE/llama-2-7b-chat_fine_tuned", tokenizer, token=hf_token) model.save_pretrained_gguf("./llama-2-7b-chat_fine_tuned", tokenizer, quantization_method = "f16") model.push_to_hub_gguf("jonACE/llama-2-7b-chat_fine_tuned", tokenizer, quantization_method = "f16", token=hf_token) model.save_pretrained_gguf("./llama-2-7b-chat_fine_tuned", tokenizer, quantization_method = "q4_k_m") model.push_to_hub_gguf("jonACE/llama-2-7b-chat_fine_tuned", tokenizer, quantization_method = "q4_k_m", token=hf_token) ``` The model object was a result of unsloth.FastLanguageModel: ```auto model, tokenizer = FastLanguageModel.from_pretrained( model_name=model_name, max_seq_length=2048 ) model = FastLanguageModel.get_peft_model(model) ``` Could it be that the python ‘unsloth’ component is still based on older llama.cpp versions? ---
**Author:** ![John6666](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/john6666/32/27664_2.png) [@John6666](https://discuss.huggingface.co/u/John6666)\ **Post date:** [April 3, 2025, 5:28am UTC](https://discuss.huggingface.co/t/issues-when-trying-to-build-llama-cpp/148603/4 "2025-04-03T05:28:12Z")
There is a new issue on github about 256 errors, so it may be a problem with the latest version of the library. I think you can get around it by saving it with safetensors and converting it to GGUF with the script attached to Llama.cpp, but it would be better if it could be fixed more easily… I wonder if it can be avoided by installing cmake or something. ```py pip install cmake ``` ### Issue > > > I tried recreating a notebook from the Documentation. I am using Ubuntu Server a…nd got at the end the following error: > > \`\`\` > make: Entering directory '/root/unsloth/Reasoner/llama.cpp' > Makefile:2: \*\*\* The Makefile build is deprecated. Use the CMake build instead. For more details, see https://github.com/ggml-org/llama.cpp/blob/master/docs/build.md. Stop. > make: Leaving directory '/root/unsloth/Reasoner/llama.cpp' > \-- The C compiler identification is GNU 11.4.0 > \-- The CXX compiler identification is GNU 11.4.0 > \-- Detecting C compiler ABI info > \-- Detecting C compiler ABI info - done > \-- Check for working C compiler: /bin/cc - skipped > \-- Detecting C compile features > \-- Detecting C compile features - done > \-- Detecting CXX compiler ABI info > \-- Detecting CXX compiler ABI info - done > \-- Check for working CXX compiler: /bin/c++ - skipped > \-- Detecting CXX compile features > \-- Detecting CXX compile features - done > \-- Found Git: /bin/git (found version "2.34.1") > \-- Looking for pthread.h > \-- Looking for pthread.h - found > \-- Performing Test CMAKE\_HAVE\_LIBC\_PTHREAD > \-- Performing Test CMAKE\_HAVE\_LIBC\_PTHREAD - Success > \-- Found Threads: TRUE > \-- Warning: ccache not found - consider installing it for faster compilation or disable this warning with GGML\_CCACHE=OFF > \-- CMAKE\_SYSTEM\_PROCESSOR: x86\_64 > \-- Including CPU backend > \-- Found OpenMP\_C: -fopenmp (found version "4.5") > \-- Found OpenMP\_CXX: -fopenmp (found version "4.5") > \-- Found OpenMP: TRUE (found version "4.5") > \-- x86 detected > \-- Adding CPU backend variant ggml-cpu: -march=native > CMake Error at /usr/share/cmake-3.22/Modules/FindPackageHandleStandardArgs.cmake:230 (message): > Could NOT find CURL (missing: CURL\_LIBRARY CURL\_INCLUDE\_DIR) > Call Stack (most recent call first): > /usr/share/cmake-3.22/Modules/FindPackageHandleStandardArgs.cmake:594 (\_FPHSA\_FAILURE\_MESSAGE) > /usr/share/cmake-3.22/Modules/FindCURL.cmake:181 (find\_package\_handle\_standard\_args) > common/CMakeLists.txt:88 (find\_package) > > > \-- Configuring incomplete, errors occurred! > See also "/root/unsloth/Reasoner/llama.cpp/build/CMakeFiles/CMakeOutput.log". > Unsloth: Saving tokenizer... Done. > Done. > Traceback (most recent call last): > File "/root/unsloth/Reasoner/train.py", line 290, in \ > main() > File "/root/unsloth/Reasoner/train.py", line 286, in main > model.save\_pretrained\_gguf("Mistral-Small-Thinker\_4", tokenizer, quantization\_method="q4\_k\_m") > File "/root/miniconda3/envs/unsloth\_env/lib/python3.11/site-packages/unsloth/save.py", line 1805, in unsloth\_save\_pretrained\_gguf > makefile = install\_llama\_cpp\_make\_non\_blocking() > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > File "/root/miniconda3/envs/unsloth\_env/lib/python3.11/site-packages/unsloth/save.py", line 785, in install\_llama\_cpp\_make\_non\_blocking > raise RuntimeError(f"\*\*\* Unsloth: Failed compiling llama.cpp using os.system(...) with error {check}. Please report this ASAP!") > RuntimeError: \*\*\* Unsloth: Failed compiling llama.cpp using os.system(...) with error 256. Please report this ASAP! > \`\`\` ### Workaround > > > I use the model: https://huggingface.co/taide/TAIDE-LX-7B-Chat to fine-tune, bu…t always got the error. training is OK, but model.save\_pretrained\_gguf failed. > > ==((====))== Unsloth: Fast Llama patching release 2024.4 > \\\\ /| GPU: NVIDIA GeForce RTX 3090. Max memory: 23.691 GB. Platform = Linux. > O^O/ \\\_/ \\ Pytorch: 2.2.2+cu121. CUDA = 8.6. CUDA Toolkit = 12.1. > \\ / Bfloat16 = TRUE. Xformers = 0.0.25.post1. FA = False. > "-\_\_\_\_-" Free Apache license: http://github.com/unslothai/unsloth > Loading checkpoint shards: 100%|████████████████████████████████████████████████████████| 3/3 \[00:03\<00:00, 1.02s/it\] > You set \`add\_prefix\_space\`. The tokenizer needs to be converted from the slow tokenizers > Unsloth 2024.4 patched 32 layers with 32 QKV layers, 32 O layers and 32 MLP layers. > Setting \`pad\_token\_id\` to \`eos\_token\_id\`:2 for open-end generation. > \['\Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.\\n\\n### Instruction:\\n\\n\\n### Input:\\n上上禮拜持續出現頭痛、噁心、頭暈的症狀，有時睡一下起來還是沒有緩解，大概都痛在太陽穴上面一點，有時痛在頭腦勺（較少），躺著起來頭暈頻率越來越高（本身有貧血，但近期只要姿勢一轉換就會頭暈眼前接近黑色），容易疲累，想問一下這些症狀有需要到醫院去檢查嗎？\\n\\n### Response:\\n\\n您好：\\n根據您的描述，您可能有以下幾種可能的原因：\\n1. 貧血：貧血是常見的問題，若沒有定期檢查，可能會導致頭暈、頭痛、疲累等症狀。\\n2. 內耳問題：內耳有平衡器官，若內耳有問題，可能會導致頭暈、頭痛、噁心等症狀。\\n3. 其他疾病：如甲狀腺疾病、心臟疾病、糖尿病、高血壓等，都可能會引起頭暈、頭痛、噁心等症狀。\\n建議您前往醫院，讓醫師為您做詳細的檢查，以確定病因，並接受適當的治療。\\n祝健康！ \'\] > Unsloth: Merging 4bit and LoRA weights to 16bit... > Unsloth: Will use up to 46.9 out of 62.57 RAM for saving. > 100%|█████████████████████████████████████████████████████████████████████████████████| 32/32 \[00:00\<00:00, 90.79it/s\] > Unsloth: Saving tokenizer... Done. > Unsloth: Saving model... This might take 5 minutes for Llama-7b... > Done. > Unsloth: Converting llama model. Can use fast conversion = True. > ==((====))== Unsloth: Conversion from QLoRA to GGUF information > \\\\ /| \[0\] Installing llama.cpp will take 3 minutes. > O^O/ \\\_/ \\ \[1\] Converting HF to GUUF 16bits will take 3 minutes. > \\ / \[2\] Converting GGUF 16bits to q4\_k\_m will take 20 minutes. > "-\_\_\_\_-" In total, you will have to wait around 26 minutes. > > Unsloth: \[0\] Installing llama.cpp. This will take 3 minutes... > Unsloth: \[1\] Converting model at ch\_taide\_medicine.gguf into f16 GGUF format. > The output location will be ./ch\_taide\_medicine.gguf-unsloth.F16.gguf > This will take 3 minutes... > Loading model file ch\_taide\_medicine.gguf/model-00001-of-00003.safetensors > Loading model file ch\_taide\_medicine.gguf/model-00001-of-00003.safetensors > Loading model file ch\_taide\_medicine.gguf/model-00002-of-00003.safetensors > Loading model file ch\_taide\_medicine.gguf/model-00003-of-00003.safetensors > params = Params(n\_vocab=56064, n\_embd=4096, n\_layer=32, n\_ctx=4096, n\_ff=11008, n\_head=32, n\_head\_kv=32, n\_experts=None, n\_experts\_used=None, f\_norm\_eps=1e-05, rope\_scaling\_type=None, f\_rope\_freq\_base=10000.0, f\_rope\_scale=None, n\_orig\_ctx=None, rope\_finetuned=None, ftype=\, path\_model=PosixPath('ch\_taide\_medicine.gguf')) > Loaded vocab file PosixPath('ch\_taide\_medicine.gguf/tokenizer.json'), type 'hfft' > Vocab info: \ > Special vocab info: \ > Permuting layer 0 > Permuting layer 1 > Permuting layer 2 > Permuting layer 3 > Permuting layer 4 > Permuting layer 5 > Permuting layer 6 > Permuting layer 7 > Permuting layer 8 > Permuting layer 9 > Permuting layer 10 > Permuting layer 11 > Permuting layer 12 > Permuting layer 13 > Permuting layer 14 > Permuting layer 15 > Permuting layer 16 > Permuting layer 17 > Permuting layer 18 > Permuting layer 19 > Permuting layer 20 > Permuting layer 21 > Permuting layer 22 > Permuting layer 23 > Permuting layer 24 > Permuting layer 25 > Permuting layer 26 > Permuting layer 27 > Permuting layer 28 > Permuting layer 29 > Permuting layer 30 > Permuting layer 31 > model.embed\_tokens.weight -\> token\_embd.weight | BF16 | \[56064, 4096\] > model.layers.0.input\_layernorm.weight -\> blk.0.attn\_norm.weight | BF16 | \[4096\] > model.layers.0.mlp.down\_proj.weight -\> blk.0.ffn\_down.weight | BF16 | \[4096, 11008\] > model.layers.0.mlp.gate\_proj.weight -\> blk.0.ffn\_gate.weight | BF16 | \[11008, 4096\] > model.layers.0.mlp.up\_proj.weight -\> blk.0.ffn\_up.weight | BF16 | \[11008, 4096\] > model.layers.0.post\_attention\_layernorm.weight -\> blk.0.ffn\_norm.weight | BF16 | \[4096\] > model.layers.0.self\_attn.k\_proj.weight -\> blk.0.attn\_k.weight | BF16 | \[4096, 4096\] > model.layers.0.self\_attn.o\_proj.weight -\> blk.0.attn\_output.weight | BF16 | \[4096, 4096\] > model.layers.0.self\_attn.q\_proj.weight -\> blk.0.attn\_q.weight | BF16 | \[4096, 4096\] > model.layers.0.self\_attn.v\_proj.weight -\> blk.0.attn\_v.weight | BF16 | \[4096, 4096\] > model.layers.1.input\_layernorm.weight -\> blk.1.attn\_norm.weight | BF16 | \[4096\] > model.layers.1.mlp.down\_proj.weight -\> blk.1.ffn\_down.weight | BF16 | \[4096, 11008\] > model.layers.1.mlp.gate\_proj.weight -\> blk.1.ffn\_gate.weight | BF16 | \[11008, 4096\] > model.layers.1.mlp.up\_proj.weight -\> blk.1.ffn\_up.weight | BF16 | \[11008, 4096\] > model.layers.1.post\_attention\_layernorm.weight -\> blk.1.ffn\_norm.weight | BF16 | \[4096\] > model.layers.1.self\_attn.k\_proj.weight -\> blk.1.attn\_k.weight | BF16 | \[4096, 4096\] > model.layers.1.self\_attn.o\_proj.weight -\> blk.1.attn\_output.weight | BF16 | \[4096, 4096\] > model.layers.1.self\_attn.q\_proj.weight -\> blk.1.attn\_q.weight | BF16 | \[4096, 4096\] > model.layers.1.self\_attn.v\_proj.weight -\> blk.1.attn\_v.weight | BF16 | \[4096, 4096\] > model.layers.10.input\_layernorm.weight -\> blk.10.attn\_norm.weight | BF16 | \[4096\] > model.layers.10.mlp.down\_proj.weight -\> blk.10.ffn\_down.weight | BF16 | \[4096, 11008\] > model.layers.10.mlp.gate\_proj.weight -\> blk.10.ffn\_gate.weight | BF16 | \[11008, 4096\] > model.layers.10.mlp.up\_proj.weight -\> blk.10.ffn\_up.weight | BF16 | \[11008, 4096\] > model.layers.10.post\_attention\_layernorm.weight -\> blk.10.ffn\_norm.weight | BF16 | \[4096\] > model.layers.10.self\_attn.k\_proj.weight -\> blk.10.attn\_k.weight | BF16 | \[4096, 4096\] > model.layers.10.self\_attn.o\_proj.weight -\> blk.10.attn\_output.weight | BF16 | \[4096, 4096\] > model.layers.10.self\_attn.q\_proj.weight -\> blk.10.attn\_q.weight | BF16 | \[4096, 4096\] > model.layers.10.self\_attn.v\_proj.weight -\> blk.10.attn\_v.weight | BF16 | \[4096, 4096\] > model.layers.11.self\_attn.k\_proj.weight -\> blk.11.attn\_k.weight | BF16 | \[4096, 4096\] > model.layers.11.self\_attn.q\_proj.weight -\> blk.11.attn\_q.weight | BF16 | \[4096, 4096\] > model.layers.2.input\_layernorm.weight -\> blk.2.attn\_norm.weight | BF16 | \[4096\] > model.layers.2.mlp.down\_proj.weight -\> blk.2.ffn\_down.weight | BF16 | \[4096, 11008\] > model.layers.2.mlp.gate\_proj.weight -\> blk.2.ffn\_gate.weight | BF16 | \[11008, 4096\] > model.layers.2.mlp.up\_proj.weight -\> blk.2.ffn\_up.weight | BF16 | \[11008, 4096\] > model.layers.2.post\_attention\_layernorm.weight -\> blk.2.ffn\_norm.weight | BF16 | \[4096\] > model.layers.2.self\_attn.k\_proj.weight -\> blk.2.attn\_k.weight | BF16 | \[4096, 4096\] > model.layers.2.self\_attn.o\_proj.weight -\> blk.2.attn\_output.weight | BF16 | \[4096, 4096\] > model.layers.2.self\_attn.q\_proj.weight -\> blk.2.attn\_q.weight | BF16 | \[4096, 4096\] > model.layers.2.self\_attn.v\_proj.weight -\> blk.2.attn\_v.weight | BF16 | \[4096, 4096\] > model.layers.3.input\_layernorm.weight -\> blk.3.attn\_norm.weight | BF16 | \[4096\] > model.layers.3.mlp.down\_proj.weight -\> blk.3.ffn\_down.weight | BF16 | \[4096, 11008\] > model.layers.3.mlp.gate\_proj.weight -\> blk.3.ffn\_gate.weight | BF16 | \[11008, 4096\] > model.layers.3.mlp.up\_proj.weight -\> blk.3.ffn\_up.weight | BF16 | \[11008, 4096\] > model.layers.3.post\_attention\_layernorm.weight -\> blk.3.ffn\_norm.weight | BF16 | \[4096\] > model.layers.3.self\_attn.k\_proj.weight -\> blk.3.attn\_k.weight | BF16 | \[4096, 4096\] > model.layers.3.self\_attn.o\_proj.weight -\> blk.3.attn\_output.weight | BF16 | \[4096, 4096\] > model.layers.3.self\_attn.q\_proj.weight -\> blk.3.attn\_q.weight | BF16 | \[4096, 4096\] > model.layers.3.self\_attn.v\_proj.weight -\> blk.3.attn\_v.weight | BF16 | \[4096, 4096\] > model.layers.4.input\_layernorm.weight -\> blk.4.attn\_norm.weight | BF16 | \[4096\] > model.layers.4.mlp.down\_proj.weight -\> blk.4.ffn\_down.weight | BF16 | \[4096, 11008\] > model.layers.4.mlp.gate\_proj.weight -\> blk.4.ffn\_gate.weight | BF16 | \[11008, 4096\] > model.layers.4.mlp.up\_proj.weight -\> blk.4.ffn\_up.weight | BF16 | \[11008, 4096\] > model.layers.4.post\_attention\_layernorm.weight -\> blk.4.ffn\_norm.weight | BF16 | \[4096\] > model.layers.4.self\_attn.k\_proj.weight -\> blk.4.attn\_k.weight | BF16 | \[4096, 4096\] > model.layers.4.self\_attn.o\_proj.weight -\> blk.4.attn\_output.weight | BF16 | \[4096, 4096\] > model.layers.4.self\_attn.q\_proj.weight -\> blk.4.attn\_q.weight | BF16 | \[4096, 4096\] > model.layers.4.self\_attn.v\_proj.weight -\> blk.4.attn\_v.weight | BF16 | \[4096, 4096\] > model.layers.5.input\_layernorm.weight -\> blk.5.attn\_norm.weight | BF16 | \[4096\] > model.layers.5.mlp.down\_proj.weight -\> blk.5.ffn\_down.weight | BF16 | \[4096, 11008\] > model.layers.5.mlp.gate\_proj.weight -\> blk.5.ffn\_gate.weight | BF16 | \[11008, 4096\] > model.layers.5.mlp.up\_proj.weight -\> blk.5.ffn\_up.weight | BF16 | \[11008, 4096\] > model.layers.5.post\_attention\_layernorm.weight -\> blk.5.ffn\_norm.weight | BF16 | \[4096\] > model.layers.5.self\_attn.k\_proj.weight -\> blk.5.attn\_k.weight | BF16 | \[4096, 4096\] > model.layers.5.self\_attn.o\_proj.weight -\> blk.5.attn\_output.weight | BF16 | \[4096, 4096\] > model.layers.5.self\_attn.q\_proj.weight -\> blk.5.attn\_q.weight | BF16 | \[4096, 4096\] > model.layers.5.self\_attn.v\_proj.weight -\> blk.5.attn\_v.weight | BF16 | \[4096, 4096\] > model.layers.6.input\_layernorm.weight -\> blk.6.attn\_norm.weight | BF16 | \[4096\] > model.layers.6.mlp.down\_proj.weight -\> blk.6.ffn\_down.weight | BF16 | \[4096, 11008\] > model.layers.6.mlp.gate\_proj.weight -\> blk.6.ffn\_gate.weight | BF16 | \[11008, 4096\] > model.layers.6.mlp.up\_proj.weight -\> blk.6.ffn\_up.weight | BF16 | \[11008, 4096\] > model.layers.6.post\_attention\_layernorm.weight -\> blk.6.ffn\_norm.weight | BF16 | \[4096\] > model.layers.6.self\_attn.k\_proj.weight -\> blk.6.attn\_k.weight | BF16 | \[4096, 4096\] > model.layers.6.self\_attn.o\_proj.weight -\> blk.6.attn\_output.weight | BF16 | \[4096, 4096\] > model.layers.6.self\_attn.q\_proj.weight -\> blk.6.attn\_q.weight | BF16 | \[4096, 4096\] > model.layers.6.self\_attn.v\_proj.weight -\> blk.6.attn\_v.weight | BF16 | \[4096, 4096\] > model.layers.7.input\_layernorm.weight -\> blk.7.attn\_norm.weight | BF16 | \[4096\] > model.layers.7.mlp.down\_proj.weight -\> blk.7.ffn\_down.weight | BF16 | \[4096, 11008\] > model.layers.7.mlp.gate\_proj.weight -\> blk.7.ffn\_gate.weight | BF16 | \[11008, 4096\] > model.layers.7.mlp.up\_proj.weight -\> blk.7.ffn\_up.weight | BF16 | \[11008, 4096\] > model.layers.7.post\_attention\_layernorm.weight -\> blk.7.ffn\_norm.weight | BF16 | \[4096\] > model.layers.7.self\_attn.k\_proj.weight -\> blk.7.attn\_k.weight | BF16 | \[4096, 4096\] > model.layers.7.self\_attn.o\_proj.weight -\> blk.7.attn\_output.weight | BF16 | \[4096, 4096\] > model.layers.7.self\_attn.q\_proj.weight -\> blk.7.attn\_q.weight | BF16 | \[4096, 4096\] > model.layers.7.self\_attn.v\_proj.weight -\> blk.7.attn\_v.weight | BF16 | \[4096, 4096\] > model.layers.8.input\_layernorm.weight -\> blk.8.attn\_norm.weight | BF16 | \[4096\] > model.layers.8.mlp.down\_proj.weight -\> blk.8.ffn\_down.weight | BF16 | \[4096, 11008\] > model.layers.8.mlp.gate\_proj.weight -\> blk.8.ffn\_gate.weight | BF16 | \[11008, 4096\] > model.layers.8.mlp.up\_proj.weight -\> blk.8.ffn\_up.weight | BF16 | \[11008, 4096\] > model.layers.8.post\_attention\_layernorm.weight -\> blk.8.ffn\_norm.weight | BF16 | \[4096\] > model.layers.8.self\_attn.k\_proj.weight -\> blk.8.attn\_k.weight | BF16 | \[4096, 4096\] > model.layers.8.self\_attn.o\_proj.weight -\> blk.8.attn\_output.weight | BF16 | \[4096, 4096\] > model.layers.8.self\_attn.q\_proj.weight -\> blk.8.attn\_q.weight | BF16 | \[4096, 4096\] > model.layers.8.self\_attn.v\_proj.weight -\> blk.8.attn\_v.weight | BF16 | \[4096, 4096\] > model.layers.9.input\_layernorm.weight -\> blk.9.attn\_norm.weight | BF16 | \[4096\] > model.layers.9.mlp.down\_proj.weight -\> blk.9.ffn\_down.weight | BF16 | \[4096, 11008\] > model.layers.9.mlp.gate\_proj.weight -\> blk.9.ffn\_gate.weight | BF16 | \[11008, 4096\] > model.layers.9.mlp.up\_proj.weight -\> blk.9.ffn\_up.weight | BF16 | \[11008, 4096\] > model.layers.9.post\_attention\_layernorm.weight -\> blk.9.ffn\_norm.weight | BF16 | \[4096\] > model.layers.9.self\_attn.k\_proj.weight -\> blk.9.attn\_k.weight | BF16 | \[4096, 4096\] > model.layers.9.self\_attn.o\_proj.weight -\> blk.9.attn\_output.weight | BF16 | \[4096, 4096\] > model.layers.9.self\_attn.q\_proj.weight -\> blk.9.attn\_q.weight | BF16 | \[4096, 4096\] > model.layers.9.self\_attn.v\_proj.weight -\> blk.9.attn\_v.weight | BF16 | \[4096, 4096\] > model.layers.11.input\_layernorm.weight -\> blk.11.attn\_norm.weight | BF16 | \[4096\] > model.layers.11.mlp.down\_proj.weight -\> blk.11.ffn\_down.weight | BF16 | \[4096, 11008\] > model.layers.11.mlp.gate\_proj.weight -\> blk.11.ffn\_gate.weight | BF16 | \[11008, 4096\] > model.layers.11.mlp.up\_proj.weight -\> blk.11.ffn\_up.weight | BF16 | \[11008, 4096\] > model.layers.11.post\_attention\_layernorm.weight -\> blk.11.ffn\_norm.weight | BF16 | \[4096\] > model.layers.11.self\_attn.o\_proj.weight -\> blk.11.attn\_output.weight | BF16 | \[4096, 4096\] > model.layers.11.self\_attn.v\_proj.weight -\> blk.11.attn\_v.weight | BF16 | \[4096, 4096\] > model.layers.12.input\_layernorm.weight -\> blk.12.attn\_norm.weight | BF16 | \[4096\] > model.layers.12.mlp.down\_proj.weight -\> blk.12.ffn\_down.weight | BF16 | \[4096, 11008\] > model.layers.12.mlp.gate\_proj.weight -\> blk.12.ffn\_gate.weight | BF16 | \[11008, 4096\] > model.layers.12.mlp.up\_proj.weight -\> blk.12.ffn\_up.weight | BF16 | \[11008, 4096\] > model.layers.12.post\_attention\_layernorm.weight -\> blk.12.ffn\_norm.weight | BF16 | \[4096\] > model.layers.12.self\_attn.k\_proj.weight -\> blk.12.attn\_k.weight | BF16 | \[4096, 4096\] > model.layers.12.self\_attn.o\_proj.weight -\> blk.12.attn\_output.weight | BF16 | \[4096, 4096\] > model.layers.12.self\_attn.q\_proj.weight -\> blk.12.attn\_q.weight | BF16 | \[4096, 4096\] > model.layers.12.self\_attn.v\_proj.weight -\> blk.12.attn\_v.weight | BF16 | \[4096, 4096\] > model.layers.13.input\_layernorm.weight -\> blk.13.attn\_norm.weight | BF16 | \[4096\] > model.layers.13.mlp.down\_proj.weight -\> blk.13.ffn\_down.weight | BF16 | \[4096, 11008\] > model.layers.13.mlp.gate\_proj.weight -\> blk.13.ffn\_gate.weight | BF16 | \[11008, 4096\] > model.layers.13.mlp.up\_proj.weight -\> blk.13.ffn\_up.weight | BF16 | \[11008, 4096\] > model.layers.13.post\_attention\_layernorm.weight -\> blk.13.ffn\_norm.weight | BF16 | \[4096\] > model.layers.13.self\_attn.k\_proj.weight -\> blk.13.attn\_k.weight | BF16 | \[4096, 4096\] > model.layers.13.self\_attn.o\_proj.weight -\> blk.13.attn\_output.weight | BF16 | \[4096, 4096\] > model.layers.13.self\_attn.q\_proj.weight -\> blk.13.attn\_q.weight | BF16 | \[4096, 4096\] > model.layers.13.self\_attn.v\_proj.weight -\> blk.13.attn\_v.weight | BF16 | \[4096, 4096\] > model.layers.14.input\_layernorm.weight -\> blk.14.attn\_norm.weight | BF16 | \[4096\] > model.layers.14.mlp.down\_proj.weight -\> blk.14.ffn\_down.weight | BF16 | \[4096, 11008\] > model.layers.14.mlp.gate\_proj.weight -\> blk.14.ffn\_gate.weight | BF16 | \[11008, 4096\] > model.layers.14.mlp.up\_proj.weight -\> blk.14.ffn\_up.weight | BF16 | \[11008, 4096\] > model.layers.14.post\_attention\_layernorm.weight -\> blk.14.ffn\_norm.weight | BF16 | \[4096\] > model.layers.14.self\_attn.k\_proj.weight -\> blk.14.attn\_k.weight | BF16 | \[4096, 4096\] > model.layers.14.self\_attn.o\_proj.weight -\> blk.14.attn\_output.weight | BF16 | \[4096, 4096\] > model.layers.14.self\_attn.q\_proj.weight -\> blk.14.attn\_q.weight | BF16 | \[4096, 4096\] > model.layers.14.self\_attn.v\_proj.weight -\> blk.14.attn\_v.weight | BF16 | \[4096, 4096\] > model.layers.15.input\_layernorm.weight -\> blk.15.attn\_norm.weight | BF16 | \[4096\] > model.layers.15.mlp.down\_proj.weight -\> blk.15.ffn\_down.weight | BF16 | \[4096, 11008\] > model.layers.15.mlp.gate\_proj.weight -\> blk.15.ffn\_gate.weight | BF16 | \[11008, 4096\] > model.layers.15.mlp.up\_proj.weight -\> blk.15.ffn\_up.weight | BF16 | \[11008, 4096\] > model.layers.15.post\_attention\_layernorm.weight -\> blk.15.ffn\_norm.weight | BF16 | \[4096\] > model.layers.15.self\_attn.k\_proj.weight -\> blk.15.attn\_k.weight | BF16 | \[4096, 4096\] > model.layers.15.self\_attn.o\_proj.weight -\> blk.15.attn\_output.weight | BF16 | \[4096, 4096\] > model.layers.15.self\_attn.q\_proj.weight -\> blk.15.attn\_q.weight | BF16 | \[4096, 4096\] > model.layers.15.self\_attn.v\_proj.weight -\> blk.15.attn\_v.weight | BF16 | \[4096, 4096\] > model.layers.16.input\_layernorm.weight -\> blk.16.attn\_norm.weight | BF16 | \[4096\] > model.layers.16.mlp.down\_proj.weight -\> blk.16.ffn\_down.weight | BF16 | \[4096, 11008\] > model.layers.16.mlp.gate\_proj.weight -\> blk.16.ffn\_gate.weight | BF16 | \[11008, 4096\] > model.layers.16.mlp.up\_proj.weight -\> blk.16.ffn\_up.weight | BF16 | \[11008, 4096\] > model.layers.16.post\_attention\_layernorm.weight -\> blk.16.ffn\_norm.weight | BF16 | \[4096\] > model.layers.16.self\_attn.k\_proj.weight -\> blk.16.attn\_k.weight | BF16 | \[4096, 4096\] > model.layers.16.self\_attn.o\_proj.weight -\> blk.16.attn\_output.weight | BF16 | \[4096, 4096\] > model.layers.16.self\_attn.q\_proj.weight -\> blk.16.attn\_q.weight | BF16 | \[4096, 4096\] > model.layers.16.self\_attn.v\_proj.weight -\> blk.16.attn\_v.weight | BF16 | \[4096, 4096\] > model.layers.17.input\_layernorm.weight -\> blk.17.attn\_norm.weight | BF16 | \[4096\] > model.layers.17.mlp.down\_proj.weight -\> blk.17.ffn\_down.weight | BF16 | \[4096, 11008\] > model.layers.17.mlp.gate\_proj.weight -\> blk.17.ffn\_gate.weight | BF16 | \[11008, 4096\] > model.layers.17.mlp.up\_proj.weight -\> blk.17.ffn\_up.weight | BF16 | \[11008, 4096\] > model.layers.17.post\_attention\_layernorm.weight -\> blk.17.ffn\_norm.weight | BF16 | \[4096\] > model.layers.17.self\_attn.k\_proj.weight -\> blk.17.attn\_k.weight | BF16 | \[4096, 4096\] > model.layers.17.self\_attn.o\_proj.weight -\> blk.17.attn\_output.weight | BF16 | \[4096, 4096\] > model.layers.17.self\_attn.q\_proj.weight -\> blk.17.attn\_q.weight | BF16 | \[4096, 4096\] > model.layers.17.self\_attn.v\_proj.weight -\> blk.17.attn\_v.weight | BF16 | \[4096, 4096\] > model.layers.18.input\_layernorm.weight -\> blk.18.attn\_norm.weight | BF16 | \[4096\] > model.layers.18.mlp.down\_proj.weight -\> blk.18.ffn\_down.weight | BF16 | \[4096, 11008\] > model.layers.18.mlp.gate\_proj.weight -\> blk.18.ffn\_gate.weight | BF16 | \[11008, 4096\] > model.layers.18.mlp.up\_proj.weight -\> blk.18.ffn\_up.weight | BF16 | \[11008, 4096\] > model.layers.18.post\_attention\_layernorm.weight -\> blk.18.ffn\_norm.weight | BF16 | \[4096\] > model.layers.18.self\_attn.k\_proj.weight -\> blk.18.attn\_k.weight | BF16 | \[4096, 4096\] > model.layers.18.self\_attn.o\_proj.weight -\> blk.18.attn\_output.weight | BF16 | \[4096, 4096\] > model.layers.18.self\_attn.q\_proj.weight -\> blk.18.attn\_q.weight | BF16 | \[4096, 4096\] > model.layers.18.self\_attn.v\_proj.weight -\> blk.18.attn\_v.weight | BF16 | \[4096, 4096\] > model.layers.19.input\_layernorm.weight -\> blk.19.attn\_norm.weight | BF16 | \[4096\] > model.layers.19.mlp.down\_proj.weight -\> blk.19.ffn\_down.weight | BF16 | \[4096, 11008\] > model.layers.19.mlp.gate\_proj.weight -\> blk.19.ffn\_gate.weight | BF16 | \[11008, 4096\] > model.layers.19.mlp.up\_proj.weight -\> blk.19.ffn\_up.weight | BF16 | \[11008, 4096\] > model.layers.19.post\_attention\_layernorm.weight -\> blk.19.ffn\_norm.weight | BF16 | \[4096\] > model.layers.19.self\_attn.k\_proj.weight -\> blk.19.attn\_k.weight | BF16 | \[4096, 4096\] > model.layers.19.self\_attn.o\_proj.weight -\> blk.19.attn\_output.weight | BF16 | \[4096, 4096\] > model.layers.19.self\_attn.q\_proj.weight -\> blk.19.attn\_q.weight | BF16 | \[4096, 4096\] > model.layers.19.self\_attn.v\_proj.weight -\> blk.19.attn\_v.weight | BF16 | \[4096, 4096\] > model.layers.20.input\_layernorm.weight -\> blk.20.attn\_norm.weight | BF16 | \[4096\] > model.layers.20.mlp.down\_proj.weight -\> blk.20.ffn\_down.weight | BF16 | \[4096, 11008\] > model.layers.20.mlp.gate\_proj.weight -\> blk.20.ffn\_gate.weight | BF16 | \[11008, 4096\] > model.layers.20.mlp.up\_proj.weight -\> blk.20.ffn\_up.weight | BF16 | \[11008, 4096\] > model.layers.20.post\_attention\_layernorm.weight -\> blk.20.ffn\_norm.weight | BF16 | \[4096\] > model.layers.20.self\_attn.k\_proj.weight -\> blk.20.attn\_k.weight | BF16 | \[4096, 4096\] > model.layers.20.self\_attn.o\_proj.weight -\> blk.20.attn\_output.weight | BF16 | \[4096, 4096\] > model.layers.20.self\_attn.q\_proj.weight -\> blk.20.attn\_q.weight | BF16 | \[4096, 4096\] > model.layers.20.self\_attn.v\_proj.weight -\> blk.20.attn\_v.weight | BF16 | \[4096, 4096\] > model.layers.21.input\_layernorm.weight -\> blk.21.attn\_norm.weight | BF16 | \[4096\] > model.layers.21.mlp.down\_proj.weight -\> blk.21.ffn\_down.weight | BF16 | \[4096, 11008\] > model.layers.21.mlp.gate\_proj.weight -\> blk.21.ffn\_gate.weight | BF16 | \[11008, 4096\] > model.layers.21.mlp.up\_proj.weight -\> blk.21.ffn\_up.weight | BF16 | \[11008, 4096\] > model.layers.21.post\_attention\_layernorm.weight -\> blk.21.ffn\_norm.weight | BF16 | \[4096\] > model.layers.21.self\_attn.k\_proj.weight -\> blk.21.attn\_k.weight | BF16 | \[4096, 4096\] > model.layers.21.self\_attn.o\_proj.weight -\> blk.21.attn\_output.weight | BF16 | \[4096, 4096\] > model.layers.21.self\_attn.q\_proj.weight -\> blk.21.attn\_q.weight | BF16 | \[4096, 4096\] > model.layers.21.self\_attn.v\_proj.weight -\> blk.21.attn\_v.weight | BF16 | \[4096, 4096\] > model.layers.22.input\_layernorm.weight -\> blk.22.attn\_norm.weight | BF16 | \[4096\] > model.layers.22.mlp.down\_proj.weight -\> blk.22.ffn\_down.weight | BF16 | \[4096, 11008\] > model.layers.22.mlp.gate\_proj.weight -\> blk.22.ffn\_gate.weight | BF16 | \[11008, 4096\] > model.layers.22.mlp.up\_proj.weight -\> blk.22.ffn\_up.weight | BF16 | \[11008, 4096\] > model.layers.22.post\_attention\_layernorm.weight -\> blk.22.ffn\_norm.weight | BF16 | \[4096\] > model.layers.22.self\_attn.k\_proj.weight -\> blk.22.attn\_k.weight | BF16 | \[4096, 4096\] > model.layers.22.self\_attn.o\_proj.weight -\> blk.22.attn\_output.weight | BF16 | \[4096, 4096\] > model.layers.22.self\_attn.q\_proj.weight -\> blk.22.attn\_q.weight | BF16 | \[4096, 4096\] > model.layers.22.self\_attn.v\_proj.weight -\> blk.22.attn\_v.weight | BF16 | \[4096, 4096\] > model.layers.23.self\_attn.k\_proj.weight -\> blk.23.attn\_k.weight | BF16 | \[4096, 4096\] > model.layers.23.self\_attn.o\_proj.weight -\> blk.23.attn\_output.weight | BF16 | \[4096, 4096\] > model.layers.23.self\_attn.q\_proj.weight -\> blk.23.attn\_q.weight | BF16 | \[4096, 4096\] > model.layers.23.self\_attn.v\_proj.weight -\> blk.23.attn\_v.weight | BF16 | \[4096, 4096\] > lm\_head.weight -\> output.weight | BF16 | \[56064, 4096\] > model.layers.23.input\_layernorm.weight -\> blk.23.attn\_norm.weight | BF16 | \[4096\] > model.layers.23.mlp.down\_proj.weight -\> blk.23.ffn\_down.weight | BF16 | \[4096, 11008\] > model.layers.23.mlp.gate\_proj.weight -\> blk.23.ffn\_gate.weight | BF16 | \[11008, 4096\] > model.layers.23.mlp.up\_proj.weight -\> blk.23.ffn\_up.weight | BF16 | \[11008, 4096\] > model.layers.23.post\_attention\_layernorm.weight -\> blk.23.ffn\_norm.weight | BF16 | \[4096\] > model.layers.24.input\_layernorm.weight -\> blk.24.attn\_norm.weight | BF16 | \[4096\] > model.layers.24.mlp.down\_proj.weight -\> blk.24.ffn\_down.weight | BF16 | \[4096, 11008\] > model.layers.24.mlp.gate\_proj.weight -\> blk.24.ffn\_gate.weight | BF16 | \[11008, 4096\] > model.layers.24.mlp.up\_proj.weight -\> blk.24.ffn\_up.weight | BF16 | \[11008, 4096\] > model.layers.24.post\_attention\_layernorm.weight -\> blk.24.ffn\_norm.weight | BF16 | \[4096\] > model.layers.24.self\_attn.k\_proj.weight -\> blk.24.attn\_k.weight | BF16 | \[4096, 4096\] > model.layers.24.self\_attn.o\_proj.weight -\> blk.24.attn\_output.weight | BF16 | \[4096, 4096\] > model.layers.24.self\_attn.q\_proj.weight -\> blk.24.attn\_q.weight | BF16 | \[4096, 4096\] > model.layers.24.self\_attn.v\_proj.weight -\> blk.24.attn\_v.weight | BF16 | \[4096, 4096\] > model.layers.25.input\_layernorm.weight -\> blk.25.attn\_norm.weight | BF16 | \[4096\] > model.layers.25.mlp.down\_proj.weight -\> blk.25.ffn\_down.weight | BF16 | \[4096, 11008\] > model.layers.25.mlp.gate\_proj.weight -\> blk.25.ffn\_gate.weight | BF16 | \[11008, 4096\] > model.layers.25.mlp.up\_proj.weight -\> blk.25.ffn\_up.weight | BF16 | \[11008, 4096\] > model.layers.25.post\_attention\_layernorm.weight -\> blk.25.ffn\_norm.weight | BF16 | \[4096\] > model.layers.25.self\_attn.k\_proj.weight -\> blk.25.attn\_k.weight | BF16 | \[4096, 4096\] > model.layers.25.self\_attn.o\_proj.weight -\> blk.25.attn\_output.weight | BF16 | \[4096, 4096\] > model.layers.25.self\_attn.q\_proj.weight -\> blk.25.attn\_q.weight | BF16 | \[4096, 4096\] > model.layers.25.self\_attn.v\_proj.weight -\> blk.25.attn\_v.weight | BF16 | \[4096, 4096\] > model.layers.26.input\_layernorm.weight -\> blk.26.attn\_norm.weight | BF16 | \[4096\] > model.layers.26.mlp.down\_proj.weight -\> blk.26.ffn\_down.weight | BF16 | \[4096, 11008\] > model.layers.26.mlp.gate\_proj.weight -\> blk.26.ffn\_gate.weight | BF16 | \[11008, 4096\] > model.layers.26.mlp.up\_proj.weight -\> blk.26.ffn\_up.weight | BF16 | \[11008, 4096\] > model.layers.26.post\_attention\_layernorm.weight -\> blk.26.ffn\_norm.weight | BF16 | \[4096\] > model.layers.26.self\_attn.k\_proj.weight -\> blk.26.attn\_k.weight | BF16 | \[4096, 4096\] > model.layers.26.self\_attn.o\_proj.weight -\> blk.26.attn\_output.weight | BF16 | \[4096, 4096\] > model.layers.26.self\_attn.q\_proj.weight -\> blk.26.attn\_q.weight | BF16 | \[4096, 4096\] > model.layers.26.self\_attn.v\_proj.weight -\> blk.26.attn\_v.weight | BF16 | \[4096, 4096\] > model.layers.27.input\_layernorm.weight -\> blk.27.attn\_norm.weight | BF16 | \[4096\] > model.layers.27.mlp.down\_proj.weight -\> blk.27.ffn\_down.weight | BF16 | \[4096, 11008\] > model.layers.27.mlp.gate\_proj.weight -\> blk.27.ffn\_gate.weight | BF16 | \[11008, 4096\] > model.layers.27.mlp.up\_proj.weight -\> blk.27.ffn\_up.weight | BF16 | \[11008, 4096\] > model.layers.27.post\_attention\_layernorm.weight -\> blk.27.ffn\_norm.weight | BF16 | \[4096\] > model.layers.27.self\_attn.k\_proj.weight -\> blk.27.attn\_k.weight | BF16 | \[4096, 4096\] > model.layers.27.self\_attn.o\_proj.weight -\> blk.27.attn\_output.weight | BF16 | \[4096, 4096\] > model.layers.27.self\_attn.q\_proj.weight -\> blk.27.attn\_q.weight | BF16 | \[4096, 4096\] > model.layers.27.self\_attn.v\_proj.weight -\> blk.27.attn\_v.weight | BF16 | \[4096, 4096\] > model.layers.28.input\_layernorm.weight -\> blk.28.attn\_norm.weight | BF16 | \[4096\] > model.layers.28.mlp.down\_proj.weight -\> blk.28.ffn\_down.weight | BF16 | \[4096, 11008\] > model.layers.28.mlp.gate\_proj.weight -\> blk.28.ffn\_gate.weight | BF16 | \[11008, 4096\] > model.layers.28.mlp.up\_proj.weight -\> blk.28.ffn\_up.weight | BF16 | \[11008, 4096\] > model.layers.28.post\_attention\_layernorm.weight -\> blk.28.ffn\_norm.weight | BF16 | \[4096\] > model.layers.28.self\_attn.k\_proj.weight -\> blk.28.attn\_k.weight | BF16 | \[4096, 4096\] > model.layers.28.self\_attn.o\_proj.weight -\> blk.28.attn\_output.weight | BF16 | \[4096, 4096\] > model.layers.28.self\_attn.q\_proj.weight -\> blk.28.attn\_q.weight | BF16 | \[4096, 4096\] > model.layers.28.self\_attn.v\_proj.weight -\> blk.28.attn\_v.weight | BF16 | \[4096, 4096\] > model.layers.29.input\_layernorm.weight -\> blk.29.attn\_norm.weight | BF16 | \[4096\] > model.layers.29.mlp.down\_proj.weight -\> blk.29.ffn\_down.weight | BF16 | \[4096, 11008\] > model.layers.29.mlp.gate\_proj.weight -\> blk.29.ffn\_gate.weight | BF16 | \[11008, 4096\] > model.layers.29.mlp.up\_proj.weight -\> blk.29.ffn\_up.weight | BF16 | \[11008, 4096\] > model.layers.29.post\_attention\_layernorm.weight -\> blk.29.ffn\_norm.weight | BF16 | \[4096\] > model.layers.29.self\_attn.k\_proj.weight -\> blk.29.attn\_k.weight | BF16 | \[4096, 4096\] > model.layers.29.self\_attn.o\_proj.weight -\> blk.29.attn\_output.weight | BF16 | \[4096, 4096\] > model.layers.29.self\_attn.q\_proj.weight -\> blk.29.attn\_q.weight | BF16 | \[4096, 4096\] > model.layers.29.self\_attn.v\_proj.weight -\> blk.29.attn\_v.weight | BF16 | \[4096, 4096\] > model.layers.30.input\_layernorm.weight -\> blk.30.attn\_norm.weight | BF16 | \[4096\] > model.layers.30.mlp.down\_proj.weight -\> blk.30.ffn\_down.weight | BF16 | \[4096, 11008\] > model.layers.30.mlp.gate\_proj.weight -\> blk.30.ffn\_gate.weight | BF16 | \[11008, 4096\] > model.layers.30.mlp.up\_proj.weight -\> blk.30.ffn\_up.weight | BF16 | \[11008, 4096\] > model.layers.30.post\_attention\_layernorm.weight -\> blk.30.ffn\_norm.weight | BF16 | \[4096\] > model.layers.30.self\_attn.k\_proj.weight -\> blk.30.attn\_k.weight | BF16 | \[4096, 4096\] > model.layers.30.self\_attn.o\_proj.weight -\> blk.30.attn\_output.weight | BF16 | \[4096, 4096\] > model.layers.30.self\_attn.q\_proj.weight -\> blk.30.attn\_q.weight | BF16 | \[4096, 4096\] > model.layers.30.self\_attn.v\_proj.weight -\> blk.30.attn\_v.weight | BF16 | \[4096, 4096\] > model.layers.31.input\_layernorm.weight -\> blk.31.attn\_norm.weight | BF16 | \[4096\] > model.layers.31.mlp.down\_proj.weight -\> blk.31.ffn\_down.weight | BF16 | \[4096, 11008\] > model.layers.31.mlp.gate\_proj.weight -\> blk.31.ffn\_gate.weight | BF16 | \[11008, 4096\] > model.layers.31.mlp.up\_proj.weight -\> blk.31.ffn\_up.weight | BF16 | \[11008, 4096\] > model.layers.31.post\_attention\_layernorm.weight -\> blk.31.ffn\_norm.weight | BF16 | \[4096\] > model.layers.31.self\_attn.k\_proj.weight -\> blk.31.attn\_k.weight | BF16 | \[4096, 4096\] > model.layers.31.self\_attn.o\_proj.weight -\> blk.31.attn\_output.weight | BF16 | \[4096, 4096\] > model.layers.31.self\_attn.q\_proj.weight -\> blk.31.attn\_q.weight | BF16 | \[4096, 4096\] > model.layers.31.self\_attn.v\_proj.weight -\> blk.31.attn\_v.weight | BF16 | \[4096, 4096\] > model.norm.weight -\> output\_norm.weight | BF16 | \[4096\] > Writing ch\_taide\_medicine.gguf-unsloth.F16.gguf, format 1 > Traceback (most recent call last): > File "/GPUData/working/unsloth/convert\_\_unsloth\_to\_gguf.py", line 44, in \ > if True: model.save\_pretrained\_gguf("ch\_taide\_medicine.gguf", tokenizer, quantization\_method = "quantized") > File "/home/chtseng/envs/LM2/lib/python3.10/site-packages/unsloth/save.py", line 1333, in unsloth\_save\_pretrained\_gguf > file\_location = save\_to\_gguf(model\_type, new\_save\_directory, quantization\_method, first\_conversion, makefile) > File "/home/chtseng/envs/LM2/lib/python3.10/site-packages/unsloth/save.py", line 957, in save\_to\_gguf > raise RuntimeError( > RuntimeError: Unsloth: Quantization failed for ./ch\_taide\_medicine.gguf-unsloth.F16.gguf > You might have to compile llama.cpp yourself, then run this again. > You do not need to close this Python program. Run the following commands in a new terminal: > You must run this in the same folder as you're saving your model. > git clone https://github.com/ggerganov/llama.cpp > cd llama.cpp && make clean && LLAMA\_CUDA=1 make all -j > Once that's done, redo the quantization. ---
**Author:** ![jonACE](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/jonace/32/44664_2.png) [@jonACE](https://discuss.huggingface.co/u/jonACE)\ **Post date:** [April 3, 2025, 8:51am UTC](https://discuss.huggingface.co/t/issues-when-trying-to-build-llama-cpp/148603/5 "2025-04-03T08:51:40Z")
Hi, I’m using streamlit which runs the python app which contains the training code as well as the conversion and uploading. I’m not sure I can do the ‘pip install cmake’. I need some more help on this. Thanks! ---
**Author:** ![John6666](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/john6666/32/27664_2.png) [@John6666](https://discuss.huggingface.co/u/John6666)\ **Post date:** [April 3, 2025, 9:00am UTC](https://discuss.huggingface.co/t/issues-when-trying-to-build-llama-cpp/148603/6 "2025-04-03T09:00:00Z")
If you are using official sample code or containers, it will probably be easier to find the problem. Or, if it is an official container, it feels like a bug if something is missing… ---
**Author:** ![jonACE](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/jonace/32/44664_2.png) [@jonACE](https://discuss.huggingface.co/u/jonACE)\ **Post date:** [April 3, 2025, 9:04am UTC](https://discuss.huggingface.co/t/issues-when-trying-to-build-llama-cpp/148603/7 "2025-04-03T09:04:34Z")
I am using this as a reference: > **[Google Colab](https://colab.research.google.com/github/unslothai/notebooks/blob/main/nb/Llama3.1_(8B)-Alpaca.ipynb#scrollTo=pCqnaKmlO1U9)** ---
**Author:** ![John6666](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/john6666/32/27664_2.png) [@John6666](https://discuss.huggingface.co/u/John6666)\ **Post date:** [April 3, 2025, 9:43am UTC](https://discuss.huggingface.co/t/issues-when-trying-to-build-llama-cpp/148603/8 "2025-04-03T09:43:48Z")
There were similar cases and ways to deal with them, but the method changes depending on the environment you’re using… If it’s a local, real Python environment, it’s pip as mentioned above, and if it’s Colab, it’s like this. ```py !pip install cmake ``` If it’s a container, the way to write and operate it differs depending on the container software… > > > As of 3rd December 2024 - fixed. > > Please update Unsloth via > \`\`\` > pip install …--upgrade --no-deps --no-cache-dir unsloth > \`\`\`