vbatts/llama.cpp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

readme: add convert-hf-to-gguf.py in example

Browse source 
I'm using huggingface model and find out that I need to use different script. So I make this change for people to save time instead of read or open new issue
This commit is contained in:
hiepxanh 2024-03-04 21:12:50 -08:00 • committed by GitHub
parent 29eee40474
commit eaff11ca55
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
1 changed files with 3 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

3
README.md
View file

@ -698,6 +698,9 @@ python convert.py models/mymodel/ --vocab-type bpe
# update the gguf filetype to current version if older version is now unsupported # update the gguf filetype to current version if older version is now unsupported
./quantize ./models/mymodel/ggml-model-Q4_K_M.gguf ./models/mymodel/ggml-model-Q4_K_M-v2.gguf COPY ./quantize ./models/mymodel/ggml-model-Q4_K_M.gguf ./models/mymodel/ggml-model-Q4_K_M-v2.gguf COPY
# convert the hunggingface model to ggml FP16 format
python3 convert-hf-to-gguf.py models/mymodel/
``` ```
### Run the quantized model ### Run the quantized model