vbatts/llama.cpp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

chore: add references to the quantisation space.

Browse source
This commit is contained in:
Vaibhav Srivastav 2024-05-14 23:01:11 +02:00
parent 9f773486ab
commit 3775d0debb
2 changed files with 7 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

4
README.md
View file

@ -712,6 +712,10 @@ Building the program with BLAS support may lead to some performance improvements
### Prepare and Quantize
Note: You can use the [GGUF-my-repo](https://huggingface.co/spaces/ggml-org/gguf-my-repo) space on Hugging Face to quantise your model weights without any setup too.
It is synced to `llama.cpp` main every 6 hours.
To obtain the official LLaMA 2 weights please see the <a href="#obtaining-and-using-the-facebook-llama-2-model">Obtaining and using the Facebook LLaMA 2 model</a> section. There is also a large selection of pre-quantized `gguf` models available on Hugging Face.
Note: `convert.py` does not support LLaMA 3, you can use `convert-hf-to-gguf.py` with LLaMA 3 downloaded from Hugging Face.

4
examples/quantize/README.md
View file

@ -1,6 +1,8 @@
# quantize
TODO
You can also use the [GGUF-my-repo](https://huggingface.co/spaces/ggml-org/gguf-my-repo) space on Hugging Face to build your own quants without any setup.
Note: It is synced to llama.cpp `main` every 6 hours.
## Llama 2 7B