vbatts/llama.cpp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

convert-llama2c-to-ggml : enable conversion of GQA models (#6237)

Browse source 
* convert-llama2c-to-ggml: enable conversion of multiqueries, #5608

* add test in build action

* Update build.yml

* Update build.yml

* Update build.yml

* gg patch
This commit is contained in:
fraxy-v 2024-03-22 20:49:06 +02:00 • committed by GitHub
parent 1d0331c12a
commit 92397d87a4
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
3 changed files with 193 additions and 207 deletions
Download patch file Download diff file Expand all files Collapse all files

2
examples/convert-llama2c-to-ggml/README.md
View file

@ -21,6 +21,8 @@ An example command using a model from [karpathy/tinyllamas](https://huggingface.
`$ ./convert-llama2c-to-ggml --copy-vocab-from-model llama-2-7b-chat.gguf.q2_K.bin --llama2c-model stories42M.bin --llama2c-output-model stories42M.gguf.bin`
Note: The vocabulary for `stories260K.bin` should be its own tokenizer `tok512.bin` found in [karpathy/tinyllamas/stories260K](https://huggingface.co/karpathy/tinyllamas/tree/main/stories260K).
Now you can use the model with a command like:
`$ ./main -m stories42M.gguf.bin -p "One day, Lily met a Shoggoth" -n 500 -c 256`