vbatts/llama.cpp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

convert-llama2c-to-ggml: enable conversion of multiqueries, #5608

Browse source
This commit is contained in:
Y. Velkov 2024-03-19 15:23:59 +02:00
parent c4d7f81786
commit 284800b1e3
2 changed files with 146 additions and 184 deletions
Download patch file Download diff file Expand all files Collapse all files

2
examples/convert-llama2c-to-ggml/README.md
View file

@ -21,6 +21,8 @@ An example command using a model from [karpathy/tinyllamas](https://huggingface.
`$ ./convert-llama2c-to-ggml --copy-vocab-from-model llama-2-7b-chat.gguf.q2_K.bin --llama2c-model stories42M.bin --llama2c-output-model stories42M.gguf.bin`
Note: The vocabulary for `stories260K.bin` should be its own tokenizer `tok512.bin` found in [karpathy/tinyllamas/stories260K](https://huggingface.co/karpathy/tinyllamas/tree/main/stories260K).
Now you can use the model with a command like:
`$ ./main -m stories42M.gguf.bin -p "One day, Lily met a Shoggoth" -n 500 -c 256`