vbatts/llama.cpp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

convert-llama2c-to-ggml : enable conversion of GQA models (#6237)

Browse source 
* convert-llama2c-to-ggml: enable conversion of multiqueries, #5608

* add test in build action

* Update build.yml

* Update build.yml

* Update build.yml

* gg patch
This commit is contained in:
fraxy-v 2024-03-22 20:49:06 +02:00 • committed by GitHub
parent 1d0331c12a
commit 92397d87a4
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
3 changed files with 193 additions and 207 deletions
Download patch file Download diff file Expand all files Collapse all files

11
.github/workflows/build.yml vendored
View file

@ -225,6 +225,17 @@ jobs:
cd build
ctest -L main --verbose --timeout 900
- name: Test llama2c conversion
id: llama2c_test
run: |
cd build
echo "Fetch tokenizer"
wget https://huggingface.co/karpathy/tinyllamas/resolve/main/stories260K/tok512.bin
echo "Fetch llama2c model"
wget https://huggingface.co/karpathy/tinyllamas/resolve/main/stories260K/stories260K.bin
./bin/convert-llama2c-to-ggml --copy-vocab-from-model ./tok512.bin --llama2c-model stories260K.bin --llama2c-output-model stories260K.gguf
./bin/main -m stories260K.gguf -p "One day, Lily met a Shoggoth" -n 500 -c 256
# ubuntu-latest-cmake-sanitizer:
# runs-on: ubuntu-latest
#