vbatts/llama.cpp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

convert-llama-hf-to-gguf.py : fix attn_q permute

Browse source
This commit is contained in:
klosax 2023-08-21 04:42:09 +02:00 • committed by GitHub
parent 58bde5c5c1
commit c818c405e0
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
1 changed files with 3 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

4
convert-llama-hf-to-gguf.py
View file

@ -264,7 +264,9 @@ for part_name in part_names:
data = data.squeeze().numpy()
# reverse permute these
if name.endswith(".q_proj.weight") or name.endswith(".k_proj.weight"):
if name.endswith(".q_proj.weight"):
data = reverse_hf_permute(data, head_count)
if name.endswith(".k_proj.weight"):
data = reverse_hf_permute(data, head_count, head_count_kv)
# map tensor names