vbatts/llama.cpp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

convert: Fix handling of LLAMA2 vocab_size = -1

Browse source 
When vocab_size is detected to be -1 simply remove its value from the
parsed params.json and fallback to using the tok_embeddings.weight.

Fixes  #3900
This commit is contained in:
Nikolay Borisov 2023-11-06 13:03:31 +02:00
parent d6d905b242
commit f36a777bbc
1 changed files with 5 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

5
convert.py
View file

@ -253,6 +253,11 @@ class Params:
elif config["norm_eps"] in (1e-05, 1e-06): elif config["norm_eps"] in (1e-05, 1e-06):
# LLaMA v2 # LLaMA v2
n_ctx = 4096 n_ctx = 4096
# For some reason FB writes -1 to vocab size for their LLAMA2 models
# simply remove this bogus value and let the return statement belo
# figure it out
if config["vocab_size"] == -1:
del config["vocab_size"]
else: else:
# LLaMA v1 # LLaMA v1
n_ctx = 2048 n_ctx = 2048