vbatts/llama.cpp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

convert: Fix detection of LLAMA2

Browse source 
In recent downloads of LLAMA2 dataset the norm_eps is set to 1e-06, this
leads to convert.py erroneously considering the model to be LLAMA1 and
setting the context to 2k tokens.

Fix it by extending the existing hack to also check for the 1e-06 value.
This commit is contained in:
Nikolay Borisov 2023-11-06 13:01:00 +02:00
parent 2833a6f63c
commit d6d905b242
1 changed files with 1 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

2
convert.py
View file

@ -250,7 +250,7 @@ class Params:
if config.get("rope_theta") == 1000000: if config.get("rope_theta") == 1000000:
# CodeLlama # CodeLlama
n_ctx = 16384 n_ctx = 16384
elif config["norm_eps"] == 1e-05: elif config["norm_eps"] in (1e-05, 1e-06):
# LLaMA v2 # LLaMA v2
n_ctx = 4096 n_ctx = 4096
else: else: