vbatts/llama.cpp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Add transformers llava next tensor name mapping

Browse source 
Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com>
This commit is contained in:
Alex-Brooks 2025-01-22 01:44:48 -07:00
parent cc1c135367
commit 50504063b2
1 changed files with 12 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

12
examples/llava/convert_image_encoder_to_gguf.py
View file

@ -37,6 +37,18 @@ def should_skip_tensor(name: str, has_text: bool, has_vision: bool, has_llava: b
def get_tensor_name(name: str) -> str:
# Standardize the transformers llava next keys for
# image newline / mm projector with the classes in haotian-liu LLaVA
if name == "image_newline":
return "model.image_newline"
if name.startswith("multi_modal_projector"):
name = name.replace("multi_modal_projector", "mm")
if name.endswith("linear_1"):
name = name.replace("linear_1", "0")
if name.endswith("linear_2"):
name = name.replace("linear_2", "1")
return name
if "projection" in name:
return name
if "mm_projector" in name: