vbatts/llama.cpp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

llava : support Minicpm-omni (#11289)

Browse source 
* init

* add readme

* update readme

* no use make

* update readme

* update fix code

* fix editorconfig-checker

* no change convert py

* use clip_image_u8_free
This commit is contained in:
tc-mb 2025-01-22 15:35:48 +08:00 • committed by GitHub
parent 6171c9d258
commit 3e3357fd77
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
6 changed files with 100 additions and 15 deletions
Download patch file Download diff file Expand all files Collapse all files

2
examples/llava/minicpmv-surgery.py
View file

@ -8,7 +8,7 @@ ap.add_argument("-m", "--model", help="Path to MiniCPM-V model")
args = ap.parse_args()
# find the model part that includes the the multimodal projector weights
model = AutoModel.from_pretrained(args.model, trust_remote_code=True, local_files_only=True)
model = AutoModel.from_pretrained(args.model, trust_remote_code=True, local_files_only=True, torch_dtype=torch.bfloat16)
checkpoint = model.state_dict()
# get a list of mm tensor names