vbatts/llama.cpp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

add READMD for llava.py

Browse source
This commit is contained in:
ningshanwutuobang 2023-06-17 16:43:36 +08:00
parent 4f1aa3cc76
commit 93c57a0571
1 changed files with 6 additions and 3 deletions
Download patch file Download diff file Expand all files Collapse all files

9
examples/embd_input/README.md
View file

@ -3,8 +3,12 @@
## LLAVA example (llava.py) ## LLAVA example (llava.py)
1. obtian llava model (following https://github.com/haotian-liu/LLaVA/ , use https://huggingface.co/liuhaotian/LLaVA-13b-delta-v1-1/) 1. obtian llava model (following https://github.com/haotian-liu/LLaVA/ , use https://huggingface.co/liuhaotian/LLaVA-13b-delta-v1-1/)
2. convert it to ggml format 2. build `libembd_input.so`
3. llava_projection.pth is [pytorch_model-00003-of-00003.bin](https://huggingface.co/liuhaotian/LLaVA-13b-delta-v1-1/blob/main/pytorch_model-00003-of-00003.bin) ```
make
```
3. convert it to ggml format
4. llava_projection.pth is [pytorch_model-00003-of-00003.bin](https://huggingface.co/liuhaotian/LLaVA-13b-delta-v1-1/blob/main/pytorch_model-00003-of-00003.bin)
``` ```
import torch import torch
@ -17,4 +21,3 @@ used_key = ["model.mm_projector.weight","model.mm_projector.bias"]
torch.save({k: dic[k] for k in used_key}, pth_path) torch.save({k: dic[k] for k in used_key}, pth_path)
``` ```