vbatts/llama.cpp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

llava : add support for moondream vision language model (#6899)

Browse source 
* add support for moondream vision language model

This required making the following changes to the CLIP model:

1. Support for patch embedding bias.
2. Make class embedding and pre-layernorm optional.
3. Add support for post-layernorm.

* Update examples/llava/clip.cpp

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
This commit is contained in:
vik 2024-04-25 12:38:31 -07:00 • committed by GitHub
parent dba497e0c1
commit 46e12c4692
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
2 changed files with 61 additions and 11 deletions
Download patch file Download diff file Expand all files Collapse all files

1
README.md
View file

@ -138,6 +138,7 @@ Typically finetunes of the base models below are supported as well.
- [x] [MobileVLM 1.7B/3B models](https://huggingface.co/models?search=mobileVLM)
- [x] [Yi-VL](https://huggingface.co/models?search=Yi-VL)
- [x] [Mini CPM](https://huggingface.co/models?search=MiniCPM)
- [x] [Moondream](https://huggingface.co/vikhyatk/moondream2)
**HTTP server**