vbatts/llama.cpp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

update README.md

Browse source
This commit is contained in:
李为 2024-11-08 22:05:57 +08:00
parent 3d9c63a3ff
commit eb6d54679e
1 changed files with 19 additions and 11 deletions
Download patch file Download diff file Expand all files Collapse all files

30
examples/omni-vlm/README.md
View file

@ -1,22 +1,30 @@
# omni-vlm # omni-vlm
Currently this implementation supports [omni-vlm](https://huggingface.co/NexaAIDev/nano-vlm-instruct) variants, Currently this implementation supports:
After API is confirmed, more models will be supported / uploaded. * [nano-vlm-instruct](https://huggingface.co/NexaAIDev/nano-vlm-instruct/tree/main) ([gguf](https://huggingface.co/NexaAIDev/nano-vlm-instruct-gguf/tree/main))
* [vlm-81-ocr](https://huggingface.co/NexaAIDev/vlm-81-ocr/tree/main) ([gguf](https://huggingface.co/NexaAIDev/vlm-81-ocr-gguf/tree/main))
* [vlm-81-instruct](https://huggingface.co/NexaAIDev/vlm-81-instruct/tree/main) ([gguf](https://huggingface.co/NexaAIDev/vlm-81-instruct-gguf/tree/main))
After API is stable, more models will be supported.
## Usage ## Usage
Build with cmake in the `llama-cpp-experiments` folder:
```bash Build with cmake in the `llama.cpp` folder:
```console
cmake -S . -B build -DCMAKE_BUILD_TYPE=RelWithDebInfo cmake -S . -B build -DCMAKE_BUILD_TYPE=RelWithDebInfo
cmake --build build --verbose -j cmake --build build --verbose -j
``` ```
After building, run: `./omni-vlm-cli` to see the usage. For example: After building, run: `./omni-vlm-cli` to see the usage. For example:
```bash ```console
./omni-vlm-cli \ ./omni-vlm-cli \
-m Nano-Llm-494M-F16.gguf \ -m <llm-F16.gguf> \
--mmproj mmproj-omni-vlm-f16.gguf \ --mmproj <mmproj-F16.gguf> \
--image example/omni-vlm/cat.png --image example/omni-vlm/cat.png \
--omni-vlm-version <vlm-81-ocr | vlm-81-instruct | nano-vlm-instruct>
``` ```
See next section to convert gguf files from original safetensors. See next section to convert gguf files from original safetensors.
@ -27,6 +35,7 @@ See next section to convert gguf files from original safetensors.
) )
## Omni-vlm gguf conversion ## Omni-vlm gguf conversion
1) First clone omni-vlm model: 1) First clone omni-vlm model:
```console ```console
git clone https://huggingface.co/NexaAIDev/nano-vlm-instruct git clone https://huggingface.co/NexaAIDev/nano-vlm-instruct
@ -34,7 +43,7 @@ git clone https://huggingface.co/NexaAIDev/nano-vlm-instruct
2) Install the required Python packages: 2) Install the required Python packages:
```sh ```console
pip install -r examples/omni-vlm/requirements.txt pip install -r examples/omni-vlm/requirements.txt
``` ```
@ -104,6 +113,5 @@ After successfully compiling omni_vlm_wrapper_shared dynamic library, run:
python omni_vlm_demo.py \ python omni_vlm_demo.py \
--model <PATH TO nano-vlm-processor>/Nano-Llm-494M-F16.gguf \ --model <PATH TO nano-vlm-processor>/Nano-Llm-494M-F16.gguf \
--mmproj <PATH TO nano-vlm-instruct>/mmproj-omni-vlm-f16.gguf \ --mmproj <PATH TO nano-vlm-instruct>/mmproj-omni-vlm-f16.gguf \
--prompt="Describe this image for me" \ --omni-vlm-version <vlm-81-ocr | vlm-81-instruct | nano-vlm-instruct>
--image-path cat.png
``` ```