diff --git a/examples/minicpmv/README.md b/examples/minicpmv/README.md index 794fbbb29..6bb0feab4 100644 --- a/examples/minicpmv/README.md +++ b/examples/minicpmv/README.md @@ -1,17 +1,31 @@ -### 所有命令在 llama.cpp 根目录执行,模型位于根目录上级目录处 -### All command should be executed under the root path of llama.cpp repo. We assume the MiniCPM-V-2.5 model are put in its parent folder. +## Instructions +Download model files from huggingface to "MiniCPM-V-2_5" folder. +Clone code +```bash +git clone -b minicpm-v2.5 https://github.com/OpenBMB/llama.cpp.git +cd llama.cpp +``` + +Prepare the model + +```bash +python ./examples/minicpmv/minicpmv-surgery.py -m ../MiniCPM-V-2_5 +python ./examples/minicpmv/minicpmv-convert-image-encoder-to-gguf.py -m ../MiniCPM-V-2_5 --minicpmv-projector ../MiniCPM-V-2_5/minicpmv.projector --output-dir ../MiniCPM-V-2_5/ --image-mean 0.5 0.5 0.5 --image-std 0.5 0.5 0.5 +python ./convert.py ../MiniCPM-V-2_5/model --outtype f16 --vocab-type bpe + +# quantize int4 version +./quantize ../MiniCPM-V-2_5/model/model-8B-F16.gguf ../MiniCPM-V-2_5/model/ggml-model-Q4_K_M.gguf Q4_K_M +``` +Try to inference ```bash make make minicpmv-cli -python ./examples/minicpmv/minicpmv-surgery.py -m ../MiniCPM-V-2_5 -python ./examples/minicpmv/minicpmv-convert-image-encoder-to-gguf.py -m ../MiniCPM-V-2_5 --minicpmv-projector ../MiniCPM-V-2_5/minicpmv.projector --output-dir ../MiniCPM-V-2_5/ --image-mean 0.5 0.5 0.5 --image-std 0.5 0.5 0.5 -python ./convert.py ../MiniCPM-V-2_5/model --outtype f16 --vocab-type bpe +# run quantize f16 version ./minicpmv-cli -m ../MiniCPM-V-2_5/model/model-8B-F16.gguf --mmproj ../MiniCPM-V-2_5/mmproj-model-f16.gguf -c 4096 --temp 0.7 --top-p 0.8 --top-k 100 --repeat-penalty 1.05 --image xx.jpg -p "What is in the image?" -# or run quantize int4 version -./quantize ../MiniCPM-V-2_5/model/model-8B-F16.gguf ../MiniCPM-V-2_5/model/ggml-model-Q4_K_M.gguf Q4_K_M +# run quantize int4 version ./minicpmv-cli -m ../MiniCPM-V-2_5/model/ggml-model-Q4_K_M.gguf --mmproj ../MiniCPM-V-2_5/mmproj-model-f16.gguf -c 4096 --temp 0.7 --top-p 0.8 --top-k 100 --repeat-penalty 1.05 --image xx.jpg -p "What is in the image?" # or run in interactive mode @@ -20,7 +34,7 @@ python ./convert.py ../MiniCPM-V-2_5/model --outtype f16 --vocab-type bpe ### Android -#### Build on Android using Termux +#### Build for Android using Termux [Termux](https://github.com/termux/termux-app#installation) is a method to execute `llama.cpp` on an Android device (no root required). ``` apt update && apt upgrade -y