Improve README.md for building in Termux on Android devices.

*Added directions to build with `make`, and simplified `OpenBlas` method. 

*Now it's more clear where to _move_ the models in Termux. 

*`llama.cpp` is diverse, so I added a Usage example for new users.

I left `CLBlast` instructions unchanged. I may try to clean it up in the future.
This commit is contained in:
JackJollimore 2023-08-27 14:27:37 -03:00 committed by GitHub
parent c10704d01e
commit 9c8b14bc47
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23

View file

@ -771,37 +771,42 @@ And after 4.45 hours, you will have the final perplexity.
### Android ### Android
#### Building the Project using Android NDK #### Building the Project in Termux (F-Droid)
You can easily run `llama.cpp` on Android device with [termux](https://termux.dev/). [Termux](https://termux.dev/) is an alternative method to run `llama.cpp` on Android devices.
First, install the essential packages for termux: Ensure Termux is up to date and clone the repo:
``` ```
pkg install clang wget git cmake apt update && apt upgrade
``` $HOME
Second, obtain the [Android NDK](https://developer.android.com/ndk) and then build with CMake: git clone https://github.com/ggerganov/llama.cpp
```
$ mkdir build-android
$ cd build-android
$ export NDK=<your_ndk_directory>
$ cmake -DCMAKE_TOOLCHAIN_FILE=$NDK/build/cmake/android.toolchain.cmake -DANDROID_ABI=arm64-v8a -DANDROID_PLATFORM=android-23 -DCMAKE_C_FLAGS=-march=armv8.4a+dotprod ..
$ make
```
Install [termux](https://termux.dev/) on your device and run `termux-setup-storage` to get access to your SD card.
Finally, copy the `llama` binary and the model files to your device storage. Here is a demo of an interactive session running on Pixel 5 phone:
https://user-images.githubusercontent.com/271616/225014776-1d567049-ad71-4ef2-b050-55b0b3b9274c.mp4
#### Building the Project using Termux (F-Droid)
Termux from F-Droid offers an alternative route to execute the project on an Android device. This method empowers you to construct the project right from within the terminal, negating the requirement for a rooted device or SD Card.
Outlined below are the directives for installing the project using OpenBLAS and CLBlast. This combination is specifically designed to deliver peak performance on recent devices that feature a GPU.
If you opt to utilize OpenBLAS, you'll need to install the corresponding package.
```
apt install libopenblas
``` ```
Subsequently, if you decide to incorporate CLBlast, you'll first need to install the requisite OpenCL packages: Build `llama.cpp`:
```
$HOME
cd llama.cpp
make
```
It's possible to include OpenBlas while building:
````
$HOME
pkg install libopenblas
cd llama.cpp
cmake -B build -DLLAMA_BLAS=ON -DLLAMA_BLAS_VENDOR=OpenBLAS
cd build
cmake --build . --config Release
```
Move your model to the $HOME directory, for example:
```
cd storage/downloads
mv 7b-model.gguf ~/
```
Usage example:`./llama.cpp/main -m ~/7b-model.gguf --color -c 2048 --keep -1 -n -2 -b 10 -i -ins`
Alternatively, to enable CLBlast then install the requisite OpenCL packages:
``` ```
apt install ocl-icd opencl-headers opencl-clhpp clinfo apt install ocl-icd opencl-headers opencl-clhpp clinfo
``` ```
@ -830,9 +835,7 @@ export LD_LIBRARY_PATH=/vendor/lib64:$LD_LIBRARY_PATH
(Note: some Android devices, like the Zenfone 8, need the following command instead - "export LD_LIBRARY_PATH=/system/vendor/lib64:$LD_LIBRARY_PATH". Source: https://www.reddit.com/r/termux/comments/kc3ynp/opencl_working_in_termux_more_in_comments/ ) (Note: some Android devices, like the Zenfone 8, need the following command instead - "export LD_LIBRARY_PATH=/system/vendor/lib64:$LD_LIBRARY_PATH". Source: https://www.reddit.com/r/termux/comments/kc3ynp/opencl_working_in_termux_more_in_comments/ )
For easy and swift re-execution, consider documenting this final part in a .sh script file. This will enable you to rerun the process with minimal hassle. For easy and swift re-execution, consider documenting this final part in a .sh script file. This will allow you to run `./main (...)` with minimal hassle.
Place your desired model into the `~/llama.cpp/models/` directory and execute the `./main (...)` script.
### Docker ### Docker