diff --git a/README.md b/README.md index bf3eb0b76..051cd803d 100644 --- a/README.md +++ b/README.md @@ -771,37 +771,42 @@ And after 4.45 hours, you will have the final perplexity. ### Android -#### Building the Project using Android NDK -You can easily run `llama.cpp` on Android device with [termux](https://termux.dev/). +#### Building the Project in Termux (F-Droid) +[Termux](https://termux.dev/) is an alternative method to run `llama.cpp` on Android devices. -First, install the essential packages for termux: +Ensure Termux is up to date and clone the repo: ``` -pkg install clang wget git cmake -``` -Second, obtain the [Android NDK](https://developer.android.com/ndk) and then build with CMake: -``` -$ mkdir build-android -$ cd build-android -$ export NDK= -$ cmake -DCMAKE_TOOLCHAIN_FILE=$NDK/build/cmake/android.toolchain.cmake -DANDROID_ABI=arm64-v8a -DANDROID_PLATFORM=android-23 -DCMAKE_C_FLAGS=-march=armv8.4a+dotprod .. -$ make -``` -Install [termux](https://termux.dev/) on your device and run `termux-setup-storage` to get access to your SD card. -Finally, copy the `llama` binary and the model files to your device storage. Here is a demo of an interactive session running on Pixel 5 phone: - -https://user-images.githubusercontent.com/271616/225014776-1d567049-ad71-4ef2-b050-55b0b3b9274c.mp4 - -#### Building the Project using Termux (F-Droid) -Termux from F-Droid offers an alternative route to execute the project on an Android device. This method empowers you to construct the project right from within the terminal, negating the requirement for a rooted device or SD Card. - -Outlined below are the directives for installing the project using OpenBLAS and CLBlast. This combination is specifically designed to deliver peak performance on recent devices that feature a GPU. - -If you opt to utilize OpenBLAS, you'll need to install the corresponding package. -``` -apt install libopenblas +apt update && apt upgrade +$HOME +git clone https://github.com/ggerganov/llama.cpp ``` -Subsequently, if you decide to incorporate CLBlast, you'll first need to install the requisite OpenCL packages: +Build `llama.cpp`: +``` +$HOME +cd llama.cpp +make +``` + +It's possible to include OpenBlas while building: +```` +$HOME +pkg install libopenblas +cd llama.cpp +cmake -B build -DLLAMA_BLAS=ON -DLLAMA_BLAS_VENDOR=OpenBLAS +cd build +cmake --build . --config Release +``` + +Move your model to the $HOME directory, for example: +``` +cd storage/downloads +mv 7b-model.gguf ~/ +``` + +Usage example:`./llama.cpp/main -m ~/7b-model.gguf --color -c 2048 --keep -1 -n -2 -b 10 -i -ins` + +Alternatively, to enable CLBlast then install the requisite OpenCL packages: ``` apt install ocl-icd opencl-headers opencl-clhpp clinfo ``` @@ -830,9 +835,7 @@ export LD_LIBRARY_PATH=/vendor/lib64:$LD_LIBRARY_PATH (Note: some Android devices, like the Zenfone 8, need the following command instead - "export LD_LIBRARY_PATH=/system/vendor/lib64:$LD_LIBRARY_PATH". Source: https://www.reddit.com/r/termux/comments/kc3ynp/opencl_working_in_termux_more_in_comments/ ) -For easy and swift re-execution, consider documenting this final part in a .sh script file. This will enable you to rerun the process with minimal hassle. - -Place your desired model into the `~/llama.cpp/models/` directory and execute the `./main (...)` script. +For easy and swift re-execution, consider documenting this final part in a .sh script file. This will allow you to run `./main (...)` with minimal hassle. ### Docker