vbatts/llama.cpp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

test anchor link

Browse source
This commit is contained in:
Yuval Peled 2023-06-02 14:47:00 +03:00
parent ffb06a345e
commit 89b377d7f8
3 changed files with 9 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

2
README.md
View file

@ -293,6 +293,8 @@ Building the program with BLAS support may lead to some performance improvements
cmake --build . -config Release
```
<a name="cublas"></a>
- **cuBLAS**
This provides BLAS acceleration using the CUDA cores of your Nvidia GPU. Make sure to have the CUDA toolkit installed. You can download it from your Linux distro's package manager or from here: [CUDA Toolkit](https://developer.nvidia.com/cuda-downloads).

0
BLIS.md → docs/BLIS.md
View file

7
docs/token_generation_performance_tips.md Normal file
View file

@ -0,0 +1,7 @@
# Token generation performance tips
## Verifying that the model is running on the GPU
Make sure you compiled llama with the correct env variables according to [this guide](../README.md#cublas)
When running `llama.cpp`, outputs some helpful diagnostic information to stderr.
To verify that the workload is