vbatts/llama.cpp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Updating the documentation

Browse source 
Co-authored-by: Johannes Gäßler <johannesg@5d6.de>
This commit is contained in:
matteo 2024-07-31 18:10:36 +02:00 • committed by GitHub
parent 390f54eb74
commit 7444bad4f2
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
1 changed files with 1 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

2
docs/build.md
View file

@ -180,7 +180,7 @@ For Jetson user, if you have Jetson Orin, you can try this: [Offical Support](ht
The environment variable [`CUDA_VISIBLE_DEVICES`](https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#env-vars) can be used to specify which GPU(s) will be used.
The environment variable `GGML_CUDA_ENABLE_UNIFIED_MEMORY=1` can be used to enable unified memory in linux. This allows using system RAM when the GPU VRAM is exhausted. Should be enabled with `-ngl 99` to avoid sharing memory bandwidth with the CPU. In windows this setting is available in the nvidia control panel as `System Memory Fallback`.
The environment variable `GGML_CUDA_ENABLE_UNIFIED_MEMORY=1` can be used to enable unified memory in Linux. This allows swapping to system RAM instead of crashing when the GPU VRAM is exhausted. In Windows this setting is available in the NVIDIA control panel as `System Memory Fallback`.
The following compilation options are also available to tweak performance: