vbatts/llama.cpp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

added numa dependencies as to use numa in numactl

Browse source
This commit is contained in:
Mohammadreza Hendiani 2024-04-20 16:24:19 +03:30
parent aed82f6837
commit cf025dfbdb
No known key found for this signature in database
GPG key ID: 6910AB8B88C616E8
1 changed files with 16 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

16
README.md
View file

@ -48,6 +48,7 @@ Inference of Meta's [LLaMA](https://arxiv.org/abs/2302.13971) model (and others)
<li><a href="#interactive-mode">Interactive mode</a></li>
<li><a href="#constrained-output-with-grammars">Constrained output with grammars</a></li>
<li><a href="#instruct-mode">Instruct mode</a></li>
<li><a href="#cli-commands">Cli commands</a></li>
<li><a href="#obtaining-and-using-the-facebook-llama-2-model">Obtaining and using the Facebook LLaMA 2 model</a></li>
<li><a href="#seminal-papers-and-background-on-the-models">Seminal papers and background on the models</a></li>
<li><a href="#perplexity-measuring-model-quality">Perplexity (measuring model quality)</a></li>
@ -893,6 +894,21 @@ cadaver, cauliflower, cabbage (vegetable), catalpa (tree) and Cailleach.
>
```
### Cli commands
- #### Numa (attempt optimizations that help on some NUMA systems):
To use it in numactl mode you will need the package
## Unix
- Fedora linux package:
```bash
sudo dnf install numactl
```
- Debian or Ubuntu linux package:
```bash
sudo apt install numactl
```
### Obtaining and using the Facebook LLaMA 2 model
- Refer to [Facebook's LLaMA download page](https://ai.meta.com/resources/models-and-libraries/llama-downloads/) if you want to access the model data.