vbatts/llama.cpp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

SWAP info added

Browse source
This commit is contained in:
Jaime R 2023-03-30 21:25:21 +03:00 • committed by GitHub
parent 3bcc129ba8
commit 85e8395944
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
1 changed files with 2 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

2
README.md
View file

@ -169,6 +169,8 @@ When running the larger models, make sure you have enough disk space to store al
As the models are currently fully loaded into memory, you will need adequate disk space to save them
and sufficient RAM to load them. At the moment, memory and disk requirements are the same.
We can use [SWAP](https://github.com/ggerganov/llama.cpp/discussions/631) to extend the virtual memory beyond the installed physical memory (RAM), to be able to use larger models if we don't have enought RAM.
| model | original size | quantized size (4-bit) |
|-------|---------------|------------------------|
| 7B | 13 GB | 3.9 GB |