vbatts/llama.cpp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Add memory/disk requirements to readme

Browse source
This commit is contained in:
Pavol Rusnak 2023-03-18 21:58:46 +01:00
parent d3f202d57b
commit 5d83a294d1
No known key found for this signature in database
GPG key ID: 91F3B339B9A02A3D
1 changed files with 11 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

12
README.md
View file

@ -155,7 +155,17 @@ python3 convert-pth-to-ggml.py models/7B/ 1
When running the larger models, make sure you have enough disk space to store all the intermediate files.
TODO: add model disk/mem requirements
### Memory/Disk Requirements
As the models are currently fully loaded into memory, you will need adequate disk space to save them
and sufficient RAM to load them. At the moment, memory and disk requirements are the same.
| model | original size | quantized size (4-bit) |
|-------|---------------|------------------------|
| 7B | 13 GB | 3.9 GB |
| 15B | 24 GB | 7.8 GB |
| 30B | 60 GB | 19.5 GB |
| 65B | 120 GB | 38.5 GB |
### Interactive mode