vbatts/llama.cpp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

update readme

Browse source
This commit is contained in:
Concedo 2023-04-18 21:39:05 +08:00
parent ea01771dd5
commit 3614956bc7
1 changed files with 7 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

7
README.md
View file

@ -8,6 +8,7 @@ What does it mean? You get llama.cpp with a fancy UI, persistent stories, editin
# Highlights
- Now has experimental CLBlast support.
- Now supports RWKV models WITHOUT pytorch or tokenizers! Yep, just GGML!
## Usage
- [Download the latest release here](https://github.com/LostRuins/koboldcpp/releases/latest) or clone the repo.
@ -50,3 +51,9 @@ What does it mean? You get llama.cpp with a fancy UI, persistent stories, editin
## Notes
- Generation delay scales linearly with original prompt length. If OpenBLAS is enabled then prompt ingestion becomes about 2-3x faster. This is automatic on windows, but will require linking on OSX and Linux.
- I have heard of someone claiming a false AV positive report. The exe is a simple pyinstaller bundle that includes the necessary python scripts and dlls to run. If this still concerns you, you might wish to rebuild everything from source code using the makefile, and you can rebuild the exe yourself with pyinstaller by using `make_pyinstaller.bat`
- Supported GGML models:
- LLAMA (All versions including ggml, ggmf, ggjt, gpt4all)
- GPT-2 (All versions, including legacy f16, newer format + quanitzed, cerebras)
- GPT-J (All versions including legacy f16, newer format + quantized, pyg.cpp, new pygmalion, janeway etc.),
- RWKV (f16 GGMF format)
- Basically every single current and historical GGML format that has ever existed should be supported, except for bloomz.cpp due to lack of demand.