vbatts/llama.cpp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Updated readme

Browse source
This commit is contained in:
Concedo 2023-03-22 16:20:37 +08:00
parent 5f142df76e
commit 4e95e7f87f
2 changed files with 6 additions and 5 deletions
Download patch file Download diff file Expand all files Collapse all files

11
README.md
View file

@ -1,10 +1,8 @@
# llama-for-kobold # llama-for-kobold
A hacky little script from Concedo that exposes llama.cpp function bindings, allowing it to be used via a simulated Kobold API endpoint. A self contained distributable from Concedo that exposes llama.cpp function bindings, allowing it to be used via a simulated Kobold API endpoint.
It's not very usable as there is a fundamental flaw with llama.cpp, which causes generation delay to scale linearly with original prompt length. Nobody knows why or really cares much, so I'm just going to publish whatever I have at this point. ![Preview](preview.png)
If you care, **please contribute to [this discussion](https://github.com/ggerganov/llama.cpp/discussions/229)** which, if resolved, will actually make this viable.
## Considerations ## Considerations
- Don't want to use pybind11 due to dependencies on MSVCC - Don't want to use pybind11 due to dependencies on MSVCC
@ -22,4 +20,7 @@ If you care, **please contribute to [this discussion](https://github.com/ggergan
## License ## License
- The original GGML library and llama.cpp by ggerganov are licensed under the MIT License - The original GGML library and llama.cpp by ggerganov are licensed under the MIT License
- However, Kobold Lite is licensed under the AGPL v3.0 License - However, Kobold Lite is licensed under the AGPL v3.0 License
- The provided python ctypes bindings in llamacpp.dll are also under the AGPL v3.0 License - The provided python ctypes bindings in llamacpp.dll are also under the AGPL v3.0 License
## Notes
- There is a fundamental flaw with llama.cpp, which causes generation delay to scale linearly with original prompt length. If you care, **please contribute to [this discussion](https://github.com/ggerganov/llama.cpp/discussions/229)** which, if resolved, will actually make this viable.

BIN
preview.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 92 KiB