Updated readme
This commit is contained in:
parent
5f142df76e
commit
4e95e7f87f
2 changed files with 6 additions and 5 deletions
11
README.md
11
README.md
|
@ -1,10 +1,8 @@
|
|||
# llama-for-kobold
|
||||
|
||||
A hacky little script from Concedo that exposes llama.cpp function bindings, allowing it to be used via a simulated Kobold API endpoint.
|
||||
A self contained distributable from Concedo that exposes llama.cpp function bindings, allowing it to be used via a simulated Kobold API endpoint.
|
||||
|
||||
It's not very usable as there is a fundamental flaw with llama.cpp, which causes generation delay to scale linearly with original prompt length. Nobody knows why or really cares much, so I'm just going to publish whatever I have at this point.
|
||||
|
||||
If you care, **please contribute to [this discussion](https://github.com/ggerganov/llama.cpp/discussions/229)** which, if resolved, will actually make this viable.
|
||||

|
||||
|
||||
## Considerations
|
||||
- Don't want to use pybind11 due to dependencies on MSVCC
|
||||
|
@ -22,4 +20,7 @@ If you care, **please contribute to [this discussion](https://github.com/ggergan
|
|||
## License
|
||||
- The original GGML library and llama.cpp by ggerganov are licensed under the MIT License
|
||||
- However, Kobold Lite is licensed under the AGPL v3.0 License
|
||||
- The provided python ctypes bindings in llamacpp.dll are also under the AGPL v3.0 License
|
||||
- The provided python ctypes bindings in llamacpp.dll are also under the AGPL v3.0 License
|
||||
|
||||
## Notes
|
||||
- There is a fundamental flaw with llama.cpp, which causes generation delay to scale linearly with original prompt length. If you care, **please contribute to [this discussion](https://github.com/ggerganov/llama.cpp/discussions/229)** which, if resolved, will actually make this viable.
|
BIN
preview.png
Normal file
BIN
preview.png
Normal file
Binary file not shown.
After Width: | Height: | Size: 92 KiB |
Loading…
Add table
Add a link
Reference in a new issue