Add an API example using server.cpp similar to OAI. (#2009)
* add api_like_OAI.py * add evaluated token count to server * add /v1/ endpoints binding
This commit is contained in:
parent
7ee76e45af
commit
f257fd2550
3 changed files with 244 additions and 5 deletions
|
@ -190,3 +190,19 @@ Run with bash:
|
|||
```sh
|
||||
bash chat.sh
|
||||
```
|
||||
|
||||
### API like OAI
|
||||
|
||||
API example using Python Flask: [api_like_OAI.py](api_like_OAI.py)
|
||||
This example must be used with server.cpp
|
||||
|
||||
```sh
|
||||
python api_like_OAI.py
|
||||
```
|
||||
|
||||
After running the API server, you can use it in Python by setting the API base URL.
|
||||
```python
|
||||
openai.api_base = "http://<Your api-server IP>:port"
|
||||
```
|
||||
|
||||
Then you can utilize llama.cpp as an OpenAI's **chat.completion** or **text_completion** API
|
||||
|
|
Loading…
Add table
Add a link
Reference in a new issue