vbatts/llama.cpp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

server: health: fix race condition on slots data using tasks queue (#5634)

Browse source 
* server: health: fix race condition on slots data using tasks queue

* server: health:
    * include_slots only if slots_endpoint
    * fix compile warning task.target_id not initialized.
This commit is contained in:
Pierrick Hymbert 2024-02-21 15:47:48 +01:00 • committed by GitHub
parent a00a35cef9
commit 1ecea255eb
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
3 changed files with 84 additions and 43 deletions
Download patch file Download diff file Expand all files Collapse all files

2
examples/server/README.md
View file

@ -140,6 +140,8 @@ node index.js
- 200 -> `{"status": "no slot available", "slots_idle": 0, "slots_processing": 32}` if no slot are currently available.
- 503 -> `{"status": "no slot available", "slots_idle": 0, "slots_processing": 32}` if the query parameter `fail_on_no_slot` is provided and no slot are currently available.
If the query parameter `include_slots` is passed, `slots` field will contain internal slots data except if `--slots-endpoint-disable` is set.
- **POST** `/completion`: Given a `prompt`, it returns the predicted completion.
*Options:*