vbatts/llama.cpp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Update server README.md

Browse source
This commit is contained in:
Burhanuddin Mustafa Lakdawala 2024-10-25 12:52:46 -07:00 • committed by GitHub
parent 668750357e
commit 006167dd65
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
1 changed files with 10 additions and 2 deletions
Download patch file Download diff file Expand all files Collapse all files

10
examples/server/README.md
View file

@ -762,7 +762,15 @@ Example:
Possible values for `slot[i].state` are: Possible values for `slot[i].state` are:
- `0`: SLOT_STATE_IDLE - `0`: SLOT_STATE_IDLE
- `1`: SLOT_STATE_PROCESSING The slot is idle and ready to use.
- `1`: SLOT_STATE_PROCESSING_PROMPT
The slot is processing the input prompt tokens.
- `2`: SLOT_STATE_DONE_PROMPT
The slot has finished processing the input prompt. For embedding and rerank tasks the slot will be released soon, otherwise the slot will be used for generation.
- `3`: SLOT_STATE_GENERATING
The slot is generating output tokens.
[State diagram](https://github.com/ggerganov/llama.cpp/pull/9283)
### GET `/metrics`: Prometheus compatible metrics exporter ### GET `/metrics`: Prometheus compatible metrics exporter