vbatts/llama.cpp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

add /apply-template documentation

Browse source
This commit is contained in:
Nigel Bosch 2025-01-29 08:28:52 -06:00
parent 10448bf934
commit 453d204d8a
1 changed files with 8 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

8
examples/server/README.md
View file

@ -572,6 +572,14 @@ With input 'á' (utf8 hex: C3 A1) on tinyllama/stories260k
`tokens`: Set the tokens to detokenize.
### POST `/apply-template`: Apply chat template to a conversation
Uses the server's prompt template formatting functionality to convert chat messages to a single string expected by a chat model as input, but does not perform inference. Instead, the prompt string is returned in the `prompt` field of the JSON response. The prompt can then be modified as desired (for example, to insert "Sure!" at the beginning of the model's response) before sending to `/completion` to generate the chat response.
*Options:*
`messages`: (Required) Chat turns in the same format as `/v1/chat/completions`.
### POST `/embedding`: Generate embedding of a given text
> [!IMPORTANT]