From 453d204d8aaf97fcfd6fb6d641a9409c61e3f06f Mon Sep 17 00:00:00 2001 From: Nigel Bosch Date: Wed, 29 Jan 2025 08:28:52 -0600 Subject: [PATCH] add /apply-template documentation --- examples/server/README.md | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/examples/server/README.md b/examples/server/README.md index 5022de672..eef328ddc 100644 --- a/examples/server/README.md +++ b/examples/server/README.md @@ -572,6 +572,14 @@ With input 'รก' (utf8 hex: C3 A1) on tinyllama/stories260k `tokens`: Set the tokens to detokenize. +### POST `/apply-template`: Apply chat template to a conversation + +Uses the server's prompt template formatting functionality to convert chat messages to a single string expected by a chat model as input, but does not perform inference. Instead, the prompt string is returned in the `prompt` field of the JSON response. The prompt can then be modified as desired (for example, to insert "Sure!" at the beginning of the model's response) before sending to `/completion` to generate the chat response. + +*Options:* + +`messages`: (Required) Chat turns in the same format as `/v1/chat/completions`. + ### POST `/embedding`: Generate embedding of a given text > [!IMPORTANT]