From 6c885dce8bb1370a67f33591b43a8a54806e8250 Mon Sep 17 00:00:00 2001 From: ochafik Date: Mon, 8 Apr 2024 23:35:28 +0100 Subject: [PATCH] server+json: update server/README w/ result_format --- examples/server/README.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/examples/server/README.md b/examples/server/README.md index a6fc92eaa..ae313120a 100644 --- a/examples/server/README.md +++ b/examples/server/README.md @@ -11,6 +11,7 @@ Set of LLM REST APIs and a simple web front end to interact with llama.cpp. * Continuous batching * Multimodal (wip) * Monitoring endpoints + * Schema-constrained JSON response format The project is under active development, and we are [looking for feedback and contributors](https://github.com/ggerganov/llama.cpp/issues/4216). @@ -250,6 +251,8 @@ node index.js `grammar`: Set grammar for grammar-based sampling. Default: no grammar + `response_format`: Set the response format. Only supports JSON (e.g. `{"type": "json_object"}`), optionally with a schema (e.g. `{"type": "json_object", "schema": {"type": "string", "minLength": 10, "maxLength": 100}}`). See [test-json-schema-to-grammar.cpp](../../tests/test-json-schema-to-grammar.cpp). Default: no response format. + `seed`: Set the random number generator (RNG) seed. Default: `-1`, which is a random seed. `ignore_eos`: Ignore end of stream token and continue generating. Default: `false`