vbatts/llama.cpp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

common, server : surface min_keep as its own parameter (#5567)

Browse source 
* Feature - surface min_keep as its own parameter

* Updated README with min_keep param
This commit is contained in:
Robey Holderith 2024-02-18 11:11:16 -08:00 • committed by GitHub
parent c145f8a132
commit 5ee99c32f5
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
6 changed files with 14 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

2
examples/server/README.md
View file

@ -199,6 +199,8 @@ node index.js
`n_probs`: If greater than 0, the response also contains the probabilities of top N tokens for each generated token (default: 0)
`min_keep`: If greater than 0, force samplers to return N possible tokens at minimum (default: 0)
`image_data`: An array of objects to hold base64-encoded image `data` and its `id`s to be reference in `prompt`. You can determine the place of the image in the prompt as in the following: `USER:[img-12]Describe the image in detail.\nASSISTANT:`. In this case, `[img-12]` will be replaced by the embeddings of the image with id `12` in the following `image_data` array: `{..., "image_data": [{"data": "<BASE64_STRING>", "id": 12}]}`. Use `image_data` only with multimodal models, e.g., LLaVA.
`slot_id`: Assign the completion task to an specific slot. If is -1 the task will be assigned to a Idle slot (default: -1)