vbatts/llama.cpp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Added colors to distinguish drafted tokens (--color). Updated README

Browse source
This commit is contained in:
Leon Ericsson 2023-12-17 13:04:46 +01:00
parent 45b8032b9c
commit 1b26d7151a
2 changed files with 30 additions and 11 deletions
Download patch file Download diff file Expand all files Collapse all files

13
examples/lookup/README.md
View file

@ -0,0 +1,13 @@
# llama.cpp/examples/lookup
Demonstration of Prompt Lookup Decoding
https://github.com/apoorvumang/prompt-lookup-decoding
The two key parameters for lookup decoding are `max_ngram_size` and `n_draft`. The first, determines how many ngrams to use when searching through the prompt for a match and the second specifies how many subsequent tokens to draft if a match is found.
More info:
https://github.com/ggerganov/llama.cpp/pull/4484
https://github.com/ggerganov/llama.cpp/issues/4226