vbatts/llama.cpp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
llama.cpp/examples/lookup
History
Exact
Georgi Gerganov f86b9d152c
lookup : minor
2023-12-17 17:25:28 +02:00
..
CMakeLists.txt initial commit, going through initializations 2023-12-04 21:52:17 +01:00
lookup.cpp lookup : minor 2023-12-17 17:25:28 +02:00
README.md Added colors to distinguish drafted tokens (--color). Updated README 2023-12-17 13:04:46 +01:00

README.md

llama.cpp/examples/lookup

Demonstration of Prompt Lookup Decoding

https://github.com/apoorvumang/prompt-lookup-decoding

The two key parameters for lookup decoding are max_ngram_size and n_draft. The first, determines how many ngrams to use when searching through the prompt for a match and the second specifies how many subsequent tokens to draft if a match is found.

More info:

https://github.com/ggerganov/llama.cpp/pull/4484 https://github.com/ggerganov/llama.cpp/issues/4226