vbatts/llama.cpp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Main+: optionally allow special tokens from user in interactive mode (#7097)

Browse source 
@hanishkvc added a new `--interactive-specials` flag which would allow for inserting special tokens from user side into the embedding stream.
This commit is contained in:
HanishKVC 2024-05-10 15:51:58 +05:30 • committed by GitHub
parent d11afd6652
commit f89fe2732c
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
3 changed files with 8 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

2
examples/main/main.cpp
View file

@ -879,7 +879,7 @@ int main(int argc, char ** argv) {
}
const auto line_pfx = ::llama_tokenize(ctx, params.input_prefix, false, true);
const auto line_inp = ::llama_tokenize(ctx, buffer, false, false);
const auto line_inp = ::llama_tokenize(ctx, buffer, false, params.interactive_specials);
const auto line_sfx = ::llama_tokenize(ctx, params.input_suffix, false, true);
LOG("input tokens: %s\n", LOG_TOKENS_TOSTR_PRETTY(ctx, line_inp).c_str());