vbatts/llama.cpp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Main+: optionally allow special tokens from user in interactive mode (#7097)

Browse source 
@hanishkvc added a new `--interactive-specials` flag which would allow for inserting special tokens from user side into the embedding stream.
This commit is contained in:
HanishKVC 2024-05-10 15:51:58 +05:30 • committed by GitHub
parent d11afd6652
commit f89fe2732c
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
3 changed files with 8 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

1
common/common.h
View file

@ -140,6 +140,7 @@ struct gpt_params {
bool random_prompt = false; // do not randomize prompt if none provided
bool use_color = false; // use color to distinguish generations and inputs
bool interactive = false; // interactive mode
bool interactive_specials = false; // whether to allow special tokens from user, during interactive mode
bool conversation = false; // conversation mode (does not print special tokens and suffix/prefix)
bool chatml = false; // chatml mode (used for models trained on chatml syntax)
bool prompt_cache_all = false; // save user input and generations to prompt cache