vbatts/llama.cpp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Improve interactive mode's coherence after EOS

Browse source 
Aims to improve coherence and ability to resume the interactive session when the user is given input back after an end of text token is reached.
Not sure what token 13 is or why it seems to help. See conversation for examples.
This commit is contained in:
rabidcopy 2023-03-20 11:33:50 -05:00 • committed by GitHub
parent 074bea2eb1
commit 4fe9734e09
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
1 changed files with 2 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

2
main.cpp
View file

@ -1084,6 +1084,8 @@ int main(int argc, char ** argv) {
if (embd.back() == EOS_TOKEN_ID) { if (embd.back() == EOS_TOKEN_ID) {
if (params.interactive) { if (params.interactive) {
is_interacting = true; is_interacting = true;
embd.back() = 13;
last_n_tokens.back() = 13;
} else { } else {
fprintf(stderr, " [end of text]\n"); fprintf(stderr, " [end of text]\n");
break; break;