vbatts/llama.cpp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

fix bug in llama_sample_token_mirostat_v2

Browse source 
when all candidates are filtered out through mu threshold, the following soft_max operation will fail.
so keep at least one.
This commit is contained in:
xaedes 2023-05-21 21:12:10 +02:00
parent ec1783c3e0
commit 2afd218479
No known key found for this signature in database
GPG key ID: 30030EDD817EA2B1
1 changed files with 4 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

4
llama.cpp
View file

@ -1882,6 +1882,10 @@ llama_token llama_sample_token_mirostat_v2(struct llama_context * ctx, llama_tok
return -log2f(candidate.p) > *mu;
}));
if (candidates->size == 0) {
candidates->size = 1;
}
// Normalize the probabilities of the remaining words
llama_sample_softmax(ctx, candidates);