vbatts/llama.cpp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Fix mirostatv2. (#338)

Browse source
This commit is contained in:
Ycros 2023-07-23 11:52:03 +10:00 • committed by GitHub
parent f036109110
commit 56995caa48
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
1 changed files with 5 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

5
gpttype_adapter.cpp
View file

@ -178,6 +178,11 @@ llama_token sample_token_mirostat_v2(llama_token_data_array * candidates, std::m
candidates->size = std::distance(candidates->data, std::find_if(candidates->data, candidates->data + candidates->size, [&](const llama_token_data & candidate) {
return -log2f(candidate.p) > *mu;
}));
if (candidates->size == 0) {
candidates->size = 1;
}
// Normalize the probabilities of the remaining words
llama_sample_softmax(nullptr, candidates);
// Sample the next word X from the remaining words