vbatts/llama.cpp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

server: fix incorrectly reported token probabilities (#7125)

Browse source 
* server: normalize token probabilities

* fix temperature == 0.0f
This commit is contained in:
Johannes Gäßler 2024-05-07 23:07:58 +02:00 • committed by GitHub
parent b6aa670203
commit af0a5b6163
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
4 changed files with 31 additions and 11 deletions
Download patch file Download diff file Expand all files Collapse all files

1
common/sampling.h
View file

@ -81,6 +81,7 @@ struct llama_sampling_context {
// TODO: replace with ring-buffer
std::vector<llama_token> prev;
std::vector<llama_token_data> cur;
size_t n_considered;
std::mt19937 rng;
};