n_probs should be view only

min_keep was being set to n_probs, which was then impacting sampler results
This commit is contained in:
Robey Holderith 2024-02-18 08:49:09 -08:00 committed by GitHub
parent e75c6279d1
commit 20906de171
No known key found for this signature in database
GPG key ID: B5690EEEBB952194

View file

@ -249,7 +249,7 @@ static llama_token llama_sampling_sample_impl(
id = llama_sample_token_mirostat_v2(ctx_main, &cur_p, mirostat_tau, mirostat_eta, &ctx_sampling->mirostat_mu);
} else {
// temperature sampling
size_t min_keep = std::max(1, params.n_probs);
size_t min_keep = 1;
sampler_queue(ctx_main, params, cur_p, min_keep);