vbatts/llama.cpp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

use nullptr in llama_sample_softmax call during llama_sample_entropy

Browse source 
this avoids counting the time taken stats twice

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
This commit is contained in:
l3utterfly 2024-01-22 22:12:33 +09:00 • committed by GitHub
parent babb76a33a
commit a98a49836c
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
1 changed files with 1 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

2
llama.cpp
View file

@ -7786,7 +7786,7 @@ void llama_sample_typical(struct llama_context * ctx, llama_token_data_array * c
void llama_sample_entropy(struct llama_context * ctx, llama_token_data_array * candidates_p, float min_temp, float max_temp, float exponent_val) { void llama_sample_entropy(struct llama_context * ctx, llama_token_data_array * candidates_p, float min_temp, float max_temp, float exponent_val) {
const int64_t t_start_sample_us = ggml_time_us(); const int64_t t_start_sample_us = ggml_time_us();
llama_sample_softmax(ctx, candidates_p); llama_sample_softmax(nullptr, candidates_p);
// Calculate entropy of the softmax probabilities // Calculate entropy of the softmax probabilities
float entropy = 0.0f; float entropy = 0.0f;