vbatts/llama.cpp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

common: ensure token addition to batch does not exceed llama_batch size

Browse source 
A crash was observed when the number of tokens added to a batch exceeds
llama_batch size. An assertion in llama_batch_add was added to protect
against llama_batch size overflow.
This commit is contained in:
Matias Lin 2024-09-27 10:03:33 -07:00
parent 95bc82fbc0
commit c197f0fcbd
1 changed files with 2 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

2
common/common.cpp
View file

@ -1432,6 +1432,8 @@ void llama_batch_add(
llama_pos pos,
const std::vector<llama_seq_id> & seq_ids,
bool logits) {
GGML_ASSERT(batch.seq_id[batch.n_tokens] && "llama_batch size exceeded");
batch.token [batch.n_tokens] = id;
batch.pos [batch.n_tokens] = pos;
batch.n_seq_id[batch.n_tokens] = seq_ids.size();