vbatts/llama.cpp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Add comment

Browse source 
Signed-off-by: mudler <mudler@localai.io>
This commit is contained in:
mudler 2023-06-19 18:43:27 +02:00
parent 7a45a13e3d
commit 7de2494a0f
1 changed files with 1 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

1
llama.h
View file

@ -83,6 +83,7 @@ extern "C" {
// context pointer passed to the progress callback // context pointer passed to the progress callback
void * progress_callback_user_data; void * progress_callback_user_data;
// Keep the booleans together to avoid misalignment during copy-by-value.
bool low_vram; // if true, reduce VRAM usage at the cost of performance bool low_vram; // if true, reduce VRAM usage at the cost of performance
bool f16_kv; // use fp16 for KV cache bool f16_kv; // use fp16 for KV cache
bool logits_all; // the llama_eval() call computes all logits, not just the last one bool logits_all; // the llama_eval() call computes all logits, not just the last one