vbatts/llama.cpp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Update common/common.h (suggested by JohannesGaessler)

Browse source 
Co-authored-by: Johannes Gäßler <johannesg@5d6.de>
This commit is contained in:
Yui 2024-04-08 09:18:00 +02:00 • committed by GitHub
parent 2325ec0550
commit a6120326b2
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
1 changed files with 1 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

2
common/common.h
View file

@ -62,7 +62,7 @@ struct gpt_params {
int32_t n_parallel = 1; // number of parallel sequences to decode
int32_t n_sequences = 1; // number of sequences to decode
float p_split = 0.1f; // speculative decoding split probability
int32_t n_gpu_layers = -1; // number of layers to store in VRAM (-1 - use default, -2 - automatically determine)
int32_t n_gpu_layers = -1; // number of layers to store in VRAM (-1 - use default, -2 - determine automatically)
int32_t n_gpu_layers_draft = -1; // number of layers to store in VRAM for the draft model (-1 - use default)
llama_split_mode split_mode = LLAMA_SPLIT_MODE_LAYER; // how to split the model across GPUs
int32_t main_gpu = 0; // the GPU that is used for scratch and small tensors