llama.cpp : split llama_context_params into model and context params (#3301)

Browse source

* llama.cpp : split llama_context_params into model and context params

ggml-ci

* fix metal build

* fix freq_base/scale default to model value

* llama-bench : keep the same model between tests when possible

* move n_threads to llama_context_params, add n_threads_batch

* fix mpi build

* remove kv_size(), cuda scratch fixes

* remove low-vram option

* add n_threads_batch to system info, refactor to get_system_info()

* add documentation about --threads-batch to the READMEs

* llama-bench fix

* main : fix rope freq/scale warning

* llama.cpp : add llama_get_model
common : add llama_tokenize from model

* remove duplicated ctx/model functions

ggml-ci

* cuda : print total VRAM used

...

This commit is contained in:

slaren 2023-09-28 21:42:38 +02:00 • committed by GitHub

parent 0512d66670

commit 16bc66d947

No known key found for this signature in database

GPG key ID: 4AEE18F83AFDEB23

27 changed files with 713 additions and 633 deletions

Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL

Download patch file Download diff file Expand all files Collapse all files

545

llama.cpp

View file

File diff suppressed because it is too large Load diff