vbatts/llama.cpp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

server: tests - passkey - first good working value of nga

Browse source
This commit is contained in:
Pierrick HYMBERT 2024-03-02 14:31:27 +01:00
parent ed60b97434
commit cf4c86ee20
1 changed files with 2 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

3
examples/server/tests/features/passkey.feature
View file

@ -16,6 +16,7 @@ Feature: Passkey / Self-extend with context shift
Given a model file <hf_file> from HF repo <hf_repo> Given a model file <hf_file> from HF repo <hf_repo>
And <n_batch> as batch size And <n_batch> as batch size
And <n_junk> as number of junk And <n_junk> as number of junk
And <n_predicted> server max tokens to predict
And a self-extend context with a factor of <n_grp> And a self-extend context with a factor of <n_grp>
And <seed> as seed And <seed> as seed
And a KV cache size based on the model trained context <n_ctx_train> extended by <n_grp> with additional <n_keep> tokens And a KV cache size based on the model trained context <n_ctx_train> extended by <n_grp> with additional <n_keep> tokens
@ -50,4 +51,4 @@ Feature: Passkey / Self-extend with context shift
Examples: Examples:
| hf_repo | hf_file | n_ctx_train | ngl | n_batch | n_slots | n_ga | n_ga_w | n_junk | n_grp | i_pos | seed | n_keep | passkey | n_predicted | re_content | | hf_repo | hf_file | n_ctx_train | ngl | n_batch | n_slots | n_ga | n_ga_w | n_junk | n_grp | i_pos | seed | n_keep | passkey | n_predicted | re_content |
| TheBloke/phi-2-GGUF | phi-2.Q4_K_M.gguf | 2048 | 5 | 512 | 1 | 4 | 2048 | 250 | 4 | 50 | 86 | 32 | 42 | -1 | .*42.* | | TheBloke/phi-2-GGUF | phi-2.Q4_K_M.gguf | 2048 | 5 | 512 | 1 | 8 | 512 | 250 | 4 | 50 | 86 | 32 | 42 | -1 | .*42.* |