server: tests - add explanation about KV Cache.

2024-02-24 11:13:31 +01:00 · 2024-02-24 11:13:31 +01:00 · 60781f0a2b
commit 60781f0a2b
parent 482eb30f89
1 changed files with 3 additions and 0 deletions
--- a/examples/server/tests/features/server.feature
+++ b/examples/server/tests/features/server.feature
@ -6,6 +6,9 @@ Feature: llama.cpp server
    And   a model file stories260K.gguf
    And   a model alias tinyllama-2
    And   42 as server seed
+      # KV Cache corresponds to the total amount of tokens
+      # that can be stored across all independent sequences: #4130
+      # see --ctx-size and #5568
    And   32 KV cache size
    And   1 slots
    And   embeddings extraction