examples : add passkey test (#3856)
* examples : add passkey test * passkey : better prints * passkey : select pass key pos from CLI * passkey : simplify n_past logic * make : add passkey target * passkey : add "self-extend"-like context extension (#4810) * llama : "self-extend"-like context extension * passkey : add comment * passkey : add readme
This commit is contained in:
parent
b7e7982953
commit
b0034d93ce
9 changed files with 361 additions and 1 deletions
|
@ -69,6 +69,7 @@ int main(int argc, char ** argv) {
|
|||
|
||||
std::vector<llama_token> tokens_list;
|
||||
tokens_list = ::llama_tokenize(model, params.prompt, true);
|
||||
|
||||
const int n_kv_req = tokens_list.size() + (n_len - tokens_list.size())*n_parallel;
|
||||
|
||||
// initialize the context
|
||||
|
|
Loading…
Add table
Add a link
Reference in a new issue