3 commits

Author	SHA1	Message	Date
Georgi Gerganov	360a333145	common : add llama_batch_add() and llama_batch_clear() helpers	2023-10-16 12:41:33 +03:00
Georgi Gerganov	4de5a2d473	speculative : add tree-based sampling support ... ggml-ci	2023-10-14 17:54:02 +03:00
Georgi Gerganov	8c70a5ff25	batched : add bench tool (#3545) ... * batched : add bench tool * batched : minor fix table * batched-bench : add readme + n_kv_max is now configurable * batched-bench : init warm-up batch * batched-bench : pass custom set of PP, TG and PL * batched-bench : add mmq CLI arg	2023-10-11 21:25:33 +03:00