8cef75c743
This change upstreams llamafile's cpu matrix multiplication kernels for ppc64le using MMA builtins for quantised int8 datatype. This change results in 10% - 70% improvement in total speed(ie all tokens/total time), across various batch sizes. The patch is tested with Meta-Lllama-3-8B, Mistral-7B, Llama-2-7B-chat-hf models on a IBM POWER10 machine. Signed-off-by: Amrita H S <amritahs@linux.vnet.ibm.com> |
||
|---|---|---|
| .. | ||
| amx | ||
| cmake | ||
| llamafile | ||
| CMakeLists.txt | ||
| cpu-feats-x86.cpp | ||
| ggml-cpu-aarch64.cpp | ||
| ggml-cpu-aarch64.h | ||
| ggml-cpu-hbm.cpp | ||
| ggml-cpu-hbm.h | ||
| ggml-cpu-impl.h | ||
| ggml-cpu-quants.c | ||
| ggml-cpu-quants.h | ||
| ggml-cpu-traits.cpp | ||
| ggml-cpu-traits.h | ||
| ggml-cpu.c | ||
| ggml-cpu.cpp | ||