vbatts/llama.cpp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

readme : add feature matrix

Browse source
This commit is contained in:
Romain “Artefact2” Dal Maso 2024-02-27 19:36:13 +01:00
parent cb49e0f8c9
commit 14bf8965c2
No known key found for this signature in database
GPG key ID: 19A9512F7A21425A
1 changed files with 10 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

10
README.md
View file

@ -161,6 +161,16 @@ Unless otherwise noted these projects are open-source with permissive licensing:
- [Msty](https://msty.app) (proprietary)
- [LLMFarm](https://github.com/guinmoon/LLMFarm?tab=readme-ov-file) (MIT)
**Feature matrix**
| | **CPU (AVX2)** | **CPU (ARM NEON)** | **Metal** | **cuBLAS** | **rocBLAS** | **SYCL** | **CLBlast** | **Vulkan** | **Kompute** |
|:--------------------:|:--------------:|--------------------|:---------:|:----------:|:----------------:|----------|:-----------:|:----------:|:-----------:|
| **K-quants** | ✅ | ✅ | ✅ | ✅ | ✅ | ❓ | ✅ | ✅ | 🚫 |
| **I-quants** | ✅ (SLOW) | ✅ | ✅ (SLOW) | ✅ | ✅ | 🚫 | 🚫 | 🚫 | 🚫 |
| **Multi-GPU** | N/A | N/A | N/A | ✅ | ❓ | 🚫 | ❓ | ✅ | ❓ |
| **K cache quants** | ✅ | ❓ | ❓ | ✅ | Only q8_0 (SLOW) | ❓ | ✅ | 🚫 | 🚫 |
| **MoE architecture** | ✅ | ❓ | ✅ | ✅ | ✅ | ❓ | Only -ngl 0 | 🚫 | 🚫 |
---
Here is a typical run using LLaMA v2 13B on M2 Ultra: