vbatts/llama.cpp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

cann: Add host buffer type for Ascend NPU (#9406)

Browse source 
* feat: Add host buffer type for Ascend NPU(CANN backend)

* fix some checking errors

* Add a few comments
This commit is contained in:
Dou Xinpeng 2024-09-12 19:46:43 +08:00 • committed by GitHub
parent e665744317
commit e6b7801bd1
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
3 changed files with 121 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

4
src/llama.cpp
View file

@ -2156,6 +2156,10 @@ static ggml_backend_buffer_type_t llama_default_buffer_type_cpu(bool host_buffer
if (host_buffer) {
buft = ggml_backend_sycl_host_buffer_type();
}
#elif defined(GGML_USE_CANN)
if (host_buffer) {
buft = ggml_backend_cann_host_buffer_type();
}
#elif defined(GGML_USE_CPU_HBM)
buft = ggml_backend_cpu_hbm_buffer_type();
#elif defined(GGML_USE_VULKAN)