vbatts/llama.cpp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

llama : use ggml_backend_dev_get_extra_bufts

Browse source 
This commit updates the `name` argument for the
`ggml_backend_reg_get_proc_address` function call in
`make_cpu_buft_list` to be `ggml_backend_dev_get_extra_bufts`
instead of `ggml_backend_cpu_get_extra_bufts`.

The motivation for this change is that in
`ggml_backend_cpu_get_proc_address`, the `name` parameter is matched against
`ggml_backend_dev_get_extra_bufts` and not
`ggml_backend_cpu_get_extra_bufts`:
```c++
static void * ggml_backend_cpu_get_proc_address(ggml_backend_reg_t reg,
    const char * name) {
    if (strcmp(name, "ggml_backend_set_n_threads") == 0) {
        return (void *)ggml_backend_cpu_set_n_threads;
    }
    if (strcmp(name, "ggml_backend_dev_get_extra_bufts") == 0) {
        return (void *)ggml_backend_cpu_get_extra_bufts;
    }

    return NULL;

    GGML_UNUSED(reg);
}
```
This commit is contained in:
Daniel Bevenius 2024-11-09 05:27:10 +01:00
parent ec450d3bbf
commit a1a7ab0958
1 changed files with 1 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

2
src/llama.cpp
View file

@ -7189,7 +7189,7 @@ static llama_model::buft_list_t make_cpu_buft_list(llama_model & model) {
auto * cpu_dev = ggml_backend_dev_by_type(GGML_BACKEND_DEVICE_TYPE_CPU);
auto * cpu_reg = ggml_backend_dev_backend_reg(cpu_dev);
auto ggml_backend_dev_get_extra_bufts_fn = (ggml_backend_dev_get_extra_bufts_t)
ggml_backend_reg_get_proc_address(cpu_reg, "ggml_backend_cpu_get_extra_bufts");
ggml_backend_reg_get_proc_address(cpu_reg, "ggml_backend_dev_get_extra_bufts");
if (ggml_backend_dev_get_extra_bufts_fn) {
ggml_backend_buffer_type_t * extra_bufts = ggml_backend_dev_get_extra_bufts_fn(cpu_dev);
while (extra_bufts && *extra_bufts) {