vbatts/llama.cpp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

[SYCL] fix the mul_mat_id ut issues (#8427)

Browse source 
* fix part of mul_mat_id

* skip the bfloat 16 sycl ut

Signed-off-by: Chen Xi <xi2chen@intel.com>

---------

Signed-off-by: Chen Xi <xi2chen@intel.com>
Co-authored-by: Meng, Hengyu <hengyu.meng@intel.com>
Co-authored-by: Chen Xi <xi2chen@intel.com>
This commit is contained in:
Chen Xi 2024-07-12 00:52:04 +00:00 • committed by GitHub
parent 368645698a
commit b549a1bbef
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
3 changed files with 14 additions and 44 deletions
Download patch file Download diff file Expand all files Collapse all files

7
src/llama.cpp
View file

@ -5883,13 +5883,6 @@ static bool llm_load_tensors(
auto & hparams = model.hparams;
#ifdef GGML_USE_SYCL
// disable MoE with SYCL until mul_mat_id is updated
if (hparams.n_expert > 0) {
n_gpu_layers = 0;
}
#endif
model.split_mode = split_mode;
model.main_gpu = main_gpu;
model.n_gpu_layers = n_gpu_layers;