vbatts/llama.cpp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

CUDA: non-contiguous (RMS) norm support (#11659)

Browse source 
* CUDA: non-contiguous (RMS) norm support

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
This commit is contained in:
Johannes Gäßler 2025-02-04 22:21:42 +01:00 • committed by GitHub
parent 3ec9fd4b77
commit fd08255d0d
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
6 changed files with 97 additions and 47 deletions
Download patch file Download diff file Expand all files Collapse all files

2
ggml/src/ggml-vulkan/ggml-vulkan.cpp
View file

@ -8182,9 +8182,11 @@ static bool ggml_backend_vk_device_supports_op(ggml_backend_dev_t dev, const ggm
case GGML_OP_VIEW:
case GGML_OP_PERMUTE:
case GGML_OP_TRANSPOSE:
return true;
case GGML_OP_NORM:
case GGML_OP_GROUP_NORM:
case GGML_OP_RMS_NORM:
return ggml_is_contiguous(op->src[0]);
case GGML_OP_ADD:
case GGML_OP_ACC:
case GGML_OP_MUL: