Vulkan: VK_KHR_cooperative_matrix support to speed up prompt processing (#10597)

Browse source

* Vulkan: Implement VK_KHR_cooperative_matrix support in the matrix matrix multiplication shader

* Improve performance with better q4_k and q5_k dequant and store unrolling

* Add Vulkan MUL_MAT and MUL_MAT_ID accumulator precision selection

* Rework mulmat shader selection and compilation logic, avoid compiling shaders that won't get used by device

* Vulkan: Implement accumulator switch for specific mul mat mat shaders

* Vulkan: Unroll more loops for more mul mat mat performance

* Vulkan: Add VK_AMD_shader_core_properties2 support to read Compute Unit count for split_k logic

* Disable coopmat support on AMD proprietary driver

* Remove redundant checks

* Add environment variable GGML_VK_DISABLE_COOPMAT to disable VK_KHR_cooperative_matrix support

* Fix rebase typo

* Fix coopmat2 MUL_MAT_ID pipeline selection

...

This commit is contained in:

0cc4m 2024-12-07 10:24:15 +01:00 • committed by GitHub

parent 86a1934978

commit 3df784b305

No known key found for this signature in database

GPG key ID: B5690EEEBB952194

3 changed files with 739 additions and 386 deletions

Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL

Download patch file Download diff file Expand all files Collapse all files

823

ggml/src/ggml-vulkan/ggml-vulkan.cpp

View file

File diff suppressed because it is too large Load diff