vbatts/llama.cpp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

use mat*vec shaders for mat*mat

Browse source 
I wrote the mat*mat shaders from scratch so I understand them better but
they are currently not faster than just multiply-invoking the mat*vec
shaders, by a significant degree - so, except for f32 which needed a new
shader, revert to the m*v ones here.
This commit is contained in:
Aaron Miller 2023-10-16 10:00:25 -07:00 • committed by cebtenzzre
parent c1fd64548d
commit cc05a602d6
7 changed files with 27 additions and 677 deletions
Download patch file Download diff file Expand all files Collapse all files

10
CMakeLists.txt
View file

@ -479,12 +479,7 @@ if (LLAMA_KOMPUTE)
kompute/op_norm.comp
kompute/op_rmsnorm.comp
kompute/op_diagmask.comp
kompute/op_mul_mat_mat_f16.comp
kompute/op_mul_mat_mat_f32.comp
kompute/op_mul_mat_mat_q4_0.comp
kompute/op_mul_mat_mat_q4_1.comp
kompute/op_mul_mat_mat_q8_0.comp
kompute/op_mul_mat_mat_q6_k.comp
kompute/op_mul_mat_f16.comp
kompute/op_mul_mat_q8_0.comp
kompute/op_mul_mat_q4_0.comp
@ -515,12 +510,7 @@ if (LLAMA_KOMPUTE)
shaderop_norm.h
shaderop_rmsnorm.h
shaderop_diagmask.h
shaderop_mul_mat_mat_f16.h
shaderop_mul_mat_mat_f32.h
shaderop_mul_mat_mat_q4_0.h
shaderop_mul_mat_mat_q4_1.h
shaderop_mul_mat_mat_q8_0.h
shaderop_mul_mat_mat_q6_k.h
shaderop_mul_mat_f16.h
shaderop_mul_mat_q8_0.h
shaderop_mul_mat_q4_0.h