vbatts/llama.cpp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

CUDA: mul_mat_q RDNA2 tunings (#2910)

Browse source 
* CUDA: mul_mat_q RDNA2 tunings

* Update ggml-cuda.cu

Co-authored-by: Henri Vasserman <henv@hot.ee>

---------

Co-authored-by: Henri Vasserman <henv@hot.ee>
This commit is contained in:
Johannes Gäßler 2023-09-13 11:20:24 +02:00 • committed by GitHub
parent 84e723653c
commit 0a5eebb45d
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
3 changed files with 426 additions and 45 deletions
Download patch file Download diff file Expand all files Collapse all files

1
Makefile
View file

@ -408,7 +408,6 @@ ifdef LLAMA_HIPBLAS
HIPFLAGS += -DGGML_CUDA_DMMV_X=$(LLAMA_CUDA_DMMV_X)
HIPFLAGS += -DGGML_CUDA_MMV_Y=$(LLAMA_CUDA_MMV_Y)
HIPFLAGS += -DK_QUANTS_PER_ITERATION=$(LLAMA_CUDA_KQUANTS_ITER)
HIPFLAGS += -DCC_TURING=1000000000
ifdef LLAMA_CUDA_FORCE_DMMV
HIPFLAGS += -DGGML_CUDA_FORCE_DMMV
endif # LLAMA_CUDA_FORCE_DMMV