vbatts/llama.cpp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

ggml: support CUDA's half type for aarch64(#1455)

Browse source 
support CUDA's half type for aarch64 in ggml_fp16_t definition
This commit is contained in:
Kylin 2023-08-20 00:07:50 +08:00
parent 1f0bccb279
commit 8a25bd41b3
1 changed files with 3 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

4
ggml.h
View file

@ -255,9 +255,11 @@
extern "C" {
#endif
#ifdef __ARM_NEON
#if defined(__ARM_NEON) && !defined(GGML_CUDA_F16)
// we use the built-in 16-bit float type
typedef __fp16 ggml_fp16_t;
#elif defined(GGML_CUDA_F16)
typedef half ggml_fp16_t;
#else
typedef uint16_t ggml_fp16_t;
#endif