vbatts/llama.cpp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

metal : support FA without mask + add asserts (#7278)

Browse source 
* ggml : fa without mask + add asserts

ggml-ci

* metal : support non-contiguous KV

ggml-ci
This commit is contained in:
Georgi Gerganov 2024-05-14 19:09:30 +03:00 • committed by GitHub
parent a5e3fde857
commit e8a7fd4fb0
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
5 changed files with 85 additions and 75 deletions
Download patch file Download diff file Expand all files Collapse all files

3
ggml.h
View file

@ -766,7 +766,8 @@ extern "C" {
GGML_API bool ggml_is_3d (const struct ggml_tensor * tensor);
GGML_API int ggml_n_dims (const struct ggml_tensor * tensor); // returns 1 for scalars
GGML_API bool ggml_are_same_shape(const struct ggml_tensor * t0, const struct ggml_tensor * t1);
GGML_API bool ggml_are_same_shape (const struct ggml_tensor * t0, const struct ggml_tensor * t1);
GGML_API bool ggml_are_same_stride(const struct ggml_tensor * t0, const struct ggml_tensor * t1);
// use this to compute the memory overhead of a tensor
GGML_API size_t ggml_tensor_overhead(void);