vbatts/llama.cpp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

feat: implemented sigmoid function (ggml/806)

Browse source 
* added sigmoid function

* implemented metal kernel for sigmoid

* implemented cuda kernel for sigmoid

* added sigmoid unary op and incremented count
This commit is contained in:
Justina Cho 2024-05-01 14:44:26 -07:00 • committed by Georgi Gerganov
parent ef0d5e3ec9
commit f5ef34e428
7 changed files with 136 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

7
ggml-metal.metal
View file

@ -229,6 +229,13 @@ kernel void kernel_relu(
dst[tpig] = max(0.0f, src0[tpig]);
}
kernel void kernel_sigmoid(
device const float * src0,
device float * dst,
uint tpig[[thread_position_in_grid]]) {
dst[tpig] = 1.0f / (1.0f + exp(-src0[tpig]));
}
kernel void kernel_tanh(
device const float * src0,
device float * dst,