vbatts/llama.cpp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

norm & rms_norm can not be threaded:

Browse source 
after investigation rms norm for quite some time I come to the conclusion that neither norm, nor rms_norm can be threaded, because we need mean over all items, not just of the slices each thread sees.
This commit is contained in:
xaedes 2023-04-24 04:13:33 +02:00
parent b164343529
commit b908007471
No known key found for this signature in database
GPG key ID: 30030EDD817EA2B1
1 changed files with 2 additions and 2 deletions
Download patch file Download diff file Expand all files Collapse all files

4
ggml.c
View file

@ -13288,8 +13288,8 @@ void ggml_graph_compute(struct ggml_context * ctx, struct ggml_cgraph * cgraph)
case GGML_OP_RMS_NORM: case GGML_OP_RMS_NORM:
case GGML_OP_RMS_NORM_BACK: case GGML_OP_RMS_NORM_BACK:
{ {
// i think this must not be threaded, because we need mean over all items, not just the slices each thread sees // i think this cannot be threaded, because we need mean over all items, not just the slices each thread sees.
node->n_tasks = n_threads; node->n_tasks = 1;
} break; } break;
case GGML_OP_MUL_MAT: case GGML_OP_MUL_MAT:
{ {