vbatts/llama.cpp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

fix bug when using ggml_opt to optimize params in one context and use a renewable context for eval and opt

Browse source 
when not keeping gradients of model parameters they are overwritten by tensors created by opt, which may be invalid after opt context is renewed.
so we need to keep the original gradients and make dups for opt
This commit is contained in:
xaedes 2023-05-06 13:05:29 +02:00
parent bc1c13bb66
commit 83ee1cd741
No known key found for this signature in database
GPG key ID: 30030EDD817EA2B1
1 changed files with 1 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

2
ggml.c
View file

@ -15131,7 +15131,7 @@ enum ggml_opt_result ggml_opt(
// build forward + backward compute graphs
struct ggml_cgraph gf = ggml_build_forward (f);
struct ggml_cgraph gb = ggml_build_backward(ctx, &gf, false);
struct ggml_cgraph gb = ggml_build_backward(ctx, &gf, true);
switch (params.type) {
case GGML_OPT_ADAM: