vbatts/llama.cpp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

reduce large memory overhead in train-text-from-scratch

Browse source 
all gradients had to be pinned so that graph_reset works correctly.
this is no longer necessary with the changes to ggml_compute_backward introduced in this PR.
This commit is contained in:
xaedes 2023-08-29 00:56:44 +02:00
parent 271c0300de
commit 9a28bce29a
No known key found for this signature in database
GPG key ID: 30030EDD817EA2B1
1 changed files with 1 additions and 9 deletions
Download patch file Download diff file Expand all files Collapse all files

10
examples/train-text-from-scratch/train-text-from-scratch.cpp
View file

@ -793,15 +793,7 @@ struct ggml_tensor * llama_build_train_graphs(
ggml_build_forward_expand(gb, ggml_scale_inplace(ctx, t36->grad, one));
GGML_ASSERT(t36->grad->data == NULL && !ggml_is_view(t36->grad));
ggml_allocr_alloc(alloc, t36->grad);
// gradient tensors (will be set to zero by ggml_graph_reset)
// pinning these produces large unnecessary memory overhead, which will be resolved by PR 2632
for (int i = 0; i < gf->n_nodes; ++i) {
if (!gf->grads[i]) continue;
if (gf->grads[i]->data == NULL && !ggml_is_view(gf->grads[i])) {
ggml_allocr_alloc(alloc, gf->grads[i]);
}
ggml_build_forward_expand(gb, ggml_scale_inplace(ctx, gf->grads[i], one));
}
// allocating checkpoints in one block to reduce memory fragmentation
// note: they will be freed in reverse order
for (int i = 0; i < (int) checkpoints.size(); ++i) {