vbatts/llama.cpp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

ggml : improve graph build time via hash table lookup (#2329)

Browse source 
* improve graph build time

* ggml_tensor : use 1 bit per flag

* use a hash table instead
This commit is contained in:
slaren 2023-07-25 14:32:20 +02:00 • committed by GitHub
parent 82552b7f54
commit da1889834a
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
3 changed files with 42 additions and 12 deletions
Download patch file Download diff file Expand all files Collapse all files

2
llama.cpp
View file

@ -1714,6 +1714,8 @@ static bool llama_eval_internal(
// run the computation
ggml_build_forward_expand(&gf, cur);
// fprintf(stderr, "graph build time: %.3f ms (%d nodes, %d leafs)\n", (ggml_time_us() - t_start_us)/1000.0, gf.n_nodes, gf.n_leafs);
#if GGML_USE_MPI
ggml_mpi_graph_compute_pre(lctx.ctx_mpi, &gf, n_layer);
#endif