vbatts/llama.cpp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

convert_hf : simplify modify_tensors for InternLM2

Browse source 
* convert_lora : lazy conversion

* llama : load and use alpha from LoRA adapters
This commit is contained in:
Francis Couture-Harpin 2024-07-15 02:35:06 -04:00
parent 9d96328bdf
commit 8956543c09
4 changed files with 123 additions and 66 deletions
Download patch file Download diff file Expand all files Collapse all files

2
gguf-py/gguf/quants.py
View file

@ -43,7 +43,7 @@ def __apply_over_grouped_rows(func: Callable[[np.ndarray], np.ndarray], arr: np.
osize *= dim
out = np.empty(shape=osize, dtype=otype)
# compute over groups of 16 rows (arbitrary, but seems good for performance)
n_groups = rows.shape[0] // 16
n_groups = (rows.shape[0] // 16) or 1
np.concatenate([func(group).ravel() for group in np.array_split(rows, n_groups)], axis=0, out=out)
return out.reshape(oshape)