vbatts/llama.cpp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

update finetune README.md

Browse source
This commit is contained in:
xaedes 2023-09-22 20:54:00 +02:00
parent 758c46cf7d
commit 9145c87acc
No known key found for this signature in database
GPG key ID: 30030EDD817EA2B1
1 changed files with 3 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

4
examples/finetune/README.md
View file

@ -66,9 +66,11 @@ The scale numbers don't need to add up to one, and you can also use numbers crea
Gradient checkpointing reduces the memory requirements by ~50% but increases the runtime.
If you have enough RAM, you can make finetuning a bit faster by disabling checkpointing with `--no-checkpointing`.
The LORA rank is configured for each model tensor type separately with these command line options:
The default LORA rank can be specified with `--lora-r N`.
The LORA rank can be configured for each model tensor type separately with these command line options:
```bash
--lora-r N LORA r: default rank. Also specifies resulting scaling together with lora-alpha. (default 4)
--rank-att-norm N LORA rank for attention norm tensor (default 1)
--rank-ffn-norm N LORA rank for feed-forward norm tensor (default 1)
--rank-out-norm N LORA rank for output norm tensor (default 1)