vbatts/llama.cpp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Add details on dataset/context length

Browse source
This commit is contained in:
Gary Linscott 2023-03-22 08:48:36 -07:00 • committed by GitHub
parent 9d9e152b6d
commit c65eff0d14
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
1 changed files with 3 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

3
README.md
View file

@ -250,6 +250,9 @@ see https://huggingface.co/docs/transformers/perplexity. However, in general, l
https://github.com/ggerganov/llama.cpp/pull/270 is the unofficial tracking page for now. llama.cpp is measuring very well https://github.com/ggerganov/llama.cpp/pull/270 is the unofficial tracking page for now. llama.cpp is measuring very well
compared to the baseline implementations. Quantization has a small negative impact to quality, but, as you can see, running compared to the baseline implementations. Quantization has a small negative impact to quality, but, as you can see, running
13B at q4_0 beats the 7B f16 model by a significant amount. 13B at q4_0 beats the 7B f16 model by a significant amount.
All measurements are done against wikitext2 test dataset (https://paperswithcode.com/dataset/wikitext-2), with default options (512 length context).
Note that the changing the context length will have a significant impact on perplexity (longer context = better perplexity).
``` ```
Perplexity - model options Perplexity - model options
5.5985 - 13B, q4_0 5.5985 - 13B, q4_0