vbatts/llama.cpp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

ggml : add Q8_0 quantization for intermediate results (#951)

Browse source 
* ggml : add Q8_0 quantization for intermediate results

* quantize-stats : fix test + add it to Makefile default

* Q8: use int8_t, AVX/AVX2 optimizations

* ggml : fix quantize_row_q8_0() ARM_NEON rounding

* minor : updates after rebase to latest master

* quantize-stats : delete obsolete strings

* ggml : fix q4_1 dot func

---------

Co-authored-by: Stephan Walter <stephan@walter.name>
This commit is contained in:
Georgi Gerganov 2023-04-15 17:53:22 +03:00 • committed by GitHub
parent aa485cee33
commit e95b6554b4
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
3 changed files with 442 additions and 18 deletions
Download patch file Download diff file Expand all files Collapse all files

2
Makefile
View file

@ -133,7 +133,7 @@ $(info I CC: $(CCV))
$(info I CXX: $(CXXV))
$(info )
default: main quantize perplexity embedding
default: main quantize quantize-stats perplexity embedding
#
# Build library