vbatts/llama.cpp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

ggml : unit test for quantization functions (#953)

Browse source 
* Unit test for quantization functions

Use the ggml_internal_get_quantize_fn function to loop through all
quantization formats and run a sanity check on the result.

Also add a microbenchmark that times these functions directly without
running the rest of the GGML graph.

* test-quantize-fns: CI fixes

Fix issues uncovered in CI
 - need to use sizes divisible by 32*8 for loop unrolling
 - use intrinsic header that should work on Mac

* test-quantize: remove

Per PR comment, subsumed by test-quantize-fns

* test-quantize: fix for q8_0 intermediates
This commit is contained in:
unbounded 2023-04-22 11:10:39 +02:00 • committed by GitHub
parent 36b4f7e064
commit 5f939498d5
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
4 changed files with 466 additions and 43 deletions
Download patch file Download diff file Expand all files Collapse all files

3
tests/CMakeLists.txt
View file

@ -6,5 +6,6 @@ function(llama_add_test source)
endfunction()
# llama_add_test(test-double-float.c) # SLOW
llama_add_test(test-quantize.c)
llama_add_test(test-quantize-fns.cpp)
llama_add_test(test-quantize-perf.cpp)
llama_add_test(test-tokenizer-0.cpp ${CMAKE_CURRENT_SOURCE_DIR}/../models/ggml-vocab.bin)