Add quantize-stats command for testing quantization (#728)

Command that calculates some statistics over the errors introduced by
quantization, like mean square error, max error and some percentile errors for layer
weights. Should be useful for testing quantization improvements.

Exposes some internal state from ggml and llama for testing
This commit is contained in:
unbounded 2023-04-08 00:09:18 +02:00 committed by GitHub
parent 698f7b5d63
commit 62cfc54f77
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
9 changed files with 415 additions and 17 deletions

View file

@ -0,0 +1,4 @@
set(TARGET quantize-stats)
add_executable(${TARGET} quantize-stats.cpp)
target_link_libraries(${TARGET} PRIVATE llama ${CMAKE_THREAD_LIBS_INIT})
target_compile_features(${TARGET} PRIVATE cxx_std_11)