add basic tensor data validation function (#6884)

* add basic tensor data validation function

* add --check-tensors command line argument

tensor validation is disabled by default and can be enabled by adding
`--check-tensors` to the command line arguments.

quantize always validates tensors.
This commit is contained in:
slaren 2024-04-26 18:39:58 +02:00 committed by GitHub
parent e2764cd7ca
commit 017e6999b5
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
6 changed files with 371 additions and 19 deletions

View file

@ -161,6 +161,7 @@ struct gpt_params {
bool dump_kv_cache = false; // dump the KV cache contents for debugging purposes
bool no_kv_offload = false; // disable KV offloading
bool warmup = true; // warmup run
bool check_tensors = false; // validate tensor data
std::string cache_type_k = "f16"; // KV cache data type for the K
std::string cache_type_v = "f16"; // KV cache data type for the V