vbatts/llama.cpp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Tests: Added integration tests for GBNF parser (#6472)

Browse source 
* Added integration tests for GBNF parser to validate correctness of parsing, as well as correctness of string matching. Intended for use to pin behavior while working on performance improvements.

* Fixing whitespace errors and cleaning error message alert to be clearer.

* Removing hacky include to llama.cpp from grammar integration test now that needed functions are available via internal API.

* Comment cleanup.

* Reorganizing tests for readability.

* Cleaning up debug message to make a bit more sense.
This commit is contained in:
Clint Herron 2024-04-06 10:31:33 -04:00 • committed by GitHub
parent 75cd4c7729
commit 57dd02c44b
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
3 changed files with 249 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

1
tests/CMakeLists.txt
View file

@ -59,6 +59,7 @@ llama_test(test-tokenizer-1-bpe.cpp NAME test-tokenizer-1-gpt2 AR
llama_test(test-grammar-parser.cpp)
llama_test(test-llama-grammar.cpp)
llama_test(test-grammar-integration.cpp)
llama_test(test-grad0.cpp)
# llama_test(test-opt.cpp) # SLOW
llama_test(test-backend-ops.cpp)