Georgi Gerganov
|
2db2471c13
|
speculative : avoid grammar_mem
|
2023-09-04 15:48:38 +03:00 |
|
Georgi Gerganov
|
e7dc5b08ac
|
speculative : reuse grammar parser + better logs and comments
|
2023-09-04 15:48:38 +03:00 |
|
Georgi Gerganov
|
6c150d763e
|
speculative : print draft token pieces
|
2023-09-04 15:48:38 +03:00 |
|
Georgi Gerganov
|
69f2fafebc
|
speculative : add grammar support
|
2023-09-04 15:48:37 +03:00 |
|
Georgi Gerganov
|
47068e5170
|
speculative : PoC for speeding-up inference via speculative sampling (#2926)
* speculative : initial example
* speculative : print encoding speed
* speculative : add --draft CLI arg
|
2023-09-03 15:12:08 +03:00 |
|