convert : add t5 tokenizer tests
This commit is contained in:
parent
6dc9eb4040
commit
9eb5d5617d
27 changed files with 47 additions and 4 deletions
|
@ -31,6 +31,7 @@
|
|||
198 284
|
||||
6 11385
|
||||
9707 11 379 64848 0 2585 525 498 26525 223 937 104100 18493 22377 99257 16 18 16 19 16 20 16 35727 21216
|
||||
17085 2928
|
||||
18
|
||||
18 18
|
||||
18 18 18
|
||||
|
|
Loading…
Add table
Add a link
Reference in a new issue