convert : add t5 tokenizer tests
This commit is contained in:
parent
6dc9eb4040
commit
9eb5d5617d
27 changed files with 47 additions and 4 deletions
|
@ -31,6 +31,7 @@
|
|||
1212 40
|
||||
18 4932
|
||||
9856 23 291 18 436 12 1265 362 299 8196 207 204 42 50087 123 2727 20300 32022 133 234 17419 30137 28 7858 181 133 236
|
||||
51520
|
||||
30
|
||||
3138
|
||||
22287
|
||||
|
|
Loading…
Add table
Add a link
Reference in a new issue