convert : add t5 tokenizer tests
This commit is contained in:
parent
6dc9eb4040
commit
9eb5d5617d
27 changed files with 47 additions and 4 deletions
|
@ -31,6 +31,7 @@
|
|||
185 405
|
||||
6 2895
|
||||
17535 11 320 6 435 0 1717 417 340 12394 233 210 3015 19100 608 9413 2668 16 18 16 19 16 20 16 1393 169 121 239
|
||||
15330 3023
|
||||
18
|
||||
18 18
|
||||
18 18 18
|
||||
|
|
Loading…
Add table
Add a link
Reference in a new issue