convert : add t5 tokenizer tests
This commit is contained in:
parent
6dc9eb4040
commit
9eb5d5617d
27 changed files with 47 additions and 4 deletions
|
@ -31,6 +31,7 @@
|
|||
203 280
|
||||
25 34666
|
||||
8279 30 533 25 464 19 4971 884 844 18458 228 1018 4982 13368 2909 9513 17827 35 37 35 38 35 39 35 11873 47838
|
||||
9163 3202
|
||||
37
|
||||
37 37
|
||||
37 37 37
|
||||
|
|
Loading…
Add table
Add a link
Reference in a new issue