convert : add t5 tokenizer tests
This commit is contained in:
parent
6dc9eb4040
commit
9eb5d5617d
27 changed files with 47 additions and 4 deletions
|
@ -31,6 +31,7 @@
|
|||
29871 13 353
|
||||
525 3152
|
||||
15043 29892 343 29915 497 29991 1128 526 366 29871 243 162 155 132 1577 30672 31522 30505 11548 31041 30732 29896 29941 29896 29946 29896 29945 29896 30408 30739
|
||||
1738 6824 21004
|
||||
29871 29941
|
||||
29871 29941 29941
|
||||
29871 29941 29941 29941
|
||||
|
|
Loading…
Add table
Add a link
Reference in a new issue