vbatts/llama.cpp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Update test-tokenizer-random.py

Browse source
This commit is contained in:
Robert 2024-11-12 22:24:35 -08:00 • committed by GitHub
parent db26ba5b5c
commit 82a4012c2a
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
1 changed files with 0 additions and 4 deletions
Download patch file Download diff file Expand all files Collapse all files

4
tests/test-tokenizer-random.py
View file

@ -25,10 +25,6 @@ from typing_extensions import Buffer
import cffi
from transformers import AutoTokenizer, PreTrainedTokenizer
#
####################################################################################################
#
# Classes:
logger = logging.getLogger("test-tokenizer-random")