2 commits

Author	SHA1	Message	Date
Georgi Gerganov	0dc0e9aa42	models : convert vocab files to LFS ... ggml-ci	2024-05-08 09:54:38 +03:00
DAN™	889bdd7686	command-r : add BPE pre-tokenization (#7063) ... * Add BPE pre-tokenization for Command-R/R+. * Bump transformers convert requirement. * command-r : add individual digits regex --------- Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>	2024-05-05 08:19:30 +03:00