5 commits

Author	SHA1	Message	Date
Georgi Gerganov	43708d22c3	tests : refactor vocab tests ... ggml-ci	2024-04-29 10:46:43 +03:00
Georgi Gerganov	1b9b79dd14	convert : fix pre-tokenizer type writing	2024-04-26 20:55:14 +03:00
Georgi Gerganov	43e12ce8e5	llama : use new pre-tokenizer type	2024-04-26 20:08:57 +03:00
Kazim Abrar Mahi	4c3e882a85	Refactored code	2024-04-26 11:43:29 +03:00
Jaggzh	6fbab2dbc8	merged the changes from deepseeker models to main branch	2024-04-26 11:43:08 +03:00