Daniel Bevenius cc1fd2fd0d llama : add remove_space_prefix to llama_detokenize ... This commit adds a new parameter to llama_detokenize to remove the leading space before tokens if they have a word boundary character. The motivation for this change is that when llama_server returns completion_propabilities, the tokens are detokenized and currently the leading space for the boundary tokens are removed. With this change llama_server can set remove_space_prefix to false and the leading space will be preserved. Resolves: https://github.com/ggerganov/llama.cpp/issues/11728		2025-02-10 16:21:06 +01:00
..
llama-cpp.h	llama : add llama_vocab, functions -> methods, naming (#11110)	2025-01-12 11:32:42 +02:00
llama.h	llama : add remove_space_prefix to llama_detokenize	2025-02-10 16:21:06 +01:00