vbatts/llama.cpp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

tool-calls: accommodate variety of wrong tool call opening tags both Qwen 32B and 7B distills like to spit out

Browse source
This commit is contained in:
Olivier Chafik 2025-02-03 18:57:55 +00:00
parent c397bd1f5f
commit 569610ee77
2 changed files with 11 additions and 5 deletions
Download patch file Download diff file Expand all files Collapse all files

3
examples/server/README.md
View file

@ -1206,8 +1206,7 @@ curl http://localhost:8080/v1/chat/completions \
llama-server --jinja -fa -hf bartowski/Mistral-Nemo-Instruct-2407-GGUF:Q6_K_L
llama-server --jinja -fa -hf bartowski/functionary-small-v3.2-GGUF:Q4_K_M
llama-server --jinja -fa -hf bartowski/Llama-3.3-70B-Instruct-GGUF:Q4_K_M
llama-server --jinja -fa -hf bartowski/DeepSeek-R1-Distill-Qwen-7B-GGUF:Q4_K_M
llama-server --jinja -fa -hf bartowski/DeepSeek-R1-Distill-Qwen-32B-GGUF:Q6_K_L
llama-server --jinja -fa -hf bartowski/DeepSeek-R1-Distill-Qwen-32B-GGUF:Q4_K_M
# Native support requires the right template for these GGUFs: