vbatts/llama.cpp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Update README.md

Browse source
This commit is contained in:
ochafik 2024-09-27 07:18:07 +01:00
parent 86e4f99092
commit 2f25ee30ef
1 changed files with 5 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

6
examples/agent/README.md
View file

@ -14,17 +14,21 @@
<details> <details>
<summary>Instructions for NousResearch/Hermes-2-Pro-Llama-3-8B (needs template override)</summary> <summary>Instructions for NousResearch/Hermes-2-Pro-Llama-3-8B (needs template override)</summary>
The HF model had two variants for its chat template (`default` and `tool_use`), but the GGUF only retained the `default` one.
```bash ```bash
./llama-server \ ./llama-server \
--jinja -fa \ --jinja -fa \
-mu https://huggingface.co/NousResearch/Hermes-2-Pro-Llama-3-8B-GGUF/resolve/main/Hermes-2-Pro-Llama-3-8B-Q8_0.gguf \ -mu https://huggingface.co/NousResearch/Hermes-2-Pro-Llama-3-8B-GGUF/resolve/main/Hermes-2-Pro-Llama-3-8B-Q8_0.gguf \
--chat-template-file tests/chat/templates/NousResearch-Hermes-2-Pro-Llama-3-8B-tool_use.jinja --chat-template-file tests/chat/templates/NousResearch-Hermes-2-Pro-Llama-3-8B-tool_use.jinja
``` ```
`
</details>
<details> <details>
<summary>Instructions for meekai/functionary-small-v3.2 (needs template override)</summary> <summary>Instructions for meekai/functionary-small-v3.2 (needs template override)</summary>
The template in the GGUF doesn't seem to support tool calls, but its bigger brother's template can be used: The template in the GGUF doesn't support tool calls, but its bigger brother's template can be used:
```bash ```bash
./llama-server \ ./llama-server \