vbatts/llama.cpp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

grammars: add troubleshooting section to readme

Browse source
This commit is contained in:
ochafik 2024-04-08 20:10:15 +01:00
parent a59e9431fc
commit 07163fb627
1 changed files with 10 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

10
grammars/README.md
View file

@ -89,3 +89,13 @@ This guide provides a brief overview. Check out the GBNF files in this directory
```
./main -m <model> --grammar-file grammars/some-grammar.gbnf -p 'Some prompt'
```
## Troubleshooting
Grammars currently have performance gotchas (see https://github.com/ggerganov/llama.cpp/issues/4218).
### Efficient optional repetitions
A common pattern is to allow repetitions of a pattern `x` up to N times.
While semantically correct, the syntax `x? x? x?.... x?` (with N repetitions) will result in extremely slow inference. Instead, you can write `(x (x (x ... (x)?...)?)?)?` (w/ N-deep nesting)