From 2ecc2ae90020c4ab5863ab8ce090d3de44c5268f Mon Sep 17 00:00:00 2001 From: ochafik Date: Sat, 20 Apr 2024 01:25:58 +0100 Subject: [PATCH] grammars: update performance gotchas w/ repetition advice --- grammars/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/grammars/README.md b/grammars/README.md index fcc6f9c57..443501325 100644 --- a/grammars/README.md +++ b/grammars/README.md @@ -102,4 +102,4 @@ Grammars currently have performance gotchas (see https://github.com/ggerganov/ll A common pattern is to allow repetitions of a pattern `x` up to N times. -While semantically correct, the syntax `x? x? x?.... x?` (with N repetitions) will result in extremely slow inference. Instead, you can write `(x (x (x ... (x)?...)?)?)?` (w/ N-deep nesting) +While semantically correct, the syntax `x? x? x?.... x?` (with N repetitions) may result in extremely slow sampling. Instead, you can write `x{0,N}` (or `(x (x (x ... (x)?...)?)?)?` w/ N-deep nesting in earlier llama.cpp versions).