vbatts/llama.cpp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

tweaks to grammar guide

Browse source
This commit is contained in:
Evan Jones 2023-08-21 23:09:59 -04:00 • committed by GitHub
parent be6faa45a5
commit fbea8db29c
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
1 changed files with 14 additions and 4 deletions
Download patch file Download diff file Expand all files Collapse all files

18
grammars/README.md
View file

@ -8,7 +8,7 @@ GBNF (GGML BNF) is a format for defining [formal grammars](https://en.wikipedia.
## Basics ## Basics
In GBNF, we define *production rules* that specify how a non-terminal (rule name) can be replaced with sequences of terminals (characters, specifically Unicode [code points](https://en.wikipedia.org/wiki/Code_point)) and other non-terminals. The basic format of a production rule is `nonterminal ::= sequence...`. In GBNF, we define *production rules* that specify how a *non-terminal* (rule name) can be replaced with sequences of *terminals* (characters, specifically Unicode [code points](https://en.wikipedia.org/wiki/Code_point)) and other non-terminals. The basic format of a production rule is `nonterminal ::= sequence...`.
## Examples ## Examples
@ -43,9 +43,9 @@ Parentheses `()` can be used to group sequences, which allows for embedding alte
## Repetition and Optional Symbols ## Repetition and Optional Symbols
`*` after a symbol or sequence means that it can be repeated zero or more times. - `*` after a symbol or sequence means that it can be repeated zero or more times.
`+` denotes that the symbol or sequence should appear one or more times. - `+` denotes that the symbol or sequence should appear one or more times.
`?` makes the preceding symbol or sequence optional. - `?` makes the preceding symbol or sequence optional.
## Comments and newlines ## Comments and newlines
@ -57,6 +57,16 @@ ws ::= [ \t\n]+
Newlines are allowed between rules and between symbols or sequences nested inside parentheses. Additionally, a newline after an alternate marker `|` will continue the current rule, even outside of parentheses. Newlines are allowed between rules and between symbols or sequences nested inside parentheses. Additionally, a newline after an alternate marker `|` will continue the current rule, even outside of parentheses.
## The root rule
In a full grammar, the `root` rule always defines the starting point of the grammar. In other words, it specifies what the entire output must match.
```
# a grammar for lists
root ::= ("- " item)+
item ::= [^\n]+ "\n"
```
## Next steps ## Next steps
This guide provides a brief overview. Check out the GBNF files in this directory (`grammars/`) for examples of full grammars. You can try them out with: This guide provides a brief overview. Check out the GBNF files in this directory (`grammars/`) for examples of full grammars. You can try them out with: