* llama, main : constrain sampling to grammar * allow loading grammar from file * fix whitespace errors * handle & print parser errors * add comments to grammar syntax and allow newlines where unambiguous * add missing include * support alternates in root rule * fix bugs with empty token and EOS * adjust JSON grammar * remove swp file * rewrite ternary expressions Co-authored-by: Henri Vasserman <henv@hot.ee> * use struct for grammar elements and add Unicode support * add unicode escapes * add inverse char ranges * only sample full tokens (no peeking or truncation) * llama : minor style changes blindly applied in online editor - hopefully I didn't break something * update help text * add warning message if EOS is disabled --------- Co-authored-by: Henri Vasserman <henv@hot.ee> Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
		
			
				
	
	
		
			13 lines
		
	
	
	
		
			565 B
		
	
	
	
		
			Text
		
	
	
	
	
	
			
		
		
	
	
			13 lines
		
	
	
	
		
			565 B
		
	
	
	
		
			Text
		
	
	
	
	
	
| # Specifies chess moves as a list in algebraic notation, using PGN conventions
 | |
| 
 | |
| # Force first move to "1. ", then any 1-2 digit number after, relying on model to follow the pattern
 | |
| root    ::= "1. " move " " move "\n" ([1-9] [0-9]? ". " move " " move "\n")+
 | |
| move    ::= (pawn | nonpawn | castle) [+#]?
 | |
| 
 | |
| # piece type, optional file/rank, optional capture, dest file & rank
 | |
| nonpawn ::= [NBKQR] [a-h]? [1-8]? "x"? [a-h] [1-8]
 | |
| 
 | |
| # optional file & capture, dest file & rank, optional promotion
 | |
| pawn    ::= ([a-h] "x")? [a-h] [1-8] ("=" [NBKQR])?
 | |
| 
 | |
| castle  ::= "O-O" "-O"?
 |