Grammar-constrained decoding: forcing LLMs to output valid syntax at the token level
LLMs hallucinate syntax because they sample tokens probabilistically.
Ask an LLM to generate a JSON object and it will eventually emit a trailing comma, an unescaped newline inside a string, or a bare word where a quoted key…