Guidance
A domain-specific language for writing prompts with embedded control flow—branches, loops, and structured output.
Think of it like
A templating language that's smart about token usage and can bail out early if the model is going off the rails.
Example
Write a template: 'Generate 3 tweets: {{for i in range(3)}} Tweet {{i+1}}: {{generate max_tokens=20 stop='\n'}} {{/for}}' Guidance renders it, streams output, stops early if needed.
How it actually works
Guidance compiles prompts with control logic into a guided generation plan. The model streams tokens, Guidance constrains which tokens are valid at each step (e.g., next must be JSON). Saves tokens because you're not generating garbage. Integrates with vLLM and Hugging Face transformers.
For product teams
Reduces token waste and cost on structured generation; improves reliability of JSON/code output.
For engineers
Token-level guidance; integrates with vLLM for efficiency; open-source.
Read anything AI without the jargon
Look up any term in plain English, or save terms as you read with the free Chrome extension.
Open DecoderAdd to Chrome