Automatic prompt engineering
Using algorithms (often other LLMs) to automatically generate and refine prompts, replacing manual trial-and-error.
Think of it like
A machine that writes recipes, tests them, and iterates — faster than a human chef.
Example
APE: give it a task and a small eval set, it generates 100 candidate prompts, scores them, and returns the best.
How it actually works
Common approaches: (1) LLM generates candidate prompts; (2) Evaluate on test set; (3) Mutate top prompts and repeat. Challenges: eval set quality (small sets are noisy), cost (evaluating many prompts is expensive), and transferability (best prompt for one model/task may not work for another).
For product teams
Faster iteration, less manual effort, sometimes better prompts.
For engineers
Use libraries like DSPy or custom scripts. Set up robust eval metrics. Validate on held-out data.
Related
- Prompt optimization — Part of prompt optimization workflow.
- Meta-prompting — Uses meta-prompting techniques.
Read anything AI without the jargon
Look up any term in plain English, or save terms as you read with the free Chrome extension.
Open DecoderAdd to Chrome