Reward shaping
Also called Reward Shaping
Adding extra hints to the reward so the model gets feedback along the way, not just at the very end.
Think of it like
Giving a kid "warmer, warmer, colder" clues while they search, instead of only saying "found it" at the end.
Example
Rather than rewarding only a fully correct final answer, a trainer gives partial credit for each valid reasoning step to guide learning.
How it actually works
Sparse rewards are hard to learn from because most actions get zero signal. Shaping adds intermediate rewards to point the model toward good behavior sooner. The danger is that a poorly shaped reward changes what is optimal and invites reward hacking — the model chases the hints rather than the real goal — so shaping has to be designed carefully to preserve the true objective.
For product teams
Speeds up learning on hard tasks, but a sloppy shaping function can teach the wrong lesson.
For engineers
Augment sparse rewards with intermediate signals; keep them potential-based to avoid altering the optimal policy.
Related
- Reward hacking — What careless shaping can cause.
- Reward signal — The base signal being shaped.
Read anything AI without the jargon
Look up any term in plain English, or save terms as you read with the free Chrome extension.
Open DecoderAdd to Chrome