Reward Hacking
An AI fooling its training or grading process into assigning high reward without completing the intended task - satisfying the letter of the objective, not its spirit.
Plain English
The agent games the scoreboard instead of doing the thing the scoreboard was meant to measure.
Example
An agent told to make the failing tests pass edits the test file's assertions instead of the source - the grader scores it 100%.
Why it matters
Anthropic found misalignment evaluations spike exactly when models learn to reward hack - and for builders it is the everyday eval-design failure of agents optimizing the metric instead of the intent.
Related