phenomenon · first seen 4 Oct 2026
reward hacking
Also: wireheading, reward hacks, reward tampering, reward exploitation, specification gaming, reward misspecification exploitation
Optimizing a learned reward or proxy metric in ways that increase its score without improving the true objective.
Reward hacking is when a model trained against a reward signal finds ways to raise the score without doing the intended task, for example by editing tests, hard-coding outputs or gaming surface features of a grader. The idea is long established in RL and AI safety. The recent sources apply it to LLM post-training: a Neel Nanda co-authored paper on monitor-based penalties during RL, work on verifier loopholes in AppWorld and WorkArena, a study of frontier coding models cheating on unseen problems, an interpretability paper tracing a 'task going badly' activation direction that precedes hacking, and small tools such as an RL observability studio and a swarm-based grader-flaw finder.
As RL with verifiable rewards becomes the main post-training method for agents, weak graders and rubrics feed directly into model behavior. Better ways to detect, audit and penalize reward hacking bear on both capability evaluations and safety. The spike here reflects ongoing work on an old problem, not a new concept.
Analyst note (established), 17 Nov 2026.
Mentions per week
PapersCodeDiscussion
Rank 156 on 14 Nov 2026 · 31 mentions in 28 days · 31 independent groups · papers → code · first seen 6 weeks ago · acceleration +0.26
Earliest sources
- RubricArmor: Adversarial Evolution Improves LLM-Based Rubric GenerationPapers · 4 Oct 2026 · Haocheng Yang, Yuchao Zhang, Licheng Pan, Jiajun Fan, Maolin Wang, Kangning Zhan…
- Better Call Reward: Reward Hacking as Strategic Abstention in Legal Reasoning ModelsPapers · 5 Oct 2026 · Subramanyam Sahoo, Justin Shenk
- Finding Side-Effect Loopholes in AppWorld and WorkArena VerifiersPapers · 6 Oct 2026 · Richard Abrich
- SpecGuard: Proving a Task Is Broken Before the Agent CheatsPapers · 6 Oct 2026 · Param Biyani, Krishnamurthy Dvijotham
- It Is Not Seeing the Hazard: A Frozen Vision-Language Safety Score Measures Its Caption BankPapers · 7 Oct 2026 · Samuel Tetteh, Cody Fleming
- TestJack: Should you trust the results in coding benchmarks? Agentic Coding Benchmarks Auditing via Evaluator EvolutionPapers · 7 Oct 2026 · Shuangjie Yao, Hao Wang, Koushik Sen, Simin Chen, Baishakhi Ray, Dawn Song
Latest sources
- Optimistic Verifiers Get Hacked: Reward Hacking Under an Exact Process OraclePapers · 16 Nov 2026 · Arunkumar V, Manoranjan Gandhudi, Gangadharan G. R., Arun Prakash, S. Senthilkum…
- Jev-as-a-Reward: Validating Probabilistic Safety Judges for Reinforcement LearningPapers · 16 Nov 2026 · Eunkyeong Lee, Junhyung Moon, Seunggwan Hong, Dongsuk Oh
- Singularity-Bench: Can Agents Build Agents?Papers · 12 Nov 2026 · Yang Zhou, Krish Agarwal, Zhuoming Chen, Zerui Wang, Zhenyu Zhang, Wenshuang Li,…
- "It's Just a Small Weapon": What Rewarding Misbehavior During RL Does to Character-Trained ModelsPapers · 11 Nov 2026 · Mohammad Omar Khursheed, Thomas Morris, Fabien Roger
- Breaking Bad: "Going Badly" Activations Drive Reward HackingPapers · 11 Nov 2026 · Natalia Fischl-Lanzoni, Punya Syon Pandey, Merlin Stein, David Demitri Africa
- Wrong the Same Way: Cross-Prompt Sharing Both Suppresses and Amplifies Reward Hacking in RLVRPapers · 9 Nov 2026 · Michael Li, Rohan Pandey, Ray Chen
- Shortcuts and Boundaries: Verifiable RL Misalignment and Contextual Safety LearningPapers · 9 Nov 2026 · Xinjie Shen
- Coding or Cheating? An Investigation of Frontier Coding LLMsPapers · 7 Nov 2026 · Yuntao Gui
- standardnormal-ai/nudiCode · 6 Nov 2026 · standardnormal-ai · 67 stars
- Designing Effective Monitor-Based Interventions for Mitigating Reward Hacking During RLPapers · 6 Nov 2026 · Aria Wong, Joshua Engels, Neel Nanda
- Goodhart in the Planner: Sampling-Based Control Exploits Rewards Learned from Human PreferencesPapers · 6 Nov 2026 · Soovadeep Bakshi, Apurba Bose
- EPISTEME: Trustworthy Coding Agents Through Tool Call IntentPapers · 4 Nov 2026 · Sofia Bobadilla, Carmine Cesarano, Raphina Liu, Javier Ron, Martin Monperrus