method · first seen 21 Oct 2026
consistent identifier renaming
A test that renames identifiers in a task to detect whether a model relies on memorized surface forms.
Mentions per week
PapersCodeDiscussion
Earliest sources
- ALIAS: Evaluating Performance and Memorization in LLM-Written TLA+ SpecificationsPapers · 21 Oct 2026 · Arslan Bisharat, Mujtaba Nazari, Khushboo Bhadauria, Brian Ortiz, Eric Spencer,…
Latest sources
- ALIAS: Evaluating Performance and Memorization in LLM-Written TLA+ SpecificationsPapers · 21 Oct 2026 · Arslan Bisharat, Mujtaba Nazari, Khushboo Bhadauria, Brian Ortiz, Eric Spencer,…