benchmark · first seen 4 Oct 2026
AI agent evaluation
Also: agent testing, LLM agent evaluation, language model agent evaluation
Assessing autonomous language model agents on multistep tasks using tools, data access, and reported outcomes.
Mentions per week
PapersCodeDiscussion
Rank 181 on 16 Nov 2026 · 7 mentions in 28 days · 7 independent groups · papers → discussion · first seen 6 weeks ago · acceleration +1.24
Earliest sources
- DelegationBench: Measuring When AI Agents Should Ask Before ActingPapers · 4 Oct 2026 · Shiva Pochampally
- BioStudyBench: Evaluating Agents on Post-Cutoff Biomedical StudiesPapers · 6 Oct 2026 · David Li, Shaamil Karim, Christian Gensbigler
- EIO-Agents: The Missing Semantic Layer for AI Agent EvaluationPapers · 6 Oct 2026 · Fouad Bousetouane
- Confidence Reasoning Graphs: Structured Confidence Estimation for LLM AgentsPapers · 6 Oct 2026 · Brendan King, Farima Fatahi Bayat, Jean-Flavien Bussotti, Pouya Pezeshkpour, Est…
- Transect: Retaining Observability for Long-Horizon LLM Agent EvaluationsPapers · 6 Oct 2026 · Toby D. Pilditch, Konstantinos Voudouris, Alexandra Abbas, Cozmin Ududec
- ACED-Bench: Evaluating How LLM Agents Acquire and Act on Causal EvidencePapers · 9 Oct 2026 · Yiwei Chen, Amila Weerasinghe, Siddharth Biswal, Utkrisht Rajkumar, Olcay Boz, T…
Latest sources
- Instrumenting Failure: Scoring Language Model Agents at Every Decision Node in Imperfect-Information GamesPapers · 16 Nov 2026 · Michael Kingston, Sonit Singh, Alan Blair
- AskALCF: A Self-Improving Agentic System for Intelligent HPC User SupportPapers · 11 Nov 2026 · Jingyan Jiang, Akhilesh Bondapalli, Krishna Rauniyar, Murat Keceli, Yasaman Ghad…
- Publications as benchmarks: auditing AI agents on spatial transcriptomics alignmentPapers · 9 Nov 2026 · Yiqun T. Chen, Jan Matthias, Jianing Yao, Hongyu Eric Zhao, Stephanie C. Hicks
- InfoOrgBench: Can Agents Turn Scattered Information into Coherent Hierarchies?Papers · 7 Nov 2026 · Justice Ou, Lanming Ma, Zijian Zhang, Yilun Zhao
- EEFA: A Framework for the Ethical Evaluation of AI AgentsPapers · 2 Nov 2026 · Swathi Pallikala, Yashas Hariprasad
- REGATTA: A Cross-Domain Measurement Framework and Fault-Injection Harness for Enterprise AI AgentsPapers · 22 Oct 2026 · Luong Tuan Thanh, Abhijit Sanyal
- AgenTest: Mining Business Rules to Test LLM Agents for Business Logic BugsPapers · 20 Oct 2026 · Lili Quan, Longtian Wang, Yuelin Wang, Mingfei Cheng, Chao Shen, Sen Chen, Xiaof…
- How Far Will AI Assistants Go for a Refund They Can't Get?Discussion · 18 Oct 2026 · DeepLogin · 108 points
- FLARE: Physics-Grounded Evaluation of Language-Model Agents for Feeder Restoration with Degraded ObservationsPapers · 16 Oct 2026 · BHVSP Subrahmanyam, Neeraj Choudhary
- JockeyBench: Evaluating LLM Agents on Horse-Racing Decision-MakingPapers · 14 Oct 2026 · Zhongwei Xie, Xingsu Li, Jingqiao Yu, Zihao Wang, Hong ChiTat, Zhaowei Wang, Wei…
- EvoProbe: Evaluation-Driven Benchmark Evolution for LLM AgentsPapers · 14 Oct 2026 · Qianhong Guo, Wei Xie, Xiaofang Cai, Shuoyoucheng Ma, Chengxin Hu, Baosheng Wang
- Can LLM Agents Efficiently Identify Appliance Faults? Insights From Diagnosing a Simulated AppliancePapers · 13 Oct 2026 · Lucas H. McCabe, Sarah Lukens, H. Howie Huang