method · first seen 5 Oct 2026
pass@k
Also: max@k, pass@1, pass at k, pass@k evaluation, pass@k optimization
Metric giving the probability that at least one of k sampled solutions passes a verifier.
Mentions per week
PapersCodeDiscussion
Rank 142 on 2 Nov 2026 · 11 mentions in 28 days · 11 independent groups · papers · first seen 4 weeks ago · acceleration +0.80
Earliest sources
- What pass@k Cannot Measure: Evaluating Diversity and Capability Retention after Post-TrainingPapers · 5 Oct 2026 · Subham Rath, Raj Dandekar, Rajat Dandekar, Sreedath Panat
- The Optimal Decoder is Tilted TemperaturePapers · 10 Oct 2026 · Nikolaos Papagiannis, Ananth Grama, Wojciech Szpankowski
- Benchmarking the Quantum Leap: Auditing LLM Evaluation for Quantum Code GenerationPapers · 19 Oct 2026 · Ethan Seefried, Romit Ghosh, Sharmin Afrose, Nicholas Stapleton, Vicente Layton…
- Elicitation or Expansion? Demystifying Gains from Agent Harness OptimizationPapers · 20 Oct 2026 · Younghyun Kim, Wooseok Seo, Minseo Kim, Seunghyuk Oh, Minjae Lee, Donghoon Kim,…
- Direct Pass@$k$ Optimization for Thought Pruning: Balancing Quality and Diversity in Test-Time ReasoningPapers · 22 Oct 2026 · Mahdi Salmani, Adel Javanmard, Devansh Gupta, Meisam Razaviyayn
- What Outcome RL Buys: Finishing, Reliability, Rarely ReachPapers · 26 Oct 2026 · Shan Yang
Latest sources
- A Hierarchical Statistical Framework for Evaluating Agent ReliabilityPapers · 14 Nov 2026 · Mirco Milletarì, Tuhin Kundu
- One Temperature to Rule Them All?Papers · 9 Nov 2026 · Aviv Orly, Ori Shem-Ur, Yaron Oz
- Primitives and Paths: Predicting What Post-Training Can Do from What the Base Model Already HasPapers · 8 Nov 2026 · Anqi Qu
- Pdb or Not to Be: Comparing Debugging Interfaces With Pass$_{X}$@$k$Papers · 5 Nov 2026 · Ryan Magnuson, Ellie Kitanidis, Ariel Lee
- Controlling Sharpening in LLM Post-TrainingPapers · 2 Nov 2026 · Simon Matrenok, Nicolas Menet, Nicolas Flammarion, Andreas Krause
- Beyond Validation Loss: Coverage and Reliability Dynamics in Robot Policy PretrainingPapers · 1 Nov 2026 · Zhanyi Sun, Max Simchowitz, Shuran Song
- Rethinking LLM Pruning via Test-Time ScalingPapers · 31 Oct 2026 · Mingyang Song, Mao Zheng
- Few Trials, Many Metrics: Beta Mixtures for Low-Budget AI EvaluationPapers · 31 Oct 2026 · Gustavo Tironi, Felipe Maia Polo, Diego Mesquita
- There Is No Free Lunch in RLVR: Every pass@1 Gain Has a pass@k PricePapers · 29 Oct 2026 · Yinuo Liu
- Seq@K: Evaluating Language Models Under Sequential FeedbackPapers · 27 Oct 2026 · Thomson Yen, Gilbert Yang, Naimeng Ye, Minghui Chen, Dong Xia, Tianyi Shen, Yech…
- What Outcome RL Buys: Finishing, Reliability, Rarely ReachPapers · 26 Oct 2026 · Shan Yang
- Direct Pass@$k$ Optimization for Thought Pruning: Balancing Quality and Diversity in Test-Time ReasoningPapers · 22 Oct 2026 · Mahdi Salmani, Adel Javanmard, Devansh Gupta, Meisam Razaviyayn