method · first seen 9 Oct 2026
Q-learning
A model-free reinforcement learning algorithm that learns action-value functions via temporal-difference updates.
Q-learning is the classic model-free reinforcement learning algorithm, introduced by Watkins in 1989, that learns state-action values through temporal-difference updates and off-policy bootstrapping. The sources are a typical cross-section of ongoing work rather than a new trend. Several are theory papers: hitting-time bounds and sharp Gaussian approximation for asynchronous Q-learning, an analysis of its transient error, and average-reward regularization. Others apply it to mmWave jammer detection, firewall feature selection, treatment rules and hardware fault-injection attacks. A Stanford paper reframes value learning as a generative classifier, and another proves that transformers can implement Q-learning in context.
The rise is routine arXiv volume for a foundational algorithm, not a new signal. The work worth watching is the theory on in-context Q-learning in transformers and the finite-sample analyses of convergence and error behaviour.
Analyst note (established), 17 Nov 2026.
Mentions per week
PapersCodeDiscussion
Earliest sources
- Hitting-Time Bounds for Asynchronous Q-LearningPapers · 9 Oct 2026 · Yang Peng
- Mixing Regularization for Average-Reward $Q$-Learning and $Q$-PlanningPapers · 10 Oct 2026 · Julien Roger, Sam Bodmer, Jian Qian, Benjamin Riviere
- Digital-Twin-Referenced Belief-Guided Q-Learning for Jammer Identification and Beam-Power Control in a mmWave ModelPapers · 12 Oct 2026 · Parmida Geranmayeh, Onur Günlü
- From Policy Evaluation to Off-Policy Control: Provable Emergence of In-Context Q-LearningPapers · 13 Oct 2026 · Yingjun Mo, Yajie Zhao, Fang Kong
- Neural Supervised Clustering for Learning Individualized Treatment Rules with Many TreatmentsPapers · 13 Oct 2026 · Daixi Huang, Yixin Yuan, Yifan Dai, Yufeng Liu
- V-SHARE: Provable Convergence Guarantees for Advantage-based AlgorithmsPapers · 15 Oct 2026 · Kintan Saha, Hamed Masoudi, Xinyi Sheng, Dominik Baumann, Joni Pajarinen, Arseni…
Latest sources
- Sharp Gaussian Approximation for Q-LearningPapers · 15 Nov 2026 · Xingtu Liu
- Optimal feature selection for firewall log analysis using Machine learning and Hybrid Metaheuristic algorithmsPapers · 12 Nov 2026 · Seungwoo Han, Gil Hong, Jewan Kim, Jeuk Yu, Sangjun Lee, Byeongok Cho, Jusung Je…
- Fault Injection Attacks on Q-Learning: Analysis, Vulnerabilities, and CountermeasuresPapers · 12 Nov 2026 · Tom Slooff, Anthony Etim, Jiaqi Yu, Francesco Regazzoni, Jakub Szefer
- Understanding the Asymptotic Transience of Q-Learning via Distributional ExpansionPapers · 11 Nov 2026 · Shengbo Wang
- Prediction with Foundation Models under Deployment BudgetsPapers · 9 Nov 2026 · Zhaohui Xu, Xiaofei Xu, Doudou Zhou
- Safety-Focused Reinforcement Learning Using Risk-Sensitive Coupled ControlPapers · 6 Nov 2026 · Jiabin Lin, Shana Moothedath, Vivek S. Borkar
- Generative Value ClassifierPapers · 15 Oct 2026 · Alexander Swerdlow, Perry Dong, Chelsea Finn
- V-SHARE: Provable Convergence Guarantees for Advantage-based AlgorithmsPapers · 15 Oct 2026 · Kintan Saha, Hamed Masoudi, Xinyi Sheng, Dominik Baumann, Joni Pajarinen, Arseni…
- Neural Supervised Clustering for Learning Individualized Treatment Rules with Many TreatmentsPapers · 13 Oct 2026 · Daixi Huang, Yixin Yuan, Yifan Dai, Yufeng Liu
- From Policy Evaluation to Off-Policy Control: Provable Emergence of In-Context Q-LearningPapers · 13 Oct 2026 · Yingjun Mo, Yajie Zhao, Fang Kong
- Digital-Twin-Referenced Belief-Guided Q-Learning for Jammer Identification and Beam-Power Control in a mmWave ModelPapers · 12 Oct 2026 · Parmida Geranmayeh, Onur Günlü
- Mixing Regularization for Average-Reward $Q$-Learning and $Q$-PlanningPapers · 10 Oct 2026 · Julien Roger, Sam Bodmer, Jian Qian, Benjamin Riviere