Early-Signal Detector

phenomenon · first seen 5 Oct 2026

reinforcement learning from policy gradients with train-inference mismatch

Also: TIM, RL rollout mismatch, training-inference mismatch

Numerical discrepancy between the learner and sampler in RL for LLMs that can destabilize policy optimization.

Mentions per week

0219 May11 Aug10 NovPapersCodeDiscussion

PapersCodeDiscussion

Earliest sources

Latest sources

Mentioned together with

NVFP4 quantizationimportance samplingreinforcement learning from policy gradients for language modelsvLLMon-policy distillation