method · first seen 1 Nov 2026
attention scaling for resolution extrapolation
Also: log-ratio attention rescaling, resolution-dependent attention temperature
Training-free rescaling of attention scores by a log ratio of token counts so softmax attention stays focused at higher resolutions.
Mentions per week
PapersCodeDiscussion
Earliest sources
- Spread Too Thin: How Higher Resolution Dilutes a Vision Transformer's Attention, and a Training-Free RescalingPapers · 1 Nov 2026 · Neil D. B. Bruce
Latest sources
- Spread Too Thin: How Higher Resolution Dilutes a Vision Transformer's Attention, and a Training-Free RescalingPapers · 1 Nov 2026 · Neil D. B. Bruce