method · first seen 6 Oct 2026
FlashAttention
Also: FlashAttention-3
An IO-aware exact attention algorithm that fuses operations and tiles computation to reduce GPU memory traffic.
Mentions per week
PapersCodeDiscussion
Rank 147 on 7 Nov 2026 · 14 mentions in 28 days · 14 independent groups · papers → discussion · first seen 5 weeks ago · acceleration -0.13
Earliest sources
- Cleave: Scaling Tensor Program Optimization via Decoupled Algebraic Search and Operator SchedulingPapers · 6 Oct 2026 · David Pissarra, Jinkun Lin, Haitian Jiang, Aurojit Panda, Jinyang Li
- FlashMask-3: Efficient and Expressive Mask-Aware Distributed AttentionPapers · 13 Oct 2026 · Guoxia Wang, Qianyue He, Siming Wu, Haoyang Xie, Qiwen Bao, Jinle Zeng, Jiabin Y…
- FlashAttention with Statistics: Low-Overhead Attention Features for Compact Hallucination DetectorsPapers · 15 Oct 2026 · Andrei Volodichev, Maksim Savichev, Nikita Sviridov, Artem Yuliy, Dmitry Simakov…
- Reorienting the Vector Axis: Restructuring FlashAttention for Long-Vector ProcessorsPapers · 21 Oct 2026 · Emmanuel Irabor, Marc Casas
- LazyFlash: Forward-Dense, Backward-Sparse Attention for Accelerated Long-Context Fine-TuningPapers · 22 Oct 2026 · Keqi Deng, Ruchao Fan, Jinyu Li
- INTRA: Interleaved Non-contiguous Token spaRse AttentionPapers · 25 Oct 2026 · Yuxuan Xia, Wenbo Lu, Shaoyi Zheng, Shengjie Wang
Latest sources
- Spatial-Spectral Coupling as Attention: Scaling Shape Correspondence to High-Resolution SurfacesPapers · 10 Nov 2026 · Bence Demeter, Kilian Ehrenreich, Amelie Kleber, Ioana Oprisiu, Philipp Spohn, C…
- An Empirical Study of Issues in GPU Kernel Libraries for LLM SystemsPapers · 6 Nov 2026 · Yanjie Gao, Jiayi Bi, Haoxiang Lin, Ling Feng
- ZYFA: Speed and Accuracy of FP8 FlashAttention on AMD CDNA4Papers · 5 Nov 2026 · Quentin Anthony, Ganesh Bikshandi, Praneeth Medepalli, Stephen Ebert
- Formalizing PTX in Lean to Prove GPU Kernel CorrectnessPapers · 3 Nov 2026 · Luk Koska, Tom Koska, Ben Koska
- Turbo-GMM: FlashAttention-Inspired GPU-Accelerated Gaussian Mixture ModelPapers · 2 Nov 2026 · Momin Abbas, Sadia Asif, Asad Ul Haq, Kaoutar El Maghraoui
- Fast and Exact Relative-Position Attention via Structure-Aware KernelsPapers · 31 Oct 2026 · Delyan Boychev, Ivo Petrov, Dimitar I. Dimitrov
- Show HN: Automatic deduction of FlashAttention kernel from ~25 lines of SQLDiscussion · 30 Oct 2026 · alxmrs · 28 points
- TriTile: Three-Tier Attention for Video Diffusion TransformersPapers · 26 Oct 2026 · Yeonjae Kim, Omin Kwon, Seong Hoon Seo, Jae W. Lee
- When and Why to Use TMA: A Workload-Grounded Characterization of Tensor Memory Acceleration on NVIDIA Hopper GPUsPapers · 26 Oct 2026 · Christin David Bose, Cesar Avalos, Junrui Pan, Timothy Rogers
- Large Language Models: A Plain-Language Overview for EngineersPapers · 25 Oct 2026 · Bahaeddin Türkoğlu
- INTRA: Interleaved Non-contiguous Token spaRse AttentionPapers · 25 Oct 2026 · Yuxuan Xia, Wenbo Lu, Shaoyi Zheng, Shengjie Wang
- LazyFlash: Forward-Dense, Backward-Sparse Attention for Accelerated Long-Context Fine-TuningPapers · 22 Oct 2026 · Keqi Deng, Ruchao Fan, Jinyu Li