architecture · first seen 5 Oct 2026
KV cache
Also: KV memory, KV caching, key-value cache, key-value caching, key-value cache memory, on-die key-value cache, key-value cache sharing
Stored attention keys and values from previous tokens, reused during autoregressive decoding.
The KV cache stores the attention keys and values computed for earlier tokens so a transformer can generate each new token without recomputing the whole prefix. It has been a standard part of LLM inference for years. The sources show it as background infrastructure rather than a new idea. Recent papers on agentic serving handle prefix-cache loss when routing between models, KV-cache translation across hybrid-attention architectures, and cache sharing in depth-recurrent models. Other sources include a chip co-design paper, local-inference tuning guides, a GPU monitor that reports KV-cache usage, and a Cloudflare post on cross-request KV cache exposure.
Most of the current activity concerns managing, sharing and securing the KV cache in multi-agent and multi-model serving. That makes cache handling a central systems constraint, not a new concept, and the weekly counts are already falling.
Analyst note (established), 17 Nov 2026.
Mentions per week
PapersCodeDiscussion
Rank 57 on 14 Nov 2026 · 52 mentions in 28 days · 51 independent groups · papers → code → discussion · first seen 6 weeks ago · acceleration -0.31
Earliest sources
- Can Agent Harnesses and Inference Engines Hear Each Other? The HEAR Protocol for Agentic LLM ServingPapers · 5 Oct 2026 · Jiaqi Zhao, Haodong Chen, Jitai Hao, Wei Zhao, Jinghao Pang, Qiang Huang, Jun Yu
- lastreload/mnemonic-proxyCode · 5 Oct 2026 · lastreload · 75 stars
- Persistent Memory in Multi-Agent LLM Inference: What It Costs, What It Buys, and When You Can TellPapers · 6 Oct 2026 · Hochan Son, Kyungdoe Han, Jaehan Koh, Xiaowu Dai, Wenlu Xu, Guang Cheng
- ReFold: Training-Free Reversible Inter-Turn Context Folding for Long-Horizon AgentsPapers · 6 Oct 2026 · Yupeng Su, Jiayi Tian, Zheng Zhang, Souvik Kundu
- Lachesis: Lifetime-Aware KV Cache Placement for Agent Serving across HBM and High-Bandwidth FlashPapers · 6 Oct 2026 · Jaehoon Yang, Jeongmin Lee, Haneul Park, Seung Yul Lee, Nam Sung Kim, Jae W. Lee
- Enabling Dynamic Computation in Looped LMsPapers · 6 Oct 2026 · Aayush Mishra, Arnau Padrés Masdemont, Victor Conchello Vendrell, Jordi Ros Gir…
Latest sources
- Reference Forcing: Long Video Generation with Region-Guided Drift CorrectionPapers · 16 Nov 2026 · Sheng Zhao, Haitian Zheng, Weikai Lin, Junjie Zhao, Yuhao Zhu
- Latent Bridge: Asking for Help Without WordsPapers · 16 Nov 2026 · Qichao Wang, Tingyu Qu, Pengcheng Wu, Yiran Zhong, Feida Zhu, Zhiqi Shen, Peilin…
- ScoutCache: Efficient Few-Step Autoregressive Video Diffusion via Conditional Readout and Selective CachingPapers · 16 Nov 2026 · Hao Tan, Jinghui Qin, Xuming Hu
- Multi-Branch TransformerPapers · 15 Nov 2026 · Gaotang Li, Lingjie Chen, Chunyuan Deng, Zhiwei Xu, Hongzheng Chen, Christopher…
- Architectures and KV Replay for Depth-Recurrent Language ModelsPapers · 14 Nov 2026 · Lucia Licakova, Niklas Schmidinger, Lukas Hauzenberger, Sebastian Böck, Sepp Ho…
- Exstill: The Most Efficient Single-Chip Processor for the Next TokenPapers · 13 Nov 2026 · Valavan Manohararajah, Vithursan Thangarasa
- Reducing Batch Makespan for LLM Agent Workloads through Inference-Environment CoordinationPapers · 12 Nov 2026 · Wei Feng, Meiling Chen, Boxiao Chen, Lu Liu, Bin Zi, Libin Ye, Xingyu Lu, Lei Xu…
- SHADOWPREFILL: Keeping Your Next Model Warm for Per-Step Model Routing in Agentic LLM ServingPapers · 9 Nov 2026 · Haoran Qiu, Rohan Mahapatra, Rodrigo Fonseca, Mark Russinovich
- Translating KV Caches Between Hybrid-Attention Language Models: MLA Latents, Linear-Attention States and Sparse-Attention IndexersPapers · 9 Nov 2026 · Povilas Kvedaras, Tomas Rasymas
- BridgeKV: Large-Model Prefill and Small-Model Decoding for LLM Inference via KV Cache TranslationPapers · 9 Nov 2026 · Payman Behnam, Debopam Sanyal, Pranav Ponnusamy, Kusum Meda Ravi, Pranav Sastry,…
- EASE-KV: Diagnosing the Gap Between Memory Availability and Effective UsePapers · 8 Nov 2026 · Di Lu, Zechen Li, Xuanyu Yue, Jiabo Fan, Hao Xue, Flora D. Salim
- Coverage-Verified Sparse Attention: Closed-Loop Quality Control for Long-Context LLM DecodingPapers · 7 Nov 2026 · Yan-Cheng Li, Chun-Yi Lee