method · first seen 7 Oct 2026
KV cache offloading
Also: KV cache offload, KV cache storage, KV cache spilling, host-tier KV cache, KV state persistence, remote KV cache tier, hierarchical KV caching
Saving transformer key-value cache state to disk and reloading it later instead of recomputing it for long prompts.
Mentions per week
PapersCodeDiscussion
Rank 105 on 16 Nov 2026 · 12 mentions in 28 days · 12 independent groups · papers → code · first seen 6 weeks ago · acceleration -0.34
Earliest sources
- Real Long-Term Memory for AI: A 50-Million-Token Window That Is Faster and Cheaper Than RecomputePapers · 7 Oct 2026 · Sietse Schelpe
- Compositional Performance Modeling of LLM Inference Clusters with Hierarchical KV CachingPapers · 11 Oct 2026 · Ruiyang Ma, Sheng Qi, DaiKang Kuang, Teng Ma, Guojie Luo
- When the KV Cache Is Slower Than the GPU: Scaling Remote KV Tiers for Context-Parallel PrefillPapers · 15 Oct 2026 · Muhammad Ibrahim
- CSUNX233/Sparkle-Qwen27b-Kvmem-ninferCode · 15 Oct 2026 · CSUNX233 · 103 stars
- Leonccaa/ReVoltaCode · 20 Oct 2026 · Leonccaa · 61 stars
- Gandalf: Flash Shall Not Die! Increasing Flash Storage Lifetime for KV Cache Offloading in LLM ServingPapers · 22 Oct 2026 · Maharani A. P. Irawan, Kuntai Du, Jiayi Yao, Dongjoo Seo, Yuyang Huang, Haryadi…
Latest sources
- Making Preemption Productive in Long-Output LLM ServingPapers · 15 Nov 2026 · Ziji Shi, Xinyi Wan, Chaoyi Ruan, Guangxing Huang, Min Lin, Jialin Li
- Recompute or Restore? Host-Tier KV for Multi-Model Agent Workflows on a Static-Graph NPU Serving StackPapers · 11 Nov 2026 · Sungwon Woo, Seyeon Kim
- Placing LLM Lookup Tables by Per-Layer Prefetch DeadlinesPapers · 10 Nov 2026 · Wen Huang, Jack Liu
- Verify Before You Fetch: Reliable KV Cache Compression for LLM InferencePapers · 8 Nov 2026 · Devanshu Jha, Gaurav Kumar, Agniva Chowdhury, Kumar Krishna Agrawal, Aditya Desa…
- FlashOffloading: Low-Latency KV Cache Offloading for Long-Context LLM ServingPapers · 5 Nov 2026 · Xuan Wang, Zhuanhao Wu, Tingzhu Fang, Xushi Li, Sicheng Zhang, Zhenyu Yang, Feng…
- LeanSparse: Memory-Efficient Serving for Native Sparse-Attention LLMsPapers · 3 Nov 2026 · Zaifeng Pan, Chris Wu, Xinwei Qiang, Zhengding Hu, Zhongkai Yu, Yufei Ding
- FlatCake: A Flat KVCache Storage Architecture for LLM ServingPapers · 28 Oct 2026 · Xin Huang, Teng Ma, Zhuoran Li, Yibo Zhao, Zhaoshi Li, Qianchao Zhu, Wei Yang, C…
- WorldPager: Zero-Stall Tiered Memory for Long-Horizon World Models on a Single Consumer GPUPapers · 26 Oct 2026 · Bingnan Liu, Ruini Xue
- DualPace: Decoupled Scheduling and Cooperative GPU--CPU Decoding for LLM ServingPapers · 24 Oct 2026 · YuXuan Lin, KeQiang Wu, Kingsum Chow
- KVStream: Adaptive Parallel Storage Management for KV Caching in Large-Scale LLM ServingPapers · 22 Oct 2026 · Avinash Maurya, Jaiaid Mobin, Jie Ye, M. Mustafa Rafique, Franck Cappello, Bogda…
- Gandalf: Flash Shall Not Die! Increasing Flash Storage Lifetime for KV Cache Offloading in LLM ServingPapers · 22 Oct 2026 · Maharani A. P. Irawan, Kuntai Du, Jiayi Yao, Dongjoo Seo, Yuyang Huang, Haryadi…
- Leonccaa/ReVoltaCode · 20 Oct 2026 · Leonccaa · 61 stars