method · first seen 4 Oct 2026
KV cache reuse
Also: KV-sharing, cache reuse, prompt cache, prefix cache, prefix reuse, prefix caching, prompt caching
Reusing key-value attention states computed by one model's prefill to speed up inference in another model.
KV cache reuse means keeping the key-value attention states from an earlier prefill and using them again instead of recomputing them. Common forms are prefix caching in serving engines and the prompt caching that vendors bill for. The sources mostly study refinements of this practice. One paper finds that selective non-prefix reuse in rolling agents depends on request history. A Korean paper compares restoring evicted prefixes from host memory against recomputing them on a static-graph NPU. A security paper looks at cross-tenant cache-hit leakage. Several GitHub repos are Claude Code and DeepSeek utilities for managing prompt-cache expiry and cost. None of the sources shows the cross-model reuse named in the description.
Long-running agents and shared-context workloads make cache reuse central to serving cost and latency. Open questions about correctness, side channels and memory tiering keep producing incremental research, but the underlying technique has been standard practice since 2023-2024.
Analyst note (established), 17 Nov 2026.
Mentions per week
PapersCodeDiscussion
Rank 172 on 14 Nov 2026 · 59 mentions in 28 days · 59 independent groups · code → papers · first seen 6 weeks ago · acceleration -0.03
Earliest sources
- obie/auto-handoffCode · 4 Oct 2026 · obie · 106 stars
- devereuxhale6/openrouter-docs-communityCode · 4 Oct 2026 · devereuxhale6 · 41 stars
- Request Order Matters: Cache-History Sensitivity in Selective KV-Cache Reuse for Rolling AgentsPapers · 5 Oct 2026 · Tiffany Gu, Annie Guan, Manshu Huang, Nitin Rao, Siddhant Shah, Margaret Capetz
- RealtimeWAM: One-Step Asynchronous World Action ModelsPapers · 5 Oct 2026 · Chengtao Lv, Jinyang Du, Shuyi Feng, Yang Yong, Shiqiao Gu, Shunzi Yang, Ruihao…
- Towards Looped Models Done Right, Part II: Rethinking at Fixed PointsPapers · 5 Oct 2026 · Benhao Huang, Chufan Shi, Junlin Chen, Shicheng Wen, Zhengzhong Liu, Eric Xing,…
- paulbkim-dev/claude-code-cache-warmerCode · 5 Oct 2026 · paulbkim-dev · 26 stars
Latest sources
- Re-Reading Is Not Free: Token Traces and Cache-Aware Energy Estimation for LLM Agents in Software EngineeringPapers · 16 Nov 2026 · Tina Vartziotis, Ketevan Peranidze, Rodopi Kosteli, Ezgi Sarikayak, George Dasou…
- Where the Savings Go: The Limits of Hiding Cross-Tenant Cache Hits in LLM ServingPapers · 15 Nov 2026 · Bowen Sun, Chaowei Xiao
- StreamEmbed: Real-Time Indexing from Pixels for Screenshot RetrievalPapers · 13 Nov 2026 · Dayoon Ko, Yichuan Wang, Soyeong Jeong, Joseph E. Gonzalez, Gunhee Kim, Sewon Mi…
- HeMeR: Heterogeneous Memory Reconciliation for Embodied Agents via Structured KV ReusePapers · 12 Nov 2026 · Saehun Chun, Wonje Choi, Jinwoo Jang, Taeyoon Kwack, Honguk Woo
- wquguru/fable52-fps-reproCode · 11 Nov 2026 · wquguru · 106 stars
- Read Once, Answer Many: Parallel Prompting for Shared-Context LLM InferencePapers · 11 Nov 2026 · Zekun Zhao, Jeffrey Flanigan
- Recompute or Restore? Host-Tier KV for Multi-Model Agent Workflows on a Static-Graph NPU Serving StackPapers · 11 Nov 2026 · Sungwon Woo, Seyeon Kim
- Can KV Cache Be Reused Across Model Updates? A DeepSeek-V4-Flash Case StudyPapers · 10 Nov 2026 · Zhikun Tao, Fangyong Jin, Junping Zhao
- Bursts That Outlast Themselves: Persistent Degradation in Prefix-Cached Agent ServingPapers · 10 Nov 2026 · Wen Huang, Jack Liu
- jeremiebergel/context-manager-intelligenceCode · 9 Nov 2026 · jeremiebergel · 10 stars
- Neal006/dejaCode · 9 Nov 2026 · Neal006 · 15 stars
- Looping Deeper Without Growing the KV Cache: Recurrent Depth for Causal Encoder-DecodersPapers · 9 Nov 2026 · Jiaqi Leng, Minshen Zhang, Zhaofeng Sun, Yu-Yang Qian, Peng Zhao, Hao Zhang