Early-Signal Detector

method · first seen 4 Oct 2026

KV cache reuse

Also: KV-sharing, cache reuse, prompt cache, prefix cache, prefix reuse, prefix caching, prompt caching

Reusing key-value attention states computed by one model's prefill to speed up inference in another model.

KV cache reuse means keeping the key-value attention states from an earlier prefill and using them again instead of recomputing them. Common forms are prefix caching in serving engines and the prompt caching that vendors bill for. The sources mostly study refinements of this practice. One paper finds that selective non-prefix reuse in rolling agents depends on request history. A Korean paper compares restoring evicted prefixes from host memory against recomputing them on a static-graph NPU. A security paper looks at cross-tenant cache-hit leakage. Several GitHub repos are Claude Code and DeepSeek utilities for managing prompt-cache expiry and cost. None of the sources shows the cross-model reuse named in the description.

Long-running agents and shared-context workloads make cache reuse central to serving cost and latency. Open questions about correctness, side channels and memory tiering keep producing incremental research, but the underlying technique has been standard practice since 2023-2024.

Analyst note (established), 17 Nov 2026.

Mentions per week

01519 May11 Aug10 NovPapersCodeDiscussion

PapersCodeDiscussion

Rank 172 on 14 Nov 2026 · 59 mentions in 28 days · 59 independent groups · code → papers · first seen 6 weeks ago · acceleration -0.03

Earliest sources

Latest sources

Mentioned together with

retrieval-augmented generationcontext compactionLLM routingselective recomputationlooped language modelspeculative decodingtime to first tokenworld action modelClaude CodeCUDA graph