method · first seen 13 Oct 2026
chunked prefill
Also: time to first token optimization
Splitting long prompt prefill into chunks interleaved with decoding steps to reduce latency interference.
Mentions per week
PapersCodeDiscussion
Earliest sources
- Stable Token-level Scheduling for Serverless LLM InferencePapers · 13 Oct 2026 · Hardi Ashishbhai Desai, Balaji N K, Sourav Kanti Addya
- TideKV: Delayed KV Eviction via Layer-Wise Chunked Prefill for Memory-Efficient Long-Context LLM InferencePapers · 17 Oct 2026 · Yu Wang, Zhu Wang, Ning Hu, Yifan Guo, Xiu Ma, Bin Guo, Zhiwen Yu
- Raincheck: Deferred Multimodal Prefill for Efficient Multi-Turn MLLM InferencePapers · 29 Oct 2026 · Wenjie Huang, Jun Zhang, Hongyaoxing Gu, Haoyang Huang, Tianqi Xu, Xinyue Hao, X…
- A Controlled Experimental Evaluation of Configuration and Cache Architecture in LLM Serving EnginesPapers · 4 Nov 2026 · Muhammad Faizan Anwar, Sina Keshvadi
Latest sources
- A Controlled Experimental Evaluation of Configuration and Cache Architecture in LLM Serving EnginesPapers · 4 Nov 2026 · Muhammad Faizan Anwar, Sina Keshvadi
- Raincheck: Deferred Multimodal Prefill for Efficient Multi-Turn MLLM InferencePapers · 29 Oct 2026 · Wenjie Huang, Jun Zhang, Hongyaoxing Gu, Haoyang Huang, Tianqi Xu, Xinyue Hao, X…
- TideKV: Delayed KV Eviction via Layer-Wise Chunked Prefill for Memory-Efficient Long-Context LLM InferencePapers · 17 Oct 2026 · Yu Wang, Zhu Wang, Ning Hu, Yifan Guo, Xiu Ma, Bin Guo, Zhiwen Yu
- Stable Token-level Scheduling for Serverless LLM InferencePapers · 13 Oct 2026 · Hardi Ashishbhai Desai, Balaji N K, Sourav Kanti Addya