phenomenon · first seen 5 Oct 2026
time to first token
Also: TTFT
The latency from submitting a request to receiving the first generated token, a key metric for LLM serving.
Mentions per week
PapersCodeDiscussion
Earliest sources
- Request Order Matters: Cache-History Sensitivity in Selective KV-Cache Reuse for Rolling AgentsPapers · 5 Oct 2026 · Tiffany Gu, Annie Guan, Manshu Huang, Nitin Rao, Siddhant Shah, Margaret Capetz
- KVCompass: A Discovery Plane for Internet-Scale Prefix Cache Sharing in LLM ServingPapers · 16 Oct 2026 · Shiyi Ling, Fei Fang, Chen Qian
- NEETnina/dsh-model-masterCode · 26 Oct 2026 · NEETnina · 21 stars
Latest sources
- NEETnina/dsh-model-masterCode · 26 Oct 2026 · NEETnina · 21 stars
- KVCompass: A Discovery Plane for Internet-Scale Prefix Cache Sharing in LLM ServingPapers · 16 Oct 2026 · Shiyi Ling, Fei Fang, Chen Qian
- Request Order Matters: Cache-History Sensitivity in Selective KV-Cache Reuse for Rolling AgentsPapers · 5 Oct 2026 · Tiffany Gu, Annie Guan, Manshu Huang, Nitin Rao, Siddhant Shah, Margaret Capetz