benchmark · first seen 13 Nov 2026
time between tokens
Also: TBT
Latency metric measuring the gap between consecutive generated tokens in streaming LLM responses.
Mentions per week
PapersCodeDiscussion
Earliest sources
- VelServe: Taming Decode Jitters in LLM Inference with Tail-Calibrated SchedulingPapers · 13 Nov 2026 · Ishaan Datta, Alind Khare, Koushik Kumar Dey, Ratnesh Jamidar, Anand Kumar Singh
Latest sources
- VelServe: Taming Decode Jitters in LLM Inference with Tail-Calibrated SchedulingPapers · 13 Nov 2026 · Ishaan Datta, Alind Khare, Koushik Kumar Dey, Ratnesh Jamidar, Anand Kumar Singh