method · first seen 13 Oct 2026
token-level scheduling
Admitting and pacing requests at per-token granularity to control inter-token latency during LLM decoding.
Mentions per week
PapersCodeDiscussion
Earliest sources
- Stable Token-level Scheduling for Serverless LLM InferencePapers · 13 Oct 2026 · Hardi Ashishbhai Desai, Balaji N K, Sourav Kanti Addya
Latest sources
- Stable Token-level Scheduling for Serverless LLM InferencePapers · 13 Oct 2026 · Hardi Ashishbhai Desai, Balaji N K, Sourav Kanti Addya