phenomenon · first seen 13 Oct 2026
CPU-based LLM inference
Also: CPU inference, CPU-only inference, CPU inference for large language models
Running large language models on system RAM and CPUs instead of GPU memory, trading throughput for capacity.
Mentions per week
PapersCodeDiscussion
Earliest sources
- Capacity over TFLOPS: Memory-Centric Deployment of Large Clinical Language Models on Consumer CPUsPapers · 13 Oct 2026 · Eli Yune, Umer Majeed, Mahesh Balan
- Indra OCR: a 1M-parameter OCR model that reads 49 pages per second on CPUDiscussion · 13 Oct 2026 · runtimewire · 39 points
- synchopate/llama-modchipCode · 29 Oct 2026 · synchopate · 1,333 stars
Latest sources
- synchopate/llama-modchipCode · 29 Oct 2026 · synchopate · 1,333 stars
- Indra OCR: a 1M-parameter OCR model that reads 49 pages per second on CPUDiscussion · 13 Oct 2026 · runtimewire · 39 points
- Capacity over TFLOPS: Memory-Centric Deployment of Large Clinical Language Models on Consumer CPUsPapers · 13 Oct 2026 · Eli Yune, Umer Majeed, Mahesh Balan