method · first seen 4 Oct 2026
self-hosted LLM deployment
Also: edge AI, local AI, edge LLM, local LLM, offline AI, LLM decode, LLM inference, on-device LLM
Running language models on private local hardware instead of calling a cloud provider.
Self-hosted or local LLM deployment means running language models on hardware you control, such as phones, workstations, private GPU servers or air-gapped machines, instead of calling a cloud API. The sources are a loose cluster. Hobbyist repos ship on-device Android agents, from-scratch inference engines for AMD Strix Halo and NVIDIA Blackwell, a multi-GPU server for very large mixture-of-experts models, and offline survival assistants. Papers cover offline voice AI for African languages, brain-to-text decoding on mobile devices, sparse diffusion inference on mobile SoCs, and satellite and IoT edge AI. The merged aliases (edge AI, LLM decode) are broader than the concept itself.
Local inference has been mainstream since llama.cpp and Ollama. The activity here reflects steady work on new hardware targets and quantization rather than a new idea. The flat growth and negative acceleration fit a long-standing topic, not an emerging one.
Analyst note (established), 17 Nov 2026.
Mentions per week
PapersCodeDiscussion
Rank 30 on 14 Nov 2026 · 83 mentions in 28 days · 83 independent groups · papers → code → discussion · first seen 6 weeks ago · acceleration +0.09
Earliest sources
- Offline AI Modules: Voice-First Offline Architecture, Hardware Reference Stack, Quantization and BenchmarkingPapers · 4 Oct 2026 · Sunday Afariogun, Odunolaoluwa Jenrola, Zeinab Nezami
- omarqaterge/hermes-mobile-appCode · 4 Oct 2026 · omarqaterge · 156 stars
- shawnshekari/strixiteCode · 4 Oct 2026 · shawnshekari · 106 stars
- WhitePepperLambSoup/agent-workspace-androidCode · 4 Oct 2026 · WhitePepperLambSoup · 29 stars
- ninja8170/VoxAsterCode · 4 Oct 2026 · ninja8170 · 26 stars
- mw00/project-mayaCode · 6 Oct 2026 · mw00 · 2,983 stars
Latest sources
- LaRazaDelSol/AIEngineOnSmallUniMemCode · 16 Nov 2026 · LaRazaDelSol · 11 stars
- A Generative AI-Driven Architecture for Whole-Home Intelligent Services: Edge Models, Memory Management, and Household ApplicationsPapers · 16 Nov 2026 · Mo Zhou
- System 1 agents: a method for local, reliable and fast agentic AIPapers · 16 Nov 2026 · Eleftherios Tzanis, Michail E. Klontzas
- shani-hamer/riverse-ai-agentCode · 15 Nov 2026 · shani-hamer · 55 stars
- hikaku-io/hikaku-extensionCode · 14 Nov 2026 · hikaku-io · 402 stars
- Detecting Small Space Objects for Space Situational Awareness with Collaborative Edge AIPapers · 14 Nov 2026 · Xingyu Qu, Peng Hu
- dit.xpu : Taming DiT Sparse Inference on Mobile SoCsPapers · 14 Nov 2026 · Jiayu Chen, Jinye Hu, Maoliang Li, Anqian Cen, Hailong Zou, Zihao Zheng, Xinming…
- From Sensors to Silicon: Causal Cross-Layer Self-Healing Edge AI for Secure and Energy-Adaptive IoT SystemsPapers · 14 Nov 2026 · Mostafa Darvishi
- FETTUCCINE: Fast and efficient brain-to-text decoding on mobile devicesPapers · 13 Nov 2026 · Jonathan D. McCart, Pranav I. Deevi, Mehdi Azabou, Nanda H. Krishna, Caleb A. Mc…
- Superfluid: Local AI Server for Distributed LLM InferencePapers · 13 Nov 2026 · Prabod Rathnayaka, Isura Manchanayake, Srikandabala Kogul, Steve Siu, Fabian Was…
- Flying without knowing how the plane worksDiscussion · 12 Nov 2026 · FriendlyMike · 40 points
- leonardo-alegria/garlic-llmCode · 11 Nov 2026 · leonardo-alegria · 146 stars