Free
Added on Nov 09,2026
Self-Hosted AI Stack is an open-source project by Lin Song for running a private AI platform on infrastructure you control. It lets you run local AI models, chat with your documents, and transcribe or generate speech on your own Linux server, either locally or in the cloud.
The stack is deployed with Docker Compose. It supports optional HTTPS and optional NVIDIA GPU acceleration, and CPU-only configurations are also available. Memory and performance needs depend on the services and models you choose.
Key components include InferCrate for running local models (powered by Ollama), AnythingLLM for browser-based chat, ParseCrate for document parsing (powered by Docling) and EmbedCrate for embeddings and reranking in semantic search and retrieval.
For audio, ScribeCrate handles transcription and SpeakCrate generates speech using Kokoro. GatewayCrate routes model requests through LiteLLM, and UplinkCrate connects supported tools through MCP.
Fresh deployments generate API keys and set up password protection for AnythingLLM by default. You can start with a preconfigured lightweight stack and only the services you need, or customise the Compose configuration.
The site notes that it is not a hosted AI service. The website provides project information, a free local AI readiness guide, and a link to the GitHub repository with architecture details and deployment instructions. A companion book, The Self-Hosted AI Builder's Guide, covers deployment, security, backups and day-to-day operation.
Pricing: the stack's configuration is free and MIT licensed. Included services and models keep their own licences, and hardware, cloud hosting or external providers may involve costs.
It is suited to developers, self-hosters and teams who want local, private AI for chat, document retrieval, transcription and text-to-speech without relying on a hosted AI provider.
By the PowerfulAI.tools Editorial Team · Updated 10 November 2026 · Researched with AI assistance and checked against the official Self-Hosted AI Stack website · How we review tools
Self-Hosted AI Stack is listed as free to use. Check Self-Hosted AI Stack's website for any usage limits or paid upgrades.
Sysadmins and privacy-minded developers who want to deploy local LLM chat, document retrieval and voice services on infrastructure they control.
The stack's configuration is open source under the MIT licence. It is maintained by Lin Song, who also sells a companion book, The Self-Hosted AI Builder's Guide.
Yes. You can pick a preconfigured lightweight stack or customise the Compose configuration to run only what you need.
Popular alternatives to Self-Hosted AI Stack include Pollinations, NemoClaw and Agnes AI. Compare their pricing and what each is best for below.
| Tool | Pricing | Best for |
|---|---|---|
| Pollinations | Freemium | Developers and hobby builders who want a single API for adding AI images, audio and text to their websites, bots and app... |
| NemoClaw | Free | Developers and platform teams who want to run always-on AI agents on NVIDIA hardware under policy-based security control... |
| Agnes AI | Freemium | Developers building AI-powered apps who want access to several models through one API. |
| OrcaRouter | Freemium | Developers and teams who want to balance AI quality and cost across many models. |
| DiagramGPT | Free | Developers and students who need to sketch system designs and workflows quickly by describing them in words. |
Own Self-Hosted AI Stack? Add this badge to your website.
Leave a review for the community
No Review Found
