Best open-source local LLM runners for Docker, enterprise

Open-source runtimes for running LLMs on your own hardware.

Filters · 3 Enterprise, Local inference, Docker / self-host

For

Personal2Enterprise5

Purpose

AI & LLM tooling65Chat UIs5Agents & frameworks29Local inference5RAG & vector DBs17Coding assistants3Evals & observability7Agent skills0MCP servers4Developer tools59Web frameworks16Databases & backends30Editors & IDEs1API tools6Self-hosted apps33Notes & docs6Automation23Data & analytics23Infra & DevOps42Security & identity19Communication15Media & photos6Design & creative1

Runs on

Web1Docker / self-host5Desktop1Mobile1CLI2Library / SDK8Extension / plugin0

License

Permissive5Copyleft0Source-available0Other / custom0

More

Has live demo0Commit in last 90 days5

5 projects

SortWorth scoreTrendingMost starsRecently updatedNewest in catalog

O

Ollamaollama/ollama

Run open-weight LLMs locally with one commandPersonalEnterpriseLocal inferenceDocker / self-hostDesktopCLI★ 182kMITcommit 17 hours ago

90worth

☆ Save

v

vLLMvllm-project/vllm

High-throughput LLM serving engineEnterpriseLocal inferenceDocker / self-hostLibrary / SDK★ 93kApache-2.0commit 9 hours ago

86worth

☆ Save

L

LocalAImudler/LocalAI

Drop-in OpenAI API replacement that runs locallyPersonalEnterpriseLocal inferenceDocker / self-host★ 49kMITcommit 13 hours ago

86worth

☆ Save

P

PrivateGPTzylon-ai/private-gpt

Private AI API over local modelsEnterpriseLocal inferenceRAG & vector DBsDocker / self-host★ 58kApache-2.0commit 2 weeks ago

83worth

☆ Save

S

SGLangsgl-project/sglang

Fast serving framework for LLMsEnterpriseLocal inferenceDocker / self-hostLibrary / SDK★ 37kApache-2.0commit 9 hours ago

81worth

☆ Save