vLLMvllm-project/vllm
High-throughput LLM serving engineEnterpriseLocal inferenceDocker / self-hostLibrary / SDK★ 93kApache-2.0commit 16 hours ago86worth
Open-source alternatives to SGLang
24 projects in the catalog share a category with SGLang (Local inference). They are ranked by how closely their categories, platforms and GitHub topics match, then by worth score. Of these, 5 run with Docker, 17 have a permissive license and 24 had a commit in the last 90 days.
Compare at a glance
| Project | Runs on | Self-host with Docker | License | Stars | Last commit |
|---|---|---|---|---|---|
| SGLang (this project) | Docker / self-host, Library / SDK | Yes | Apache-2.0 | 37k | |
| vLLM | Docker / self-host, Library / SDK | Yes | Apache-2.0 | 93k | |
| Ollama | Docker / self-host, Desktop, CLI | Yes | MIT | 182k | |
| llama.cpp | CLI, Library / SDK | Not listed | MIT | 130k | |
| LocalAI | Docker / self-host | Yes | MIT | 49k | |
| PrivateGPT | Docker / self-host | Yes | Apache-2.0 | 58k | |
| Unsloth | Web, Library / SDK | Not listed | Apache-2.0 | 77k | |
| OpenVINO | Library / SDK | Not listed | Apache-2.0 | 11k | |
| whisper.cpp | CLI, Library / SDK | Not listed | MIT | 54k | |
| AirLLM | Library / SDK | Not listed | Apache-2.0 | 35k | |
| RunAnywhere SDKs | Mobile, Library / SDK | Not listed | Custom (read before use) | 10k | |
| ncnn | Library / SDK | Not listed | Custom (read before use) | 24k | |
| BitNet | Library / SDK | Not listed | MIT | 40k | |
| oMLX | Desktop | Not listed | Apache-2.0 | 22k | |
| Colibri | CLI | Not listed | Apache-2.0 | 39k | |
| llmfit | CLI | Not listed | MIT | 37k | |
| Jan | Desktop | Not listed | Custom (read before use) | 45k | |
| Stability Matrix | Desktop | Not listed | AGPL-3.0 | 8.9k | |
| llamafile | CLI | Not listed | Custom (read before use) | 26k | |
| Text Generation WebUI | Web, Desktop | Not listed | AGPL-3.0 | 48k | |
| Mochi Diffusion | Desktop | Not listed | GPL-3.0 | 8k | |
| exo | CLI | Not listed | Apache-2.0 | 48k | |
| SwarmUI | Web | Not listed | MIT | 4.6k | |
| ds4 | CLI | Not listed | MIT | 23k | |
| Crawl4AI | Docker / self-host, Library / SDK | Yes | Apache-2.0 | 85k |
Self-hosted Docker apps (5)
Ollamaollama/ollama
Run open-weight LLMs locally with one commandPersonalEnterpriseLocal inferenceDocker / self-hostDesktopCLI★ 182kMITcommit 23 hours ago90worth
LocalAImudler/LocalAI
Drop-in OpenAI API replacement that runs locallyPersonalEnterpriseLocal inferenceDocker / self-host★ 49kMITcommit 19 hours ago86worth
PrivateGPTzylon-ai/private-gpt
Private AI API over local modelsEnterpriseLocal inferenceRAG & vector DBsDocker / self-host★ 58kApache-2.0commit 2 weeks ago83worth
Crawl4AIunclecode/crawl4ai
Web crawler that turns sites into LLM-ready MarkdownPersonalEnterpriseRAG & vector DBsDocker / self-hostLibrary / SDK★ 85kApache-2.0commit last week88worth
Libraries & SDKs (8)
llama.cppggml-org/llama.cpp
LLM inference in C/C++ on CPUs and GPUsPersonalLocal inferenceCLILibrary / SDK★ 130kMITcommit 17 hours ago86worth
Unslothunslothai/unsloth
Fast local fine-tuning and running of LLMsPersonalEnterpriseLocal inferenceWebLibrary / SDK★ 77kApache-2.0commit 16 hours ago81worth
OpenVINOopenvinotoolkit/openvino
Optimize and deploy AI inferenceEnterpriseLocal inferenceLibrary / SDK★ 11kApache-2.0commit yesterday80worth
whisper.cppggml-org/whisper.cpp
Whisper speech recognition in C/C++PersonalEnterpriseLocal inferenceMedia & photosCLILibrary / SDK★ 54kMITcommit yesterday79worth
AirLLMlyogavin/airllm
Run 70B models on a 4GB GPUPersonalLocal inferenceLibrary / SDK★ 35kApache-2.0commit 2 days ago73worth
RunAnywhere SDKsRunanywhereAI/runanywhere-sdks
Run AI locally on devicesPersonalEnterpriseLocal inferenceMobileLibrary / SDK★ 10kcommit 2 weeks ago70worth
ncnnTencent/ncnn
Neural network inference for mobilePersonalEnterpriseLocal inferenceLibrary / SDK★ 24kcommit 2 days ago66worth
BitNetmicrosoft/BitNet
Inference framework for 1-bit LLMsPersonalEnterpriseLocal inferenceLibrary / SDK★ 40kMITcommit 2 months ago63worth
Web apps (2)
Text Generation WebUIoobabooga/textgen
Desktop app for running local LLMsPersonalChat UIsLocal inferenceWebDesktop★ 48kAGPL-3.0commit 2 months ago69worth
SwarmUImcmonkeyprojects/SwarmUI
Modular Stable Diffusion web UIPersonalLocal inferenceMedia & photosWeb★ 4.6kMITcommit 2 days ago64worth
Desktop apps (4)
oMLXjundot/omlx
LLM server for Apple SiliconPersonalLocal inferenceDesktop★ 22kApache-2.0commit yesterday80worth
Janjanhq/jan
Offline ChatGPT alternative that runs on your computerPersonalChat UIsLocal inferenceDesktop★ 45kcommit yesterday73worth
Stability MatrixLykosAI/StabilityMatrix
Package manager for Stable Diffusion UIsPersonalLocal inferenceMedia & photosDesktop★ 8.9kAGPL-3.0commit 3 days ago72worth
Mochi DiffusionMochiDiffusion/MochiDiffusion
Stable Diffusion on MacPersonalLocal inferenceMedia & photosDesktop★ 8kGPL-3.0commit 4 days ago68worth
Command-line tools (5)
ColibriJustVugg/colibri
Run large MoE models on everyday hardwarePersonalLocal inferenceCLI★ 39kApache-2.0commit last week79worth
llmfitAlexsJones/llmfit
Find which LLMs run on your hardwarePersonalLocal inferenceCLI★ 37kMITcommit 20 hours ago79worth
llamafilemozilla-ai/llamafile
Run an LLM from a single filePersonalLocal inferenceCLI★ 26kcommit 3 days ago71worth
exoexo-explore/exo
Run frontier AI models across your own devicesPersonalLocal inferenceCLI★ 48kApache-2.0commit last month67worth
63worth