Ollamaollama/ollama
Run open-weight LLMs locally with one commandPersonalEnterpriseLocal inferenceDocker / self-hostDesktopCLI★ 182kMITcommit 13 hours ago90worth
Open-source alternatives to llmfit
24 projects in the catalog share a category with llmfit (Local inference). They are ranked by how closely their categories, platforms and GitHub topics match, then by worth score. Of these, 5 run with Docker, 17 have a permissive license and 24 had a commit in the last 90 days.
Compare at a glance
| Project | Runs on | Self-host with Docker | License | Stars | Last commit |
|---|---|---|---|---|---|
| llmfit (this project) | CLI | Not listed | MIT | 37k | |
| Ollama | Docker / self-host, Desktop, CLI | Yes | MIT | 182k | |
| llama.cpp | CLI, Library / SDK | Not listed | MIT | 130k | |
| whisper.cpp | CLI, Library / SDK | Not listed | MIT | 54k | |
| Colibri | CLI | Not listed | Apache-2.0 | 39k | |
| llamafile | CLI | Not listed | Custom (read before use) | 26k | |
| exo | CLI | Not listed | Apache-2.0 | 48k | |
| ds4 | CLI | Not listed | MIT | 23k | |
| LocalAI | Docker / self-host | Yes | MIT | 49k | |
| vLLM | Docker / self-host, Library / SDK | Yes | Apache-2.0 | 93k | |
| PrivateGPT | Docker / self-host | Yes | Apache-2.0 | 58k | |
| Unsloth | Web, Library / SDK | Not listed | Apache-2.0 | 77k | |
| SGLang | Docker / self-host, Library / SDK | Yes | Apache-2.0 | 37k | |
| oMLX | Desktop | Not listed | Apache-2.0 | 22k | |
| OpenVINO | Library / SDK | Not listed | Apache-2.0 | 11k | |
| Jan | Desktop | Not listed | Custom (read before use) | 45k | |
| AirLLM | Library / SDK | Not listed | Apache-2.0 | 35k | |
| Stability Matrix | Desktop | Not listed | AGPL-3.0 | 8.9k | |
| RunAnywhere SDKs | Mobile, Library / SDK | Not listed | Custom (read before use) | 10k | |
| Text Generation WebUI | Web, Desktop | Not listed | AGPL-3.0 | 48k | |
| Mochi Diffusion | Desktop | Not listed | GPL-3.0 | 8k | |
| ncnn | Library / SDK | Not listed | Custom (read before use) | 24k | |
| SwarmUI | Web | Not listed | MIT | 4.6k | |
| BitNet | Library / SDK | Not listed | MIT | 40k | |
| Gemini CLI | CLI | Not listed | Apache-2.0 | 107k |
Command-line tools (8)
llama.cppggml-org/llama.cpp
LLM inference in C/C++ on CPUs and GPUsPersonalLocal inferenceCLILibrary / SDK★ 130kMITcommit 7 hours ago86worth
whisper.cppggml-org/whisper.cpp
Whisper speech recognition in C/C++PersonalEnterpriseLocal inferenceMedia & photosCLILibrary / SDK★ 54kMITcommit 18 hours ago79worth
ColibriJustVugg/colibri
Run large MoE models on everyday hardwarePersonalLocal inferenceCLI★ 39kApache-2.0commit last week79worth
llamafilemozilla-ai/llamafile
Run an LLM from a single filePersonalLocal inferenceCLI★ 26kcommit 3 days ago71worth
exoexo-explore/exo
Run frontier AI models across your own devicesPersonalLocal inferenceCLI★ 48kApache-2.0commit last month67worth
63worth
Gemini CLIgoogle-gemini/gemini-cli
Gemini as an agent in your terminalPersonalEnterpriseCoding assistantsCLI★ 107kApache-2.0commit yesterday90worth
Web apps (3)
Unslothunslothai/unsloth
Fast local fine-tuning and running of LLMsPersonalEnterpriseLocal inferenceWebLibrary / SDK★ 77kApache-2.0commit 6 hours ago81worth
Text Generation WebUIoobabooga/textgen
Desktop app for running local LLMsPersonalChat UIsLocal inferenceWebDesktop★ 48kAGPL-3.0commit 2 months ago69worth
SwarmUImcmonkeyprojects/SwarmUI
Modular Stable Diffusion web UIPersonalLocal inferenceMedia & photosWeb★ 4.6kMITcommit 2 days ago64worth
Self-hosted Docker apps (4)
LocalAImudler/LocalAI
Drop-in OpenAI API replacement that runs locallyPersonalEnterpriseLocal inferenceDocker / self-host★ 49kMITcommit 9 hours ago86worth
vLLMvllm-project/vllm
High-throughput LLM serving engineEnterpriseLocal inferenceDocker / self-hostLibrary / SDK★ 93kApache-2.0commit 6 hours ago86worth
PrivateGPTzylon-ai/private-gpt
Private AI API over local modelsEnterpriseLocal inferenceRAG & vector DBsDocker / self-host★ 58kApache-2.0commit 2 weeks ago83worth
SGLangsgl-project/sglang
Fast serving framework for LLMsEnterpriseLocal inferenceDocker / self-hostLibrary / SDK★ 37kApache-2.0commit 5 hours ago81worth
Desktop apps (4)
oMLXjundot/omlx
LLM server for Apple SiliconPersonalLocal inferenceDesktop★ 22kApache-2.0commit 22 hours ago80worth
Janjanhq/jan
Offline ChatGPT alternative that runs on your computerPersonalChat UIsLocal inferenceDesktop★ 45kcommit yesterday73worth
Stability MatrixLykosAI/StabilityMatrix
Package manager for Stable Diffusion UIsPersonalLocal inferenceMedia & photosDesktop★ 8.9kAGPL-3.0commit 2 days ago72worth
Mochi DiffusionMochiDiffusion/MochiDiffusion
Stable Diffusion on MacPersonalLocal inferenceMedia & photosDesktop★ 8kGPL-3.0commit 3 days ago68worth
Mobile apps (1)
RunAnywhere SDKsRunanywhereAI/runanywhere-sdks
Run AI locally on devicesPersonalEnterpriseLocal inferenceMobileLibrary / SDK★ 10kcommit last week70worth
Libraries & SDKs (4)
OpenVINOopenvinotoolkit/openvino
Optimize and deploy AI inferenceEnterpriseLocal inferenceLibrary / SDK★ 11kApache-2.0commit 15 hours ago80worth
AirLLMlyogavin/airllm
Run 70B models on a 4GB GPUPersonalLocal inferenceLibrary / SDK★ 35kApache-2.0commit 2 days ago73worth
ncnnTencent/ncnn
Neural network inference for mobilePersonalEnterpriseLocal inferenceLibrary / SDK★ 24kcommit 2 days ago66worth
BitNetmicrosoft/BitNet
Inference framework for 1-bit LLMsPersonalEnterpriseLocal inferenceLibrary / SDK★ 40kMITcommit 2 months ago63worth