Ollamaollama/ollama
Run open-weight LLMs locally with one commandPersonalEnterpriseLocal inferenceDocker / self-hostDesktopCLI★ 182kMITcommit 2 days ago90worth
Best open-source local LLM runners
Open-source runtimes for running LLMs on your own hardware.
24 projects · showing 1–20
llama.cppggml-org/llama.cpp
LLM inference in C/C++ on CPUs and GPUsPersonalLocal inferenceCLILibrary / SDK★ 130kMITcommit 22 hours ago86worth
vLLMvllm-project/vllm
High-throughput LLM serving engineEnterpriseLocal inferenceDocker / self-hostLibrary / SDK★ 93kApache-2.0commit 19 hours ago86worth
LocalAImudler/LocalAI
Drop-in OpenAI API replacement that runs locallyPersonalEnterpriseLocal inferenceDocker / self-host★ 49kMITcommit 23 hours ago86worth
PrivateGPTzylon-ai/private-gpt
Private AI API over local modelsEnterpriseLocal inferenceRAG & vector DBsDocker / self-host★ 58kApache-2.0commit 2 weeks ago83worth
Unslothunslothai/unsloth
Fast local fine-tuning and running of LLMsPersonalEnterpriseLocal inferenceWebLibrary / SDK★ 77kApache-2.0commit 21 hours ago81worth
SGLangsgl-project/sglang
Fast serving framework for LLMsEnterpriseLocal inferenceDocker / self-hostLibrary / SDK★ 37kApache-2.0commit 19 hours ago81worth
oMLXjundot/omlx
LLM server for Apple SiliconPersonalLocal inferenceDesktop★ 22kApache-2.0commit 2 days ago80worth
OpenVINOopenvinotoolkit/openvino
Optimize and deploy AI inferenceEnterpriseLocal inferenceLibrary / SDK★ 11kApache-2.0commit 2 days ago80worth
whisper.cppggml-org/whisper.cpp
Whisper speech recognition in C/C++PersonalEnterpriseLocal inferenceMedia & photosCLILibrary / SDK★ 54kMITcommit 2 days ago79worth
ColibriJustVugg/colibri
Run large MoE models on everyday hardwarePersonalLocal inferenceCLI★ 39kApache-2.0commit last week79worth
llmfitAlexsJones/llmfit
Find which LLMs run on your hardwarePersonalLocal inferenceCLI★ 38kMITcommit 2 days ago79worth
Janjanhq/jan
Offline ChatGPT alternative that runs on your computerPersonalChat UIsLocal inferenceDesktop★ 45kcommit 3 days ago73worth
AirLLMlyogavin/airllm
Run 70B models on a 4GB GPUPersonalLocal inferenceLibrary / SDK★ 35kApache-2.0commit 2 days ago73worth
Stability MatrixLykosAI/StabilityMatrix
Package manager for Stable Diffusion UIsPersonalLocal inferenceMedia & photosDesktop★ 8.9kAGPL-3.0commit 4 days ago72worth
llamafilemozilla-ai/llamafile
Run an LLM from a single filePersonalLocal inferenceCLI★ 26kcommit 4 days ago71worth
RunAnywhere SDKsRunanywhereAI/runanywhere-sdks
Run AI locally on devicesPersonalEnterpriseLocal inferenceMobileLibrary / SDK★ 10kcommit 2 weeks ago70worth
Text Generation WebUIoobabooga/textgen
Desktop app for running local LLMsPersonalChat UIsLocal inferenceWebDesktop★ 48kAGPL-3.0commit 2 months ago69worth
Mochi DiffusionMochiDiffusion/MochiDiffusion
Stable Diffusion on MacPersonalLocal inferenceMedia & photosDesktop★ 8kGPL-3.0commit 5 days ago68worth
exoexo-explore/exo
Run frontier AI models across your own devicesPersonalLocal inferenceCLI★ 48kApache-2.0commit last month67worth