Ollamaollama/ollama
Run open-weight LLMs locally with one commandPersonalEnterpriseLocal inferenceDocker / self-hostDesktopCLI★ 182kMITcommit yesterday90worth
Best open-source local LLM runners for enterprise
Open-source runtimes for running LLMs on your own hardware.
11 projects
vLLMvllm-project/vllm
High-throughput LLM serving engineEnterpriseLocal inferenceDocker / self-hostLibrary / SDK★ 93kApache-2.0commit 22 hours ago86worth
LocalAImudler/LocalAI
Drop-in OpenAI API replacement that runs locallyPersonalEnterpriseLocal inferenceDocker / self-host★ 49kMITcommit yesterday86worth
PrivateGPTzylon-ai/private-gpt
Private AI API over local modelsEnterpriseLocal inferenceRAG & vector DBsDocker / self-host★ 58kApache-2.0commit 2 weeks ago83worth
Unslothunslothai/unsloth
Fast local fine-tuning and running of LLMsPersonalEnterpriseLocal inferenceWebLibrary / SDK★ 77kApache-2.0commit 23 hours ago81worth
SGLangsgl-project/sglang
Fast serving framework for LLMsEnterpriseLocal inferenceDocker / self-hostLibrary / SDK★ 37kApache-2.0commit 22 hours ago81worth
OpenVINOopenvinotoolkit/openvino
Optimize and deploy AI inferenceEnterpriseLocal inferenceLibrary / SDK★ 11kApache-2.0commit yesterday80worth
whisper.cppggml-org/whisper.cpp
Whisper speech recognition in C/C++PersonalEnterpriseLocal inferenceMedia & photosCLILibrary / SDK★ 54kMITcommit yesterday79worth
RunAnywhere SDKsRunanywhereAI/runanywhere-sdks
Run AI locally on devicesPersonalEnterpriseLocal inferenceMobileLibrary / SDK★ 10kcommit 2 weeks ago70worth
ncnnTencent/ncnn
Neural network inference for mobilePersonalEnterpriseLocal inferenceLibrary / SDK★ 24kcommit 2 days ago66worth
BitNetmicrosoft/BitNet
Inference framework for 1-bit LLMsPersonalEnterpriseLocal inferenceLibrary / SDK★ 40kMITcommit 2 months ago63worth