Best open-source local LLM libraries & SDKs for enterprise

Open-source runtimes for running LLMs on your own hardware.

Filters · 3 Enterprise, Local inference, Library / SDK

For

Personal7Enterprise8

Purpose

AI & LLM tooling124Agents & frameworks32Local inference8RAG & vector DBs28Coding assistants2Evals & observability7Agent skills3MCP servers4Developer tools175Web frameworks96Databases & backends15Editors & IDEs3API tools13Self-hosted apps4Notes & docs4Automation16Data & analytics22Infra & DevOps5Security & identity5Communication8Media & photos17Design & creative39

Runs on

Web1Docker / self-host5Desktop1Mobile1CLI2Library / SDK8Extension / plugin0

License

Permissive6Copyleft0Source-available0Other / custom2

More

Has live demo0Commit in last 90 days8

8 projects

SortWorth scoreTrendingMost starsRecently updatedNewest in catalog

v

vLLMvllm-project/vllm

High-throughput LLM serving engineEnterpriseLocal inferenceDocker / self-hostLibrary / SDK★ 93kApache-2.0commit 19 hours ago

86worth

☆ Save

U

Unslothunslothai/unsloth

Fast local fine-tuning and running of LLMsPersonalEnterpriseLocal inferenceWebLibrary / SDK★ 77kApache-2.0commit 20 hours ago

81worth

☆ Save

S

SGLangsgl-project/sglang

Fast serving framework for LLMsEnterpriseLocal inferenceDocker / self-hostLibrary / SDK★ 37kApache-2.0commit 19 hours ago

81worth

☆ Save

O

OpenVINOopenvinotoolkit/openvino

Optimize and deploy AI inferenceEnterpriseLocal inferenceLibrary / SDK★ 11kApache-2.0commit yesterday

80worth

☆ Save

w

whisper.cppggml-org/whisper.cpp

Whisper speech recognition in C/C++PersonalEnterpriseLocal inferenceMedia & photosCLILibrary / SDK★ 54kMITcommit yesterday

79worth

☆ Save

R

RunAnywhere SDKsRunanywhereAI/runanywhere-sdks

Run AI locally on devicesPersonalEnterpriseLocal inferenceMobileLibrary / SDK★ 10kcommit 2 weeks ago

70worth

☆ Save

n

ncnnTencent/ncnn

Neural network inference for mobilePersonalEnterpriseLocal inferenceLibrary / SDK★ 24kcommit 2 days ago

66worth

☆ Save

B

BitNetmicrosoft/BitNet

Inference framework for 1-bit LLMsPersonalEnterpriseLocal inferenceLibrary / SDK★ 40kMITcommit 2 months ago

63worth

☆ Save