Best open-source local LLM libraries & SDKs for personal use

Open-source runtimes for running LLMs on your own hardware.

Filters · 3 Personal, Local inference, Library / SDK

For

Personal7Enterprise8

Purpose

AI & LLM tooling119Agents & frameworks30Local inference7RAG & vector DBs25Coding assistants4Evals & observability5Agent skills4MCP servers6Developer tools160Web frameworks89Databases & backends14Editors & IDEs3API tools9Self-hosted apps6Notes & docs6Automation17Data & analytics23Infra & DevOps2Security & identity4Communication8Media & photos24Design & creative50

Runs on

Web3Docker / self-host2Desktop6Mobile1CLI8Library / SDK7Extension / plugin0

License

Permissive5Copyleft0Source-available0Other / custom2

More

Has live demo0Commit in last 90 days7

7 projects

SortWorth scoreTrendingMost starsRecently updatedNewest in catalog

l

llama.cppggml-org/llama.cpp

LLM inference in C/C++ on CPUs and GPUsPersonalLocal inferenceCLILibrary / SDK★ 130kMITcommit 11 hours ago

86worth

☆ Save

U

Unslothunslothai/unsloth

Fast local fine-tuning and running of LLMsPersonalEnterpriseLocal inferenceWebLibrary / SDK★ 77kApache-2.0commit 11 hours ago

81worth

☆ Save

w

whisper.cppggml-org/whisper.cpp

Whisper speech recognition in C/C++PersonalEnterpriseLocal inferenceMedia & photosCLILibrary / SDK★ 54kMITcommit 22 hours ago

79worth

☆ Save

A

AirLLMlyogavin/airllm

Run 70B models on a 4GB GPUPersonalLocal inferenceLibrary / SDK★ 35kApache-2.0commit 2 days ago

73worth

☆ Save

R

RunAnywhere SDKsRunanywhereAI/runanywhere-sdks

Run AI locally on devicesPersonalEnterpriseLocal inferenceMobileLibrary / SDK★ 10kcommit last week

70worth

☆ Save

n

ncnnTencent/ncnn

Neural network inference for mobilePersonalEnterpriseLocal inferenceLibrary / SDK★ 24kcommit 2 days ago

66worth

☆ Save

B

BitNetmicrosoft/BitNet

Inference framework for 1-bit LLMsPersonalEnterpriseLocal inferenceLibrary / SDK★ 40kMITcommit 2 months ago

63worth

☆ Save