SGLang

Fast serving framework for LLMs

About

High-throughput inference for language and multimodal models.

From the repository: “SGLang is a high-performance serving framework for large language models and multimodal models.”

vLLMHigh-throughput LLM serving engineDocker / self-hostLibrary / SDK86

OllamaRun open-weight LLMs locally with one commandDocker / self-hostDesktop90

llama.cppLLM inference in C/C++ on CPUs and GPUsCLILibrary / SDK86

LocalAIDrop-in OpenAI API replacement that runs locallyDocker / self-host86

81top 31% of catalog

Popularity23/25

Momentum10/20

Maintenance25/25

Community15/15

Readiness8/15

Momentum is estimated (striped) until 90 days of star history exist.

Put this in your README. It links back to this page, and the score updates every day.

Worth score badge (Markdown)

[![DigGitHub worth](https://diggithub.com/-/badge/sgl-project/sglang.svg)](https://diggithub.com/sgl-project/sglang)

Featured badge, no score (Markdown)

[![Featured on DigGitHub](https://diggithub.com/-/badge/sgl-project/sglang/featured.svg)](https://diggithub.com/sgl-project/sglang)

Need a small one for a row of badges? Add ?style=flat after .svg.

None of these are confirmed for this project yet. Do you know it has one? Tell usEvaluate for your company Beta