ollama / ollama

Runs open-weight LLMs such as Gemma, Qwen, DeepSeek, and gpt-oss locally or in the cloud, with a CLI, REST API, and Python and JavaScript libraries.

vllm-project / vllm

A high-throughput LLM inference and serving engine featuring PagedAttention, continuous batching, and an OpenAI-compatible API server.

sgl-project / sglang

SGLang is an open-source inference framework for serving large language, vision-language, and diffusion models.

lightseekorg / tokenspeed

TokenSpeed is an LLM inference engine built for agentic workloads, with a C++ scheduler control plane and Python execution plane.