vllm-project / vllm
A high-throughput LLM inference and serving engine featuring PagedAttention, continuous batching, and an OpenAI-compatible API server.
vllm-project / vllm
sgl-project / sglang
SGLang is an open-source inference framework for serving large language, vision-language, and diffusion models.