bentoml / OpenLLM

git clone https://github.com/bentoml/OpenLLM.git

Run any open-source LLMs, such as DeepSeek and Llama, as OpenAI compatible API endpoint in the cloud.

—

stars, 24h

—

stars, 7 days

—

stars, 30 days

13k

★ total

895

forks · 21 open issues

Star history starts 2026-10-20; the chart appears after the second daily snapshot.

LanguagePython

LicenseApache-2.0

Created2023-04-19 (3y ago)

Last pushtoday

OwnerOrganization

GitHub topics

bentoml fine-tuning llama llama2 llama3-1 llama3-2 llama3-2-vision llm llm-inference llm-ops llm-serving llmops mistral mlops model-inference open-source-llm openllm vicuna

Links

GitHub: github.com/bentoml/OpenLLM
Homepage: https://bentoml.com
Releases · Issues · Hacker News mentions