jundot / omlx

LLM inference server with continuous batching & SSD caching for Apple Silicon — managed from the macOS menu bar

triton-inference-server / server

The Triton Inference Server provides an optimized cloud and edge inferencing solution.

raullenchai / Rapid-MLX

Rapid-MLX is an open-source (Apache 2.0) OpenAI- and Anthropic-compatible LLM inference server for Apple Silicon, built on MLX, focused on reliable tool calling for coding agents. Up to 4× faster than Apple's MLX (mlx-lm) on the same weights.

superlinked / sie

Open-source inference server and production cluster for all the models your agent needs.

containers / ramalama

RamaLama is an open-source developer tool that simplifies the local serving of AI models from any source and facilitates their use for inference in production, all through the familiar language of containers.

roboflow / inference

Turn any computer or edge device into a command center for your computer vision projects.

CoinCheung / BiSeNet

Add bisenetv2. My implementation of BiSeNet

waybarrios / vllm-mlx

High-performance OpenAI and Anthropic compatible LLM inference server for Apple Silicon. Native MLX, continuous batching, multimodal models, MCP tool calling, and Claude Code support.

basetenlabs / truss

The simplest way to serve AI/ML models in production

triton-inference-server / tutorials

This repository contains tutorials and examples for Triton Inference Server

triton-inference-server / model_analyzer

Triton Model Analyzer is a CLI tool to help with better understanding of the compute and memory requirements of the Triton Inference Server models.

ZinYY / AdaFlash

[COLMW'26 Spotlight] AdaFlash: Adaptive Speculative Decoding via On-Policy Distilled Diffusion Drafters