sigoden / dufs
A file server that supports static serving, uploading, searching, accessing control, webdav...
sigoden / dufs
ai-dynamo / dynamo
A Datacenter Scale Distributed Inference Serving Framework
NVIDIA-NeMo / Switchyard
Switchyard lets LLM applications route traffic across models and providers while preserving native OpenAI and Anthropic API compatibility - enabling flexible model selection, benchmarking, and cost/performance optimization.
michaelfeil / infinity
Infinity is a high-throughput, low-latency serving engine for text-embeddings, reranking models, clip, clap and colpali
openlake-project / openlake
OpenLake is a high performance storage engine for efficient LLM inference and GPU Training
static-web-server / static-web-server
A cross-platform, high-performance and asynchronous web server for static files-serving. ⚡
secluso / core
A privacy-preserving Raspberry Pi home security camera that uses advanced end-to-end encryption.
intentee / paddler
Open-source LLM/VLM load balancer and serving platform for self-hosting LLMs (and VLMs) at scale 🏓🦙 Alternative to projects like llm-d, Docker Model Runner, etc but with less moving parts and simple deployments built around ggml ecosystem. Runs on CPU and GPU.
junkurihara / rust-rpxy
A simple and ultrafast http reverse proxy serving multiple domain names and terminating TLS for http/1.1, 2 and 3, written in Rust
EricLBuehler / candle-vllm
Efficent platform for inference and serving local LLMs including an OpenAI compatible API server.
pegainfer-project / pegainfer
Pure Rust + CUDA LLM inference engine — no PyTorch, OpenAI-compatible, serves Qwen3 to Kimi-K2
truespar / paddock
Native Rust inference server for open models on NVIDIA and Apple Silicon. OpenAI- and Anthropic-compatible APIs, GGUF + safetensors + MLX, FP8/NVFP4/MXFP4/Q8/Q4, built-in Studio
jyjeanne / okf-rs
A fast, open-source Rust toolkit for generating, validating and serving Open Knowledge Format (OKF) knowledge bases from source code.
pegainfer-project / pega-omni
OpenAI-compatible speech serving in Rust: streaming TTS and full-duplex voice over GPT-Live. 128 concurrent PersonaPlex sessions on one GPU.
ArrowSK / ha-astronomy-observer
A practical night-planning companion for amateur astronomers, with observing windows, forecasts and targets.