flashrt-project / FlashRT

FlashRT is a high-performance realtime inference engine for small-batch, latency-sensitive AI workloads. The flagship integration is production VLA control for Pi0, Pi0.5, GROOT N1.6, and Pi0-FAST. Also support llm e.g, qwen3.6-27B

h5i-dev / h5i

An Axum-based Rust web framework for building and verifying web apps with Lean 4, featuring optional GPU acceleration and integrated development infrastructure for red-teaming and CI/CD.

xlite-dev / LeetCUDA

An Open-source book with Modern CUDA learning Notes for Beginners - BF16/FP8/FP4, HGEMM, CuTe, Flash-Attention, etc.

Luce-Org / lucebox

LLM speculative inference server for heterogeneous hardware & consumer GPUs

pegainfer-project / pegainfer

Pure Rust + CUDA LLM inference engine — no PyTorch, OpenAI-compatible, serves Qwen3 to Kimi-K2

InternLM / lmdeploy

LMDeploy is a toolkit for compressing, deploying, and serving LLMs.

NVIDIA / cudnn-frontend

cuDNN Frontend is NVIDIA's modern, open-source entry point to the cuDNN library and a growing collection of high-performance open-source kernels.

Rust-GPU / rust-cuda

Ecosystem of libraries and tools for writing and executing fast GPU code fully in Rust.

chelsea0x3b / cudarc

Safe rust wrapper around CUDA toolkit

NVIDIA / nvbench

CUDA Kernel Benchmarking Library

dnv2003 / v100-skinny

Hand-written NVFP4 W4A16 CUDA kernels for Volta

yanght27 / GPU-Perf-Playground

GPU 性能与 AI Infra 学习项目：CUDA/Triton 算子、NCU/NSYS、vLLM/SGLang/TRT-LLM/ms-swift、PyTorch/DeepSpeed/ms-swift 训练、并行架构

NVIDIA / cccl

CUDA Core Compute Libraries