All repos — GH Trending Explorer

CUDA on non-NVIDIA GPUs

DeepSeek-V4.1-Flash-1017 (552B MoE) on 2x RTX 3090 + 128 GB RAM: Rust + CUDA engine and 2-bit quant. 30-37 tok/s decode.

tracel-ai / burn

Burn is a next generation tensor library and Deep Learning Framework that doesn't compromise on flexibility, efficiency and portability.

Rust-GPU / rust-cuda

Ecosystem of libraries and tools for writing and executing fast GPU code fully in Rust.

NVIDIA / cuda-rust

NVIDIA's CUDA platform for Rust. Host runtime crates plus Tile (cutile-rs) and SIMT (cuda-oxide) kernel programming models in idiomatic Rust.

tracel-ai / cubecl

Multi-platform high-performance compute language extension for Rust.

apache / mahout

Apache Mahout - an environment for quickly creating scalable, performant machine learning applications.

chelsea0x3b / cudarc

Safe rust wrapper around CUDA toolkit

h5i-dev / h5i

Unified Rust framework for building, verifying, and testing web apps. Kernel-based architecture for Lean 4 verification and GPU acceleration. Built-in red-teaming tools for automated vulnerability detection.

pegainfer-project / pegainfer

Pure Rust + CUDA LLM inference engine — no PyTorch, OpenAI-compatible, serves Qwen3 to Kimi-K2

Avarok-Cybersecurity / atlas

Pure Rust Inference Engine

surya-koritala / Glyd

Lossless AI compression: 33% less GPU memory, bit for bit.

truespar / paddock

Native Rust inference server for open models on NVIDIA and Apple Silicon. OpenAI- and Anthropic-compatible APIs, GGUF + safetensors + MLX, FP8/NVFP4/MXFP4/Q8/Q4, built-in Studio

pegainfer-project / kern

A model-agnostic GPU runtime for shipping models as verified programs.

Sparticle62ops / pssa

PSSA: a non-transformer language model written from scratch in Rust. Recurrent state-space layer, episodic memory, self-modifying weights, CUDA training, and a full terminal dashboard.

lateos-ai / reflex

A high-performance, GGUF-native Rust & CUDA inference engine optimized for cold-start latency and real-time 'System 1' agent decision loops.

antonellof / frink

Pure-Rust GGUF inference engine with quantized CPU, Metal & CUDA kernels, MoE support, OpenAI-compatible server. Benchmarked head-to-head against llama.cpp.

formetaohy / Neura

Deep learning for games. Cross-platform. No CUDA.

pegainfer-project / pega-omni

OpenAI-compatible speech serving in Rust: streaming TTS and full-duplex voice over GPT-Live. 128 concurrent PersonaPlex sessions on one GPU.

midagedev / bloomery

Hybrid GPU + CPU inference for mixture-of-experts models, written in Rust down to the CUDA kernels. A drop-in llama-server.

gpillon / ignis

OpenAI-compatible inference server for Qwen3.8 on one RTX 5090 / RTX PRO 6000: the 27B at ~170 tok/s (peaks ~370), Flash-Next (125B MoE) at ~100 tok/s, built for local coding agents. Rust + CUDA, Windows and Linux.

infervisor / plow

Packet Language for On-device Warps — a GPU inference compiler and runtime

shuqi2077 / RUDA

https://shuqi2077.github.io/RUDA/ Rust Full GPU computing stack