lightseekorg / tokenspeed
TokenSpeed is a speed-of-light LLM inference engine.
lightseekorg / tokenspeed
SoundsGoodAI / fast-gpu-asr
High-throughput batched ASR for NVIDIA GPUs. Run Zipformer and NVIDIA NeMo Parakeet models through one Python API with TensorRT, custom CUDA plugins, GPU beam search, and word timestamps - up to 25,000 RTFx on B300.
lyuyiqi / open-jev-fast
Faster inference backend for Open-Jev-27B: fused CUDA kernels, prefix tree, CUDA Graphs (B300, bf16)