ultralytics / yolov5
Ultralytics YOLOv5 in PyTorch for object detection, instance segmentation, classification, training, and export.
ultralytics / yolov5
NVIDIA / TensorRT-LLM
TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT LLM also contains components to create Python and C++ runtimes that orchestrate the inference execution in a performant way.
ultralytics / yolov3
PyTorch implementation of YOLOv3, YOLOv3-SPP, and YOLOv3-tiny for real-time object detection with training, validation, inference, and multi-format export.
mikel-brostrom / boxmot
BoxMOT: Pluggable Python and C++ SOTA multi-object tracking modules with support for axis-aligned and oriented bounding boxes
RapidAI / RapidOCR
📄 Awesome OCR multiple programing languages toolkits based on ONNX Runtime, OpenVINO, MNN, PaddlePaddle, TensorRT and PyTorch.
xlite-dev / Awesome-LLM-Inference
📚A curated list of Awesome LLM/VLM Inference Papers with Codes: Flash-Attention, Paged-Attention, WINT8/4, Parallelism, etc.🎉
NVIDIA / Model-Optimizer
A unified library of SOTA model optimization techniques like quantization, distillation, pruning, neural architecture search, speculative decoding, etc. It compresses deep learning models for downstream deployment frameworks like TensorRT-LLM, TensorRT, vLLM, etc. to optimize inference speed.
collabora / WhisperLive
A nearly-live implementation of OpenAI's Whisper.
pytorch / TensorRT
PyTorch/TorchScript/FX compiler for NVIDIA GPUs using TensorRT
roboflow / inference
Turn any computer or edge device into a command center for your computer vision projects.
SemiAnalysisAI / InferenceX
Open Source AI Accelerator Research Platform Standard / 开源推理研究平台
CoinCheung / BiSeNet
Add bisenetv2. My implementation of BiSeNet
insight-platform / Savant
Python Computer Vision & Video Analytics Framework With Batteries Included
NVIDIA / TensorRT-Edge-LLM
High-performance, light-weight C++ LLM and VLM Inference Software for Physical AI
vrgamegirl19 / VRGDG-SeedVR2-TensorRT-Studio
Local JavaScript + FastAPI video restoration studio for SeedVR2 with TensorRT acceleration, previews, synchronized comparisons, resumable renders, and post-processing.
yanght27 / GPU-Perf-Playground
GPU 性能与 AI Infra 学习项目：CUDA/Triton 算子、NCU/NSYS、vLLM/SGLang/TRT-LLM/ms-swift、PyTorch/DeepSpeed/ms-swift 训练、并行架构
Jinlong-cs / pi.cpp
TensorRT-first robot foundation model inference for edge GPUs
SoundsGoodAI / fast-gpu-asr
High-throughput batched ASR for NVIDIA GPUs. Run Zipformer and NVIDIA NeMo Parakeet models through one Python API with TensorRT, custom CUDA plugins, GPU beam search, and word timestamps - up to 25,000 RTFx on B300.