ultralytics / yolov5
Ultralytics YOLOv5 in PyTorch for object detection, instance segmentation, classification, training, and export.
ultralytics / yolov5
NVIDIA / TensorRT-LLM
TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT LLM also contains components to create Python and C++ runtimes that orchestrate the inference execution in a performant way.
NVIDIA / TensorRT
NVIDIA® TensorRT™ is an SDK for high-performance deep learning inference on NVIDIA GPUs. This repository contains the open source components of TensorRT.
ultralytics / yolov3
PyTorch implementation of YOLOv3, YOLOv3-SPP, and YOLOv3-tiny for real-time object detection with training, validation, inference, and multi-format export.
ai-dynamo / dynamo
A Datacenter Scale Distributed Inference Serving Framework
mikel-brostrom / boxmot
BoxMOT: Pluggable Python and C++ SOTA multi-object tracking modules with support for axis-aligned and oriented bounding boxes
RapidAI / RapidOCR
📄 Awesome OCR multiple programing languages toolkits based on ONNX Runtime, OpenVINO, MNN, PaddlePaddle, TensorRT and PyTorch.
xlite-dev / Awesome-LLM-Inference
📚A curated list of Awesome LLM/VLM Inference Papers with Codes: Flash-Attention, Paged-Attention, WINT8/4, Parallelism, etc.🎉
NVIDIA / Model-Optimizer
A unified library of SOTA model optimization techniques like quantization, distillation, pruning, neural architecture search, speculative decoding, etc. It compresses deep learning models for downstream deployment frameworks like TensorRT-LLM, TensorRT, vLLM, etc. to optimize inference speed.
TingsongYu / PyTorch-Tutorial-2nd
《Pytorch实用教程》（第二版）无论是零基础入门，还是CV、NLP、LLM项目应用，或是进阶工程化部署落地，在这里都有。相信在本书的帮助下，读者将能够轻松掌握 PyTorch 的使用，成为一名优秀的深度学习工程师。
collabora / WhisperLive
A nearly-live implementation of OpenAI's Whisper.
onnx / onnx-tensorrt
ONNX-TensorRT: TensorRT backend for ONNX
pytorch / TensorRT
PyTorch/TorchScript/FX compiler for NVIDIA GPUs using TensorRT
jolibrain / deepdetect
Deep Learning Server and CLI for Torch and TensorRT
roboflow / inference
Turn any computer or edge device into a command center for your computer vision projects.
SemiAnalysisAI / InferenceX
Open Source AI Accelerator Research Platform Standard / 开源推理研究平台
CoinCheung / BiSeNet
Add bisenetv2. My implementation of BiSeNet
FeiYull / TensorRT-Alpha
🔥🔥🔥TensorRT for YOLO26 (det/pose/seg/depth/obb/cls), YOLOv11,YOLOv10 YOLOv9,YOLOv8, ... 🚀🚀🚀 CUDA IS ALL YOU NEED. 🍎🍎🍎
aiptimizer / TurboOCR
TurboOCR, >200 img/s OmnidocBench. TensorRT FP16, PP-OCRv6, HTTP + gRPC
datawhalechina / base-llm
从 NLP 到 LLM 的算法全栈教程，在线阅读地址：https://datawhalechina.github.io/base-llm/
triton-inference-server / tensorrtllm_backend
The Triton TensorRT-LLM Backend
insight-platform / Savant
Python Computer Vision & Video Analytics Framework With Batteries Included
NickSwardh / YoloDotNet
YoloDotNet - A C# .NET 8.0 project for Classification, Object Detection, OBB Detection, Segmentation and Pose Estimation in both images and live video streams.
gotonote / Autopilot-Notes
自动驾驶笔记，以解析各模块知识点、整合行业优秀解决方案进行阐述，以帮助自己及有需要的读者；包含深度学习、deeplearning、无人驾驶、BEV、Transformer、ADAS、CVPR、特斯拉AI DAY、大模型、chatgpt等内容.
the-database / mpv-AnimeJaNai
Real-time anime upscaling to 4K in mpv with lightweight super-resolution models
smg-project / smg
Engine-agnostic LLM gateway in Rust. Full OpenAI & Anthropic API compatibility across vLLM, TRT-LLM, TokenSpeed, SGLang, OpenAI, Gemini & more. Industry-first gRPC pipeline, KV cache-aware routing, chat history, tokenization caching, Responses API, embeddings, WASM plugins, MCP, and multi-tenant auth.
yzfzzz / depth-detect
mono / stereo depth estimation + yolo object detection with deployment on Jeston nano / TX2 / GeForce by TensorRT
NVIDIA / TensorRT-Edge-LLM
High-performance, light-weight C++ LLM and VLM Inference Software for Physical AI
ome-projects / ome
Open Model Engine (OME) — Kubernetes operator for LLM serving, GPU scheduling, and model lifecycle management. Works with SGLang, vLLM, TensorRT-LLM, and Triton
NVIDIA-ISAAC-ROS / isaac_ros_pose_estimation
Deep learned, NVIDIA-accelerated 3D object pose estimation
vrgamegirl19 / VRGDG-SeedVR2-TensorRT-Studio
Local JavaScript + FastAPI video restoration studio for SeedVR2 with TensorRT acceleration, previews, synchronized comparisons, resumable renders, and post-processing.
yanght27 / GPU-Perf-Playground
GPU 性能与 AI Infra 学习项目：CUDA/Triton 算子、NCU/NSYS、vLLM/SGLang/TRT-LLM/ms-swift、PyTorch/DeepSpeed/ms-swift 训练、并行架构
seandavedalaguit-ui / edge-vision-forge
Netspresso Trainer 2026: Train, Compress and Deploy Vision AI Models on Edge Devices
MuhammadAhsan7866 / edge-vision-forge
The 2026 Edge Vision Toolkit: Train, Compress, and Deploy ViT Models on Any Device
VANdexj / OmniRisk
OmniRisk: omnidirectional trajectory-risk learning for agile quadrotor dynamic avoidance. One forward pass scores every maneuver onboard; real-world evasion at up to 15 m/s.
Jinlong-cs / pi.cpp
TensorRT-first robot foundation model inference for edge GPUs
SoundsGoodAI / fast-gpu-asr
High-throughput batched ASR for NVIDIA GPUs. Run Zipformer and NVIDIA NeMo Parakeet models through one Python API with TensorRT, custom CUDA plugins, GPU beam search, and word timestamps - up to 25,000 RTFx on B300.