Neroued / ninfer

High-performance single-GPU inference for selected model checkpoints and GPUs.

1314521gjy / ninfer-fusion-kvmem

—

CSUNX233 / qwen3.8-27b-ninfer-5070tilaptop-kvmem-local-agent

RTX 5070 Ti Laptop 12GB 跑 Qwen3.8-27B：NInfer + KVMem、256K 逻辑上下文、RAM KV、MTP、快速缓存续接与 dsh 本地 Agent / 搜索 / 浏览器。含源码、Windows 部署及实测。

melis2023 / ninfer-kvmem-actkv

—

Wallawalla47 / Infernix

C++/CUDA single-GPU inference engine for Qwen3.8 (incl. Qwen3.8-Flash-Next with offloaded experts) on the RTX 5090; grew from NInfer

Arize-ai / openinference

OpenTelemetry Instrumentation for AI Observability

5258MF / ninfer-rtx3060-27b

RTX 3060 12GB/8GB 本地跑 27B 大模型：256K 上下文、约 45 token/秒，能看图、调工具，附懒人包和源码

headpiece747 / ninfer-5090-windows

Native Windows port of NInfer engine for RTX 5090. Features Qwen3.8-27B with QUASAR and NInfer models, MTP/DFlash2 with vision and 262,144 context

dollarwong / ninfer-v100

NInfer on Tesla V100 (sm_70): Qwen3.8-27B deployment, build/convert scripts, systemd service, and cross-engine benchmark (vs llama.cpp). Fork of Neroued/ninfer with sm70 vision-encoding speedup.

paicat1 / Bonsai-27B-NInfer

Bonsai-27B x NInfer local deployment project (RTX 5080, ternary quantization)

JGamboa / ninfer-4090-windows

—

mr-september / ninfer-2080ti-22g

—

CraneBW / ninfer-ternary-bonsai-ada

Ternary (2.125 bit/weight) Bonsai 2 27B on the NINFER engine — Ada sm_89, Linux. Derivative work of Neroued/ninfer; see README.

alphastorm / omp-ninfer

Durable local inference for Oh My Pi: NInfer + Qwen3.8 27B on one RTX 3090/4090/5090, with restart-resumable OpenAI Responses state.

giveen / ninfer-studio

Desktop UI for the NInfer local LLM inference engine

alanthinker / ninfer-4090-yarn

NInfer-4090: NInfer inference engine ported to RTX 4090 (sm_89) with YaRN support for Qwen3.8-27B

Ithrial / ninfer-cmp170hx

NInfer SM80 port for NVIDIA CMP 170HX