jmaczan / tiny-vllm

git clone https://github.com/jmaczan/tiny-vllm.git

Build your own high performance LLM inference engine in C++ and CUDA - a smaller version of vLLM