vllm-project / speculators

git clone https://github.com/vllm-project/speculators.git

A unified library for building, evaluating, and storing speculative decoding algorithms for LLM inference in vLLM