grwth404 / Sequence_Scheduling

git clone https://github.com/grwth404/Sequence_Scheduling.git

PyTorch implementation of paper "Response Length Perception and Sequence Scheduling: An LLM-Empowered LLM Inference Pipeline".