Last Updated on 2023-12-14 by Clay Introduction vLLM is a large language model (LLM) acceleration framework that developed by the research team of University of California, Berkeley. It is used PagedAttention to increase the usage rate of GPU VRAM, and this method do not change the model architecture. PagedAttention is inspired by the classic virtual … Continue reading Using vLLM To Accelerate The Decoding Of Large Language Model