Last Updated on 2024-11-08 by Clay When we use large language models for generative tasks, particularly in auto-regressive tasks, the model essentially performs a massive classification task. The classification targets are the tokens in our vocabulary, which are the smallest building blocks that make up words. If we want to use greedy decoding, we can … Continue reading A Note Of Large Language Model Decode Sampling