Language & LLMs

What Is Top-k Sampling?

Top-k sampling is a decoding method that restricts token selection to the k most likely candidates and then samples randomly among them. This introduces variety while avoiding very unlikely tokens. The value of k controls how much diversity the generated text has.