Language & LLMs

What Is Top-p Sampling?

Top-p sampling, also called nucleus sampling, chooses the next token from the smallest group of top tokens whose cumulative probability exceeds a threshold p. The size of this group varies dynamically based on the probability distribution. This balances diversity and coherence in generated text.