Language & LLMs

What Is Next-Token Prediction?

Next-token prediction is the training objective where a model learns to predict the next token given all previous tokens in a sequence. Repeatedly applying this prediction lets the model generate coherent text. It is the foundation of autoregressive language models like the GPT family.