Language & LLMs

What Is a Causal Language Model?

A causal language model predicts each token based only on the tokens that come before it, using masking to hide future positions. This left-to-right constraint makes it suitable for text generation. GPT-style decoder models are causal language models.