Core Concepts

What Is the Attention Mechanism?

Attention is a technique that lets a model decide which parts of its input are most relevant to each part of the output it produces. Rather than processing a sequence strictly in order, an attention-based model can relate distant elements directly. It is the core idea that makes transformers effective at language and other sequence tasks.