Language & LLMs

What Is a Transformer Encoder?

A transformer encoder is the part of a transformer that reads an entire input sequence and produces contextual representations for each token. It uses stacked layers of self-attention and feedforward networks, allowing every token to consider all others. Encoder-only models like BERT rely on this component.