Language & LLMs

What Is Positional Encoding?

Positional encoding is a method for injecting information about the position of each token into a transformer, which otherwise processes all tokens in parallel without inherent order. It typically adds or combines position-dependent values with token embeddings. This allows the model to distinguish word order and sequence structure.