Language & LLMs

What Is XLNet?

XLNet is a transformer-based language model that combines the benefits of autoregressive and bidirectional training. It uses a permutation language modeling objective, considering many possible orderings of the input tokens. This allows it to capture bidirectional context without the masking approach used by BERT.