Language & LLMs

What Is Prefix Tuning?

Prefix tuning is a parameter-efficient method that prepends a set of learned continuous vectors to the input of each transformer layer. Only these prefix vectors are trained, while the model's original parameters stay frozen. This steers the model toward a task with very few trainable parameters.