Prompt tuning is a parameter-efficient method for adapting frozen language models to downstream tasks by learning "soft prompts" that condition the model. Unlike discrete prompts used by GPT-3, soft prompts are learned through backpropagation and can incorporate signals from labeled examples. This approach outperforms GPT-3's few-shot learning and becomes more competitive as model size increases, eventually matching the performance of full model tuning. Prompt tuning simplifies the recently proposed prefix tuning and enables efficient prompt ensembling, improving robustness to domain shifts and reducing parameter usage. Prompt tuning involves adding a small number of tunable tokens to the input, which are trained end-to-end. This method allows a single frozen model to be reused for multiple tasks, reducing storage and computational costs. Experiments show that prompt tuning outperforms model tuning on domain shift tasks and achieves strong performance on the SuperGLUE benchmark. Prompt tuning is more parameter-efficient than other methods like prefix tuning and WARP, requiring less than 0.01% task-specific parameters for large models. It also performs better than model tuning on out-of-domain datasets, suggesting that freezing general-purpose language understanding parameters and using a lightweight parameter footprint can improve generalization. Prompt ensembling, where multiple prompts are trained for the same task, further enhances performance and efficiency. This approach allows for efficient inference by using a single frozen model with multiple prompts, reducing storage and computational costs. Prompt tuning is also more interpretable than other methods, with learned prompts forming semantic clusters that reflect the task's requirements. This suggests that prompts can effectively prime the model to interpret inputs in specific domains or contexts. Overall, prompt tuning offers a scalable and efficient way to adapt frozen language models to a wide range of tasks, with strong performance on benchmark datasets and improved robustness to domain shifts.Prompt tuning is a parameter-efficient method for adapting frozen language models to downstream tasks by learning "soft prompts" that condition the model. Unlike discrete prompts used by GPT-3, soft prompts are learned through backpropagation and can incorporate signals from labeled examples. This approach outperforms GPT-3's few-shot learning and becomes more competitive as model size increases, eventually matching the performance of full model tuning. Prompt tuning simplifies the recently proposed prefix tuning and enables efficient prompt ensembling, improving robustness to domain shifts and reducing parameter usage. Prompt tuning involves adding a small number of tunable tokens to the input, which are trained end-to-end. This method allows a single frozen model to be reused for multiple tasks, reducing storage and computational costs. Experiments show that prompt tuning outperforms model tuning on domain shift tasks and achieves strong performance on the SuperGLUE benchmark. Prompt tuning is more parameter-efficient than other methods like prefix tuning and WARP, requiring less than 0.01% task-specific parameters for large models. It also performs better than model tuning on out-of-domain datasets, suggesting that freezing general-purpose language understanding parameters and using a lightweight parameter footprint can improve generalization. Prompt ensembling, where multiple prompts are trained for the same task, further enhances performance and efficiency. This approach allows for efficient inference by using a single frozen model with multiple prompts, reducing storage and computational costs. Prompt tuning is also more interpretable than other methods, with learned prompts forming semantic clusters that reflect the task's requirements. This suggests that prompts can effectively prime the model to interpret inputs in specific domains or contexts. Overall, prompt tuning offers a scalable and efficient way to adapt frozen language models to a wide range of tasks, with strong performance on benchmark datasets and improved robustness to domain shifts.