This paper introduces P-Tuning, a novel method for improving the performance and stability of pretrained language models (PLMs) in natural language understanding (NLU) tasks. P-Tuning combines trainable continuous prompt embeddings with discrete prompts, which helps stabilize training and improve performance across various NLU benchmarks, including LAMA and SuperGLUE. Unlike manual discrete prompts, which are unstable and sensitive to small changes, P-Tuning uses continuous prompts that are optimized through backpropagation to better align with the task objective. This approach not only enhances performance but also reduces the performance gap between different discrete prompts, leading to more stable language model adaptation. P-Tuning is effective for both frozen and fine-tuned language models under both fully-supervised and few-shot learning settings. On the LAMA dataset, P-Tuning outperforms manual discrete prompts and searched prompts by significant margins. On SuperGLUE, P-Tuning surpasses the best discrete prompts in both fully-supervised and few-shot settings. Additionally, P-Tuning reduces the variance in performance across different prompts, making it more reliable for NLU tasks. The method involves using a prompt encoder, such as LSTM or MLP, to model the dependencies between continuous prompt embeddings. Experiments show that P-Tuning consistently outperforms other methods, including PET and Prompt Tuning, on various NLU tasks. The results demonstrate that P-Tuning not only improves performance but also enhances the stability of language model adaptation by reducing the differences between different prompts. This makes P-Tuning a promising approach for improving the effectiveness and reliability of pretrained language models in NLU tasks.This paper introduces P-Tuning, a novel method for improving the performance and stability of pretrained language models (PLMs) in natural language understanding (NLU) tasks. P-Tuning combines trainable continuous prompt embeddings with discrete prompts, which helps stabilize training and improve performance across various NLU benchmarks, including LAMA and SuperGLUE. Unlike manual discrete prompts, which are unstable and sensitive to small changes, P-Tuning uses continuous prompts that are optimized through backpropagation to better align with the task objective. This approach not only enhances performance but also reduces the performance gap between different discrete prompts, leading to more stable language model adaptation. P-Tuning is effective for both frozen and fine-tuned language models under both fully-supervised and few-shot learning settings. On the LAMA dataset, P-Tuning outperforms manual discrete prompts and searched prompts by significant margins. On SuperGLUE, P-Tuning surpasses the best discrete prompts in both fully-supervised and few-shot settings. Additionally, P-Tuning reduces the variance in performance across different prompts, making it more reliable for NLU tasks. The method involves using a prompt encoder, such as LSTM or MLP, to model the dependencies between continuous prompt embeddings. Experiments show that P-Tuning consistently outperforms other methods, including PET and Prompt Tuning, on various NLU tasks. The results demonstrate that P-Tuning not only improves performance but also enhances the stability of language model adaptation by reducing the differences between different prompts. This makes P-Tuning a promising approach for improving the effectiveness and reliability of pretrained language models in NLU tasks.