Parameter-Efficient Fine-Tuning (PEFT) is a critical technique for adapting large models to specific tasks while minimizing computational and memory costs. This survey provides a comprehensive overview of PEFT algorithms, their performance, computational overhead, and applications across various domains, including natural language processing (NLP), computer vision (CV), and multimodal tasks. PEFT enables efficient fine-tuning by selectively adjusting a small subset of parameters, reducing the need for extensive retraining. The survey categorizes PEFT methods into four main types: additive, selective, reparameterized, and hybrid. Additive PEFT introduces new parameters or modifies existing ones, while selective PEFT tunes only a subset of parameters. Reparameterized PEFT transforms model parameters into low-dimensional forms for training, and hybrid PEFT combines multiple techniques to optimize performance. The survey also discusses system-level challenges, such as efficient implementation, memory management, and deployment strategies for PEFT. Key algorithms include LoRA, adapters, soft prompts, and prefix-tuning, each with unique mechanisms for parameter efficiency. The survey highlights recent advancements in PEFT, including techniques like LoRA, which uses low-rank matrices to adapt models, and hybrid methods that combine different approaches for improved performance. The study also explores the application of PEFT in various tasks, such as text generation, translation, and image recognition, demonstrating its versatility and effectiveness. Overall, PEFT is essential for making large models more practical and efficient in real-world applications.Parameter-Efficient Fine-Tuning (PEFT) is a critical technique for adapting large models to specific tasks while minimizing computational and memory costs. This survey provides a comprehensive overview of PEFT algorithms, their performance, computational overhead, and applications across various domains, including natural language processing (NLP), computer vision (CV), and multimodal tasks. PEFT enables efficient fine-tuning by selectively adjusting a small subset of parameters, reducing the need for extensive retraining. The survey categorizes PEFT methods into four main types: additive, selective, reparameterized, and hybrid. Additive PEFT introduces new parameters or modifies existing ones, while selective PEFT tunes only a subset of parameters. Reparameterized PEFT transforms model parameters into low-dimensional forms for training, and hybrid PEFT combines multiple techniques to optimize performance. The survey also discusses system-level challenges, such as efficient implementation, memory management, and deployment strategies for PEFT. Key algorithms include LoRA, adapters, soft prompts, and prefix-tuning, each with unique mechanisms for parameter efficiency. The survey highlights recent advancements in PEFT, including techniques like LoRA, which uses low-rank matrices to adapt models, and hybrid methods that combine different approaches for improved performance. The study also explores the application of PEFT in various tasks, such as text generation, translation, and image recognition, demonstrating its versatility and effectiveness. Overall, PEFT is essential for making large models more practical and efficient in real-world applications.