This study introduces parameter-efficient fine-tuning (PEFT) methods to large protein language models (PLMs) for two important proteomic tasks: predicting protein-protein interactions (PPIs) and predicting the symmetry of homooligomer quaternary structures. The approach leverages the LoRA method, which adds low-rank adapter matrices to the model, enabling efficient adaptation with significantly reduced memory and parameter usage compared to traditional full fine-tuning (FT). The results show that PEFT models are competitive with FT models in performance while requiring fewer resources, making them more accessible to research groups with limited computational capabilities. For PPI prediction, training only the classification head of the model also achieves strong performance with five orders of magnitude fewer parameters. The study also demonstrates that PEFT models outperform state-of-the-art PPI prediction methods in terms of computational efficiency. A comprehensive evaluation of hyperparameters shows that PEFT is robust to variations in these parameters, and best practices for PEFT in proteomics differ from those in natural language processing. The code for model adaptation and evaluation is available open-source at https://github.com/microsoft/peft_proteomics. The work provides a blueprint for democratizing the power of PLM adaptation to groups with limited computational resources.This study introduces parameter-efficient fine-tuning (PEFT) methods to large protein language models (PLMs) for two important proteomic tasks: predicting protein-protein interactions (PPIs) and predicting the symmetry of homooligomer quaternary structures. The approach leverages the LoRA method, which adds low-rank adapter matrices to the model, enabling efficient adaptation with significantly reduced memory and parameter usage compared to traditional full fine-tuning (FT). The results show that PEFT models are competitive with FT models in performance while requiring fewer resources, making them more accessible to research groups with limited computational capabilities. For PPI prediction, training only the classification head of the model also achieves strong performance with five orders of magnitude fewer parameters. The study also demonstrates that PEFT models outperform state-of-the-art PPI prediction methods in terms of computational efficiency. A comprehensive evaluation of hyperparameters shows that PEFT is robust to variations in these parameters, and best practices for PEFT in proteomics differ from those in natural language processing. The code for model adaptation and evaluation is available open-source at https://github.com/microsoft/peft_proteomics. The work provides a blueprint for democratizing the power of PLM adaptation to groups with limited computational resources.