PRoLoRA is a parameter-efficient method for low-rank adaptation (LoRA) that introduces an intra-layer sharing mechanism to enhance parameter efficiency. It consists of four key components: broadcast reduction, rotation enhancement, partially-sharing refinement, and rectified initialization strategy. PRoLoRA improves upon LoRA by reducing the number of trainable parameters while maintaining or improving performance. It achieves higher parameter efficiency by allowing the same parameter budget to support a higher rank, leading to better performance and lower resource consumption. PRoLoRA is particularly effective in multi-LoRA scenarios, where it significantly reduces storage and GPU memory usage. Empirical experiments on multiple instruction tuning datasets show that PRoLoRA outperforms LoRA in terms of parameter efficiency, with a smaller number of trainable parameters achieving better performance. Ablation studies validate the necessity of each component and demonstrate PRoLoRA's superiority over alternative methods. PRoLoRA is a resource-friendly alternative to LoRA, offering better capacity, practical feasibility, and broader applicability. The method is scalable to larger language models and has the potential to be integrated with other parameter-sharing techniques. Overall, PRoLoRA provides a more efficient solution for parameter-efficient fine-tuning of large language models.PRoLoRA is a parameter-efficient method for low-rank adaptation (LoRA) that introduces an intra-layer sharing mechanism to enhance parameter efficiency. It consists of four key components: broadcast reduction, rotation enhancement, partially-sharing refinement, and rectified initialization strategy. PRoLoRA improves upon LoRA by reducing the number of trainable parameters while maintaining or improving performance. It achieves higher parameter efficiency by allowing the same parameter budget to support a higher rank, leading to better performance and lower resource consumption. PRoLoRA is particularly effective in multi-LoRA scenarios, where it significantly reduces storage and GPU memory usage. Empirical experiments on multiple instruction tuning datasets show that PRoLoRA outperforms LoRA in terms of parameter efficiency, with a smaller number of trainable parameters achieving better performance. Ablation studies validate the necessity of each component and demonstrate PRoLoRA's superiority over alternative methods. PRoLoRA is a resource-friendly alternative to LoRA, offering better capacity, practical feasibility, and broader applicability. The method is scalable to larger language models and has the potential to be integrated with other parameter-sharing techniques. Overall, PRoLoRA provides a more efficient solution for parameter-efficient fine-tuning of large language models.