The paper introduces PROLoRA, a parameter-efficient method for fine-tuning large language models (LLMs) using low-rank adaptation (LoRA). PROLoRA addresses the challenges of high costs and limited model capacity in concurrent LoRA deployments by introducing an intra-layer sharing mechanism. This mechanism consists of four key components: broadcast reduction, rotation enhancement, partially-sharing refinement, and rectified initialization strategy. These components work together to reduce the number of tunable parameters while maintaining or improving model performance. Empirical experiments demonstrate that PROLoRA achieves significantly higher parameter efficiency compared to LoRA, both in terms of specific parameter budgets and performance targets. PROLoRA also scales well to larger LLMs, outperforming LoRA with the same number of tunable parameters. An ablation study further validates the necessity of each component and highlights PROLoRA's superiority over other intra-layer sharing variants. Overall, PROLoRA is presented as a resource-friendly alternative to LoRA, offering better parameter efficiency and broader applicability.The paper introduces PROLoRA, a parameter-efficient method for fine-tuning large language models (LLMs) using low-rank adaptation (LoRA). PROLoRA addresses the challenges of high costs and limited model capacity in concurrent LoRA deployments by introducing an intra-layer sharing mechanism. This mechanism consists of four key components: broadcast reduction, rotation enhancement, partially-sharing refinement, and rectified initialization strategy. These components work together to reduce the number of tunable parameters while maintaining or improving model performance. Empirical experiments demonstrate that PROLoRA achieves significantly higher parameter efficiency compared to LoRA, both in terms of specific parameter budgets and performance targets. PROLoRA also scales well to larger LLMs, outperforming LoRA with the same number of tunable parameters. An ablation study further validates the necessity of each component and highlights PROLoRA's superiority over other intra-layer sharing variants. Overall, PROLoRA is presented as a resource-friendly alternative to LoRA, offering better parameter efficiency and broader applicability.