AutoLoRA is a meta learning-based framework that automatically determines the optimal rank for each layer in low-rank adaptation (LoRA). Traditional LoRA assigns the same rank to all layers, which can lead to suboptimal performance and inefficiency. AutoLoRA addresses this by using selection variables to determine which rank-1 matrices in the update matrix are retained. These variables are learned through meta learning, allowing the framework to adaptively determine the optimal rank for each layer. The method involves retraining the update matrices on combined training and validation data after determining the optimal ranks. Experiments on natural language understanding, generation, and sequence labeling tasks show that AutoLoRA outperforms LoRA and other baselines in terms of performance and efficiency. AutoLoRA achieves better results by allowing different layers to have distinct ranks, which better reflects their varying importance to downstream tasks. The method also reduces the number of trainable parameters compared to full fine-tuning, making it more efficient. The framework is implemented using PyTorch and is publicly available. AutoLoRA's effectiveness is demonstrated across various NLP tasks, including GLUE benchmark, E2E, WebNLG, and BioNLP datasets. The method is also more efficient in terms of computational cost compared to grid search-based approaches. AutoLoRA's ability to automatically determine optimal ranks without manual tuning makes it a promising approach for parameter-efficient fine-tuning of large language models.AutoLoRA is a meta learning-based framework that automatically determines the optimal rank for each layer in low-rank adaptation (LoRA). Traditional LoRA assigns the same rank to all layers, which can lead to suboptimal performance and inefficiency. AutoLoRA addresses this by using selection variables to determine which rank-1 matrices in the update matrix are retained. These variables are learned through meta learning, allowing the framework to adaptively determine the optimal rank for each layer. The method involves retraining the update matrices on combined training and validation data after determining the optimal ranks. Experiments on natural language understanding, generation, and sequence labeling tasks show that AutoLoRA outperforms LoRA and other baselines in terms of performance and efficiency. AutoLoRA achieves better results by allowing different layers to have distinct ranks, which better reflects their varying importance to downstream tasks. The method also reduces the number of trainable parameters compared to full fine-tuning, making it more efficient. The framework is implemented using PyTorch and is publicly available. AutoLoRA's effectiveness is demonstrated across various NLP tasks, including GLUE benchmark, E2E, WebNLG, and BioNLP datasets. The method is also more efficient in terms of computational cost compared to grid search-based approaches. AutoLoRA's ability to automatically determine optimal ranks without manual tuning makes it a promising approach for parameter-efficient fine-tuning of large language models.