PV-Tuning: Beyond Straight-Through Estimation for Extreme LLM Compression This paper introduces PV-Tuning, a new framework for fine-tuning large language models (LLMs) under extreme compression, aiming to achieve better accuracy and efficiency in quantization. The paper highlights the limitations of existing methods, particularly the use of straight-through estimation (STE) in quantization-aware fine-tuning, which can lead to sub-optimal results. PV-Tuning is designed to optimize both continuous and discrete parameters in quantized representations, leading to improved performance in terms of accuracy and compression efficiency. The paper discusses the challenges of extreme LLM compression, where models are compressed to 1-2 bits per parameter, making them suitable for deployment on resource-constrained devices. It reviews existing techniques for quantization, including improved weight representations and algorithms for learning these representations. However, the paper argues that current fine-tuning strategies are not optimal, and there is a need for more effective methods. PV-Tuning is proposed as a representation-agnostic framework that generalizes and improves upon existing fine-tuning strategies. It provides convergence guarantees in restricted cases and has been shown to outperform prior techniques for highly-performant models such as Llama and Mistral. The paper demonstrates that PV-Tuning achieves the first Pareto-optimal quantization for Llama-2 family models at 2 bits per parameter. The paper also discusses the practical implementation of PV-Tuning, including the use of adaptive learning rates and subspace updates to improve efficiency. It evaluates the performance of PV-Tuning on various LLMs, showing that it outperforms existing methods in terms of accuracy and compression efficiency. The paper concludes that PV-Tuning provides a significant improvement in the accuracy vs. bit-width trade-off for extreme LLM compression.PV-Tuning: Beyond Straight-Through Estimation for Extreme LLM Compression This paper introduces PV-Tuning, a new framework for fine-tuning large language models (LLMs) under extreme compression, aiming to achieve better accuracy and efficiency in quantization. The paper highlights the limitations of existing methods, particularly the use of straight-through estimation (STE) in quantization-aware fine-tuning, which can lead to sub-optimal results. PV-Tuning is designed to optimize both continuous and discrete parameters in quantized representations, leading to improved performance in terms of accuracy and compression efficiency. The paper discusses the challenges of extreme LLM compression, where models are compressed to 1-2 bits per parameter, making them suitable for deployment on resource-constrained devices. It reviews existing techniques for quantization, including improved weight representations and algorithms for learning these representations. However, the paper argues that current fine-tuning strategies are not optimal, and there is a need for more effective methods. PV-Tuning is proposed as a representation-agnostic framework that generalizes and improves upon existing fine-tuning strategies. It provides convergence guarantees in restricted cases and has been shown to outperform prior techniques for highly-performant models such as Llama and Mistral. The paper demonstrates that PV-Tuning achieves the first Pareto-optimal quantization for Llama-2 family models at 2 bits per parameter. The paper also discusses the practical implementation of PV-Tuning, including the use of adaptive learning rates and subspace updates to improve efficiency. It evaluates the performance of PV-Tuning on various LLMs, showing that it outperforms existing methods in terms of accuracy and compression efficiency. The paper concludes that PV-Tuning provides a significant improvement in the accuracy vs. bit-width trade-off for extreme LLM compression.