**VuLLM: Generalization-Enhanced Code Vulnerability Detection via Multi-Task Instruction Fine-Tuning** **Authors:** Xiaohu Du, Ming Wen, Jiahao Zhu, Zifan Xie, Bin Ji, Huijun Liu, Xuanhua Shi, Hai Jin **Institution:** School of Cyber Science and Engineering, Huazhong University of Science and Technology; School of Computer Science and Technology, Huazhong University of Science and Technology; College of Computer, National University of Defense Technology **Abstract:** Code Pre-trained Models (CodePTMs) have shown promising results in automated code vulnerability detection, but they struggle with generalization due to their shallow understanding of code vulnerabilities. To address this, we introduce VuLLM, a novel framework that integrates multi-task learning with Large Language Models (LLMs) to enhance the detection of deep-seated vulnerability features. VuLLM constructs two auxiliary tasks: vulnerability localization and vulnerability interpretation, leveraging GPT-4 to generate interpretations. This approach forces the model to capture root causes rather than overfitting to spurious features. Experiments on six large datasets demonstrate that VuLLM outperforms seven state-of-the-art models in effectiveness, generalization, and robustness. **Introduction:** CodePTMs like CodeBERT, GraphCodeBERT, and UniXcoder have achieved state-of-the-art results in automated code vulnerability detection. However, they struggle with generalization, especially with out-of-distribution (OOD) data. To tackle this, we propose VuLLM, which uses multi-task learning to enhance LLMs' understanding of vulnerabilities. The framework includes vulnerability localization and interpretation tasks, leveraging GPT-4 for interpretation generation. This approach improves the model's ability to capture root causes and enhances its robustness to adversarial attacks. **Methodology:** VuLLM consists of three main components: vulnerability feature extraction, vulnerability interpretation generation, and multi-task instruction fine-tuning. Vulnerability features are extracted from patches, and GPT-4 is used to generate interpretations. Multi-task instruction fine-tuning optimizes the model's response to specific directives, enhancing its adaptability and efficiency. **Evaluation:** Extensive experiments on six datasets show that VuLLM outperforms existing models in effectiveness, generalization, and robustness. Ablation studies confirm the importance of multi-task learning and data augmentation. Sensitivity analysis to hyperparameters and auxiliary task samples further validate the model's performance. **Conclusion:** VuLLM significantly enhances the capability of LLMs to detect code vulnerabilities by integrating multi-task learning and data augmentation. Its effectiveness, generalization, and robustness make it a promising approach for automated code vulnerability detection.**VuLLM: Generalization-Enhanced Code Vulnerability Detection via Multi-Task Instruction Fine-Tuning** **Authors:** Xiaohu Du, Ming Wen, Jiahao Zhu, Zifan Xie, Bin Ji, Huijun Liu, Xuanhua Shi, Hai Jin **Institution:** School of Cyber Science and Engineering, Huazhong University of Science and Technology; School of Computer Science and Technology, Huazhong University of Science and Technology; College of Computer, National University of Defense Technology **Abstract:** Code Pre-trained Models (CodePTMs) have shown promising results in automated code vulnerability detection, but they struggle with generalization due to their shallow understanding of code vulnerabilities. To address this, we introduce VuLLM, a novel framework that integrates multi-task learning with Large Language Models (LLMs) to enhance the detection of deep-seated vulnerability features. VuLLM constructs two auxiliary tasks: vulnerability localization and vulnerability interpretation, leveraging GPT-4 to generate interpretations. This approach forces the model to capture root causes rather than overfitting to spurious features. Experiments on six large datasets demonstrate that VuLLM outperforms seven state-of-the-art models in effectiveness, generalization, and robustness. **Introduction:** CodePTMs like CodeBERT, GraphCodeBERT, and UniXcoder have achieved state-of-the-art results in automated code vulnerability detection. However, they struggle with generalization, especially with out-of-distribution (OOD) data. To tackle this, we propose VuLLM, which uses multi-task learning to enhance LLMs' understanding of vulnerabilities. The framework includes vulnerability localization and interpretation tasks, leveraging GPT-4 for interpretation generation. This approach improves the model's ability to capture root causes and enhances its robustness to adversarial attacks. **Methodology:** VuLLM consists of three main components: vulnerability feature extraction, vulnerability interpretation generation, and multi-task instruction fine-tuning. Vulnerability features are extracted from patches, and GPT-4 is used to generate interpretations. Multi-task instruction fine-tuning optimizes the model's response to specific directives, enhancing its adaptability and efficiency. **Evaluation:** Extensive experiments on six datasets show that VuLLM outperforms existing models in effectiveness, generalization, and robustness. Ablation studies confirm the importance of multi-task learning and data augmentation. Sensitivity analysis to hyperparameters and auxiliary task samples further validate the model's performance. **Conclusion:** VuLLM significantly enhances the capability of LLMs to detect code vulnerabilities by integrating multi-task learning and data augmentation. Its effectiveness, generalization, and robustness make it a promising approach for automated code vulnerability detection.