This paper presents the results of fine-tuning large language models (LLMs) for detecting vulnerabilities in Java source code. The authors leverage WizardCoder, an advanced version of the state-of-the-art LLM StarCoder, and adapt it for vulnerability detection through further fine-tuning. To accelerate training, they modify the training procedure and investigate optimal training regimes. For imbalanced datasets with many more negative examples than positive ones, they explore different techniques to improve classification performance. The finetuned WizardCoder model achieves improved ROC AUC and F1 measures on both balanced and imbalanced vulnerability datasets compared to CodeBERT-like models, demonstrating the effectiveness of adapting pre-trained LLMs for vulnerability detection in source code. Key contributions include finetuning the state-of-the-art code LLM, WizardCoder, increasing its training speed without performance harm, optimizing the training procedure and regimes, handling class imbalance, and improving performance on challenging vulnerability detection datasets. This research highlights the potential for transfer learning by fine-tuning large pre-trained language models for specialized source code analysis tasks.This paper presents the results of fine-tuning large language models (LLMs) for detecting vulnerabilities in Java source code. The authors leverage WizardCoder, an advanced version of the state-of-the-art LLM StarCoder, and adapt it for vulnerability detection through further fine-tuning. To accelerate training, they modify the training procedure and investigate optimal training regimes. For imbalanced datasets with many more negative examples than positive ones, they explore different techniques to improve classification performance. The finetuned WizardCoder model achieves improved ROC AUC and F1 measures on both balanced and imbalanced vulnerability datasets compared to CodeBERT-like models, demonstrating the effectiveness of adapting pre-trained LLMs for vulnerability detection in source code. Key contributions include finetuning the state-of-the-art code LLM, WizardCoder, increasing its training speed without performance harm, optimizing the training procedure and regimes, handling class imbalance, and improving performance on challenging vulnerability detection datasets. This research highlights the potential for transfer learning by fine-tuning large pre-trained language models for specialized source code analysis tasks.