This study investigates the use of Large Language Models (LLMs) in generating lateral spear phishing emails targeting a large public university with approximately 9,000 employees over an 11-month period. It evaluates the effectiveness of LLM-generated phishing emails compared to human-generated ones and assesses the capability of the university's email filtering infrastructure to detect such attacks. The study also explores the impact of phishing training and warnings on reducing susceptibility to phishing attacks. The findings reveal that LLM-generated phishing emails are highly effective, with a significant percentage of recipients clicking on links and entering sensitive information. The study also highlights the need for updated organizational policies and advanced detection techniques to counter LLM-driven phishing threats. The research proposes machine learning-based detection methods that achieve an F1-score of 98.96 in identifying LLM-generated phishing emails. The study underscores the importance of continuous cybersecurity awareness training and the integration of advanced detection systems to enhance organizational resilience against sophisticated phishing attacks. The results emphasize the urgent need for updated policies and enhanced detection mechanisms to address the growing threat of LLM-generated phishing attacks.This study investigates the use of Large Language Models (LLMs) in generating lateral spear phishing emails targeting a large public university with approximately 9,000 employees over an 11-month period. It evaluates the effectiveness of LLM-generated phishing emails compared to human-generated ones and assesses the capability of the university's email filtering infrastructure to detect such attacks. The study also explores the impact of phishing training and warnings on reducing susceptibility to phishing attacks. The findings reveal that LLM-generated phishing emails are highly effective, with a significant percentage of recipients clicking on links and entering sensitive information. The study also highlights the need for updated organizational policies and advanced detection techniques to counter LLM-driven phishing threats. The research proposes machine learning-based detection methods that achieve an F1-score of 98.96 in identifying LLM-generated phishing emails. The study underscores the importance of continuous cybersecurity awareness training and the integration of advanced detection systems to enhance organizational resilience against sophisticated phishing attacks. The results emphasize the urgent need for updated policies and enhanced detection mechanisms to address the growing threat of LLM-generated phishing attacks.