This study investigates the use of Large Language Models (LLMs) to generate targeted lateral spear phishing emails within a large educational organization, specifically a university with approximately 9,000 employees. The research addresses two critical issues: the lack of large-scale studies on LLM-facilitated phishing and the inadequate capabilities of existing anti-phishing infrastructure to detect LLM-generated attacks. The study spans 11 months and involves the creation of various phishing email templates, including those crafted by LLMs. The results show that LLM-generated phishing emails are highly effective, with a data entry rate of 10% among recipients. The study also evaluates the effectiveness of existing email filtering infrastructure and proposes machine learning-based detection techniques with an F1-score of 98.96. The findings highlight the need for integrating LLM-generated phishing email detection methods into anti-phishing infrastructure and updating organizational policies to mitigate these threats. The research contributes to the understanding of the evolving cyber threat landscape and provides insights into the effectiveness of different phishing tactics and the importance of advanced detection methods.This study investigates the use of Large Language Models (LLMs) to generate targeted lateral spear phishing emails within a large educational organization, specifically a university with approximately 9,000 employees. The research addresses two critical issues: the lack of large-scale studies on LLM-facilitated phishing and the inadequate capabilities of existing anti-phishing infrastructure to detect LLM-generated attacks. The study spans 11 months and involves the creation of various phishing email templates, including those crafted by LLMs. The results show that LLM-generated phishing emails are highly effective, with a data entry rate of 10% among recipients. The study also evaluates the effectiveness of existing email filtering infrastructure and proposes machine learning-based detection techniques with an F1-score of 98.96. The findings highlight the need for integrating LLM-generated phishing email detection methods into anti-phishing infrastructure and updating organizational policies to mitigate these threats. The research contributes to the understanding of the evolving cyber threat landscape and provides insights into the effectiveness of different phishing tactics and the importance of advanced detection methods.