This paper investigates the dual role of Large Language Models (LLMs) in defending against chat-based social engineering (CSE) attacks. The study introduces SEConvo, a novel dataset simulating CSE scenarios in academic and recruitment contexts, and proposes ConvoSentinel, a modular defense pipeline that enhances CSE detection at both message and conversation levels. The research highlights the challenges of using off-the-shelf LLMs for CSE detection, as they generate high-quality CSE content but have limited detection capabilities. ConvoSentinel integrates a retrieval-augmented generation (RAG) module that identifies malicious intent by comparing messages to a database of similar conversations, improving detection efficiency and reducing operational costs. The study evaluates the performance of LLMs in detecting CSE attempts using zero-shot and few-shot prompts, revealing that while LLMs can be manipulated to conduct CSE attacks, their detection capabilities are limited. ConvoSentinel outperforms baseline models in CSE detection, achieving higher accuracy and cost-effectiveness. The study also explores the effectiveness of message-level analysis in enhancing CSE detection and highlights the importance of interpretability in LLM-based defense systems. The research underscores the need for advanced strategies to leverage LLMs in cybersecurity, emphasizing the importance of robust defense mechanisms against LLM-initiated CSE attacks. The dataset and code are available for further research and development.This paper investigates the dual role of Large Language Models (LLMs) in defending against chat-based social engineering (CSE) attacks. The study introduces SEConvo, a novel dataset simulating CSE scenarios in academic and recruitment contexts, and proposes ConvoSentinel, a modular defense pipeline that enhances CSE detection at both message and conversation levels. The research highlights the challenges of using off-the-shelf LLMs for CSE detection, as they generate high-quality CSE content but have limited detection capabilities. ConvoSentinel integrates a retrieval-augmented generation (RAG) module that identifies malicious intent by comparing messages to a database of similar conversations, improving detection efficiency and reducing operational costs. The study evaluates the performance of LLMs in detecting CSE attempts using zero-shot and few-shot prompts, revealing that while LLMs can be manipulated to conduct CSE attacks, their detection capabilities are limited. ConvoSentinel outperforms baseline models in CSE detection, achieving higher accuracy and cost-effectiveness. The study also explores the effectiveness of message-level analysis in enhancing CSE detection and highlights the importance of interpretability in LLM-based defense systems. The research underscores the need for advanced strategies to leverage LLMs in cybersecurity, emphasizing the importance of robust defense mechanisms against LLM-initiated CSE attacks. The dataset and code are available for further research and development.