This paper introduces a neuro-symbolic framework called Logic-Explainer to enhance the logical validity and alignment of ethical explanations generated by Large Language Models (LLMs). The framework integrates LLMs with an external backward-chaining solver to refine step-wise natural language explanations and jointly verify their correctness, reduce incompleteness, and minimize redundancy. Logic-Explainer uses an abductive-deductive approach to iteratively refine explanations by dropping irrelevant facts and generating missing premises through abductive inference, while also revising hypotheses via deductive inference. The framework is evaluated on ethical NLI benchmarks, demonstrating significant improvements in the accuracy of identifying underlying moral violations compared to in-context learning and Chain-of-Thought methods. Results show that Logic-Explainer increases the logical validity of ethical explanations from 22.9% to 65.1% and 10.3% to 55.2% on easy and hard settings, respectively, and reduces redundancy from 86.6% to 4.6% and 78.3% to 6.2% after three refinement cycles. The framework also generates formal proofs supporting the reasoning process. The paper also introduces a new corpus, ExplainEthics, containing 311 statements with generated explanations and annotated moral violations, to augment existing datasets and encourage future research in ethical NLI. The study highlights the effectiveness of neuro-symbolic methods for multi-step NLI and opens new opportunities to enhance the logical consistency, reliability, and alignment of LLMs.This paper introduces a neuro-symbolic framework called Logic-Explainer to enhance the logical validity and alignment of ethical explanations generated by Large Language Models (LLMs). The framework integrates LLMs with an external backward-chaining solver to refine step-wise natural language explanations and jointly verify their correctness, reduce incompleteness, and minimize redundancy. Logic-Explainer uses an abductive-deductive approach to iteratively refine explanations by dropping irrelevant facts and generating missing premises through abductive inference, while also revising hypotheses via deductive inference. The framework is evaluated on ethical NLI benchmarks, demonstrating significant improvements in the accuracy of identifying underlying moral violations compared to in-context learning and Chain-of-Thought methods. Results show that Logic-Explainer increases the logical validity of ethical explanations from 22.9% to 65.1% and 10.3% to 55.2% on easy and hard settings, respectively, and reduces redundancy from 86.6% to 4.6% and 78.3% to 6.2% after three refinement cycles. The framework also generates formal proofs supporting the reasoning process. The paper also introduces a new corpus, ExplainEthics, containing 311 statements with generated explanations and annotated moral violations, to augment existing datasets and encourage future research in ethical NLI. The study highlights the effectiveness of neuro-symbolic methods for multi-step NLI and opens new opportunities to enhance the logical consistency, reliability, and alignment of LLMs.