End-to-End Neuro-Symbolic Reinforcement Learning with Textual Explanations

End-to-End Neuro-Symbolic Reinforcement Learning with Textual Explanations

2024 | Lirui Luo, Guoxi Zhang, Hongming Xu, Yaodong Yang, Cong Fang, Qing Li
Neuro-symbolic reinforcement learning (NS-RL) has emerged as a promising paradigm for explainable decision-making, characterized by the interpretability of symbolic policies. However, previous methods struggle to refine structured states with rewards due to computational efficiency issues and require extensive domain knowledge to interpret symbolic policies. This paper presents INSIGHT, an interpretable neuro-symbolic framework for visual reinforcement learning that jointly learns structured states and symbolic policies. The key idea is to distill vision foundation models into an efficient perception module and refine it during policy learning. Additionally, a pipeline is designed to prompt GPT-4 to generate textual explanations for learned policies and decisions, reducing cognitive load for users. The efficacy of INSIGHT is demonstrated on nine Atari tasks, showing improved performance over existing NS-RL approaches and providing GPT-generated explanations for policies and decisions. The contributions of INSIGHT include a framework that refines structured states with visual and reward information, a pipeline for generating natural language explanations, and empirical validation on Atari games.Neuro-symbolic reinforcement learning (NS-RL) has emerged as a promising paradigm for explainable decision-making, characterized by the interpretability of symbolic policies. However, previous methods struggle to refine structured states with rewards due to computational efficiency issues and require extensive domain knowledge to interpret symbolic policies. This paper presents INSIGHT, an interpretable neuro-symbolic framework for visual reinforcement learning that jointly learns structured states and symbolic policies. The key idea is to distill vision foundation models into an efficient perception module and refine it during policy learning. Additionally, a pipeline is designed to prompt GPT-4 to generate textual explanations for learned policies and decisions, reducing cognitive load for users. The efficacy of INSIGHT is demonstrated on nine Atari tasks, showing improved performance over existing NS-RL approaches and providing GPT-generated explanations for policies and decisions. The contributions of INSIGHT include a framework that refines structured states with visual and reward information, a pipeline for generating natural language explanations, and empirical validation on Atari games.
Reach us at info@study.space
[slides] End-to-End Neuro-Symbolic Reinforcement Learning with Textual Explanations | StudySpace