LEARNING WITH LOGICAL CONSTRAINTS BUT WITHOUT SHORTCUT SATISFACTION

LEARNING WITH LOGICAL CONSTRAINTS BUT WITHOUT SHORTCUT SATISFACTION

1 Mar 2024 | Zenan Li, Zehua Liu, Yuan Yao, Jingwei Xu, Taolue Chen, Xiaoxing Ma, Jian Lü
This paper addresses the issue of shortcut satisfaction in integrating logical constraints into deep learning, where models often overfit to easy satisfying assignments of logical constraints rather than fully satisfying them. To overcome this, the authors propose a new framework that introduces dual variables for logical connectives to encode how the constraints are satisfied. This approach ensures monotonicity in the satisfaction degree of logical constraints and improves interpretability and robustness. The framework is further extended to a variational learning framework, where the logical constraint is expressed as a distributional loss compatible with the original training loss. Theoretical analysis and experimental results on various tasks demonstrate the superior performance of the proposed approach in both model generalizability and constraint satisfaction.This paper addresses the issue of shortcut satisfaction in integrating logical constraints into deep learning, where models often overfit to easy satisfying assignments of logical constraints rather than fully satisfying them. To overcome this, the authors propose a new framework that introduces dual variables for logical connectives to encode how the constraints are satisfied. This approach ensures monotonicity in the satisfaction degree of logical constraints and improves interpretability and robustness. The framework is further extended to a variational learning framework, where the logical constraint is expressed as a distributional loss compatible with the original training loss. Theoretical analysis and experimental results on various tasks demonstrate the superior performance of the proposed approach in both model generalizability and constraint satisfaction.
Reach us at info@study.space
Understanding Learning with Logical Constraints but without Shortcut Satisfaction