18 Mar 2024 | Miriam Wanner*, Seth Ebner*, Zhengping Jiang, Mark Dredze, Benjamin Van Durme
The paper investigates the impact of claim decomposition methods on the evaluation of textual support, particularly focusing on the recently proposed FACTSCORE metric. It finds that the quality of decomposition significantly affects the results of downstream textual support metrics. To address this, the authors introduce DECOMPSCORE, a metric designed to measure the quality of decomposition by evaluating the number of subclaims supported by the original claim. They also propose an LLM-based decomposition approach inspired by Bertrand Russell's logical atomism and neo-Davidsonian semantics, which generates more subclaims with higher atomicity and coverage compared to previous methods. The study concludes that the proposed decomposition method outperforms existing methods in terms of both qualitative and quantitative measures, enhancing the reliability of textual support evaluations.The paper investigates the impact of claim decomposition methods on the evaluation of textual support, particularly focusing on the recently proposed FACTSCORE metric. It finds that the quality of decomposition significantly affects the results of downstream textual support metrics. To address this, the authors introduce DECOMPSCORE, a metric designed to measure the quality of decomposition by evaluating the number of subclaims supported by the original claim. They also propose an LLM-based decomposition approach inspired by Bertrand Russell's logical atomism and neo-Davidsonian semantics, which generates more subclaims with higher atomicity and coverage compared to previous methods. The study concludes that the proposed decomposition method outperforms existing methods in terms of both qualitative and quantitative measures, enhancing the reliability of textual support evaluations.