On the Affinity, Rationality, and Diversity of Hierarchical Topic Modeling

On the Affinity, Rationality, and Diversity of Hierarchical Topic Modeling

1 Feb 2024 | Xiaobao Wu, Fengjun Pan, Thong Nguyen, Yichao Feng, Chaoqun Liu, Cong-Duy Nguyen, Anh Tuan Luu
This paper proposes a novel neural hierarchical topic model called Transport Plan and Context-aware Hierarchical Topic Model (TraCo) to address the issues of low affinity, rationality, and diversity in hierarchical topic modeling. Existing methods struggle with producing topic hierarchies that are not sufficiently related, rational, or diverse. TraCo introduces a new Transport Plan Dependency (TPD) method that regularizes topic hierarchy building with sparse and balanced dependencies, improving affinity and diversity. It also proposes a Context-aware Disentangled Decoder (CDD) that distributes different semantic granularity to topics at different levels, enhancing rationality. Experiments on benchmark datasets show that TraCo outperforms state-of-the-art baselines, significantly improving the affinity, rationality, and diversity of hierarchical topic modeling. The model is evaluated on various downstream tasks, including text classification and clustering, demonstrating its effectiveness in generating high-quality topic hierarchies. The results show that TraCo consistently outperforms other methods, producing more accurate and diverse topic distributions. The paper also includes case studies illustrating the model's ability to discover affinitive, rational, and diverse topic hierarchies. Overall, TraCo provides a more effective solution for hierarchical topic modeling by addressing the key challenges of affinity, rationality, and diversity.This paper proposes a novel neural hierarchical topic model called Transport Plan and Context-aware Hierarchical Topic Model (TraCo) to address the issues of low affinity, rationality, and diversity in hierarchical topic modeling. Existing methods struggle with producing topic hierarchies that are not sufficiently related, rational, or diverse. TraCo introduces a new Transport Plan Dependency (TPD) method that regularizes topic hierarchy building with sparse and balanced dependencies, improving affinity and diversity. It also proposes a Context-aware Disentangled Decoder (CDD) that distributes different semantic granularity to topics at different levels, enhancing rationality. Experiments on benchmark datasets show that TraCo outperforms state-of-the-art baselines, significantly improving the affinity, rationality, and diversity of hierarchical topic modeling. The model is evaluated on various downstream tasks, including text classification and clustering, demonstrating its effectiveness in generating high-quality topic hierarchies. The results show that TraCo consistently outperforms other methods, producing more accurate and diverse topic distributions. The paper also includes case studies illustrating the model's ability to discover affinitive, rational, and diverse topic hierarchies. Overall, TraCo provides a more effective solution for hierarchical topic modeling by addressing the key challenges of affinity, rationality, and diversity.
Reach us at info@study.space