17 May 2024 | Xinhao Zhang, Zaitian Wang, Lu Jiang, Wanfu Gao, Pengfei Wang, Kunpeng Liu
This paper proposes TFWT, a novel feature weighting method for tabular data using the Transformer architecture. Traditional feature weighting methods assume all features are equally important, which can lead to suboptimal performance in complex datasets. TFWT leverages the self-attention mechanism of Transformers to capture complex feature dependencies and assign appropriate weights to discrete and continuous features. It also employs a reinforcement learning strategy to fine-tune the weighting process. The method aligns discrete and continuous features into uniform-length vectors, encodes them using a Transformer, and decodes the embeddings into feature weights. A reinforcement learning approach is used to refine the weights based on feedback from downstream tasks. The method is evaluated on various real-world datasets and downstream tasks, showing significant performance improvements over baseline methods. The results demonstrate that TFWT effectively enhances feature weighting in tabular data analysis by capturing intricate contextual relationships and reducing information redundancy. The method outperforms existing approaches in terms of accuracy, F1 score, and variance reduction. The experiments show that TFWT achieves superior performance across multiple metrics and datasets, highlighting its effectiveness in improving classification tasks.This paper proposes TFWT, a novel feature weighting method for tabular data using the Transformer architecture. Traditional feature weighting methods assume all features are equally important, which can lead to suboptimal performance in complex datasets. TFWT leverages the self-attention mechanism of Transformers to capture complex feature dependencies and assign appropriate weights to discrete and continuous features. It also employs a reinforcement learning strategy to fine-tune the weighting process. The method aligns discrete and continuous features into uniform-length vectors, encodes them using a Transformer, and decodes the embeddings into feature weights. A reinforcement learning approach is used to refine the weights based on feedback from downstream tasks. The method is evaluated on various real-world datasets and downstream tasks, showing significant performance improvements over baseline methods. The results demonstrate that TFWT effectively enhances feature weighting in tabular data analysis by capturing intricate contextual relationships and reducing information redundancy. The method outperforms existing approaches in terms of accuracy, F1 score, and variance reduction. The experiments show that TFWT achieves superior performance across multiple metrics and datasets, highlighting its effectiveness in improving classification tasks.