Medformer: A Multi-Granularity Patching Transformer for Medical Time-Series Classification

Medformer: A Multi-Granularity Patching Transformer for Medical Time-Series Classification

19 Oct 2024 | Yihe Wang*, Nan Huang*, Taida Li*, Yujun Yan, Xiang Zhang
Medformer is a multi-granularity patching transformer designed for medical time-series (MedTS) classification. It introduces three novel mechanisms: cross-channel patching to capture inter-channel correlations, multi-granularity embedding to capture features at different scales, and a two-stage multi-granularity self-attention mechanism to learn features and correlations within and across granularities. The model is evaluated on five public datasets, including three EEG and two ECG datasets, under both subject-dependent and subject-independent setups. Results show that Medformer outperforms 10 baselines, achieving top averaged rankings across five datasets on all six evaluation metrics. These findings highlight the effectiveness of Medformer in healthcare applications such as diagnosing Myocardial Infarction, Alzheimer's, and Parkinson's disease. The model is released with source code at https://github.com/DL4mHealth/Medformer. The paper discusses the challenges of MedTS classification, including capturing local temporal dynamics, inter-channel correlations, and multi-scale feature analysis. Medformer addresses these challenges by integrating cross-channel, multi-timestamp, and multi-granularity features, enhancing its capacity to capture subject-invariant representations. The model is evaluated in the subject-independent setup, which is more aligned with real-world applications, and demonstrates strong performance in this challenging scenario. The results confirm Medformer's capability to handle subject-independent challenges. The paper also presents ablation studies and additional experiments on human activity recognition datasets to demonstrate the model's learning ability on general time series with potential channel correlations. The conclusion highlights the effectiveness and robustness of Medformer in MedTS classification, with limitations including the need for careful tuning of patch lengths and the lack of specific mechanisms for subject-independent setups. Future work could explore automatic selection of patch lengths and decomposition of subject-specific features to enhance learning in subject-independent scenarios.Medformer is a multi-granularity patching transformer designed for medical time-series (MedTS) classification. It introduces three novel mechanisms: cross-channel patching to capture inter-channel correlations, multi-granularity embedding to capture features at different scales, and a two-stage multi-granularity self-attention mechanism to learn features and correlations within and across granularities. The model is evaluated on five public datasets, including three EEG and two ECG datasets, under both subject-dependent and subject-independent setups. Results show that Medformer outperforms 10 baselines, achieving top averaged rankings across five datasets on all six evaluation metrics. These findings highlight the effectiveness of Medformer in healthcare applications such as diagnosing Myocardial Infarction, Alzheimer's, and Parkinson's disease. The model is released with source code at https://github.com/DL4mHealth/Medformer. The paper discusses the challenges of MedTS classification, including capturing local temporal dynamics, inter-channel correlations, and multi-scale feature analysis. Medformer addresses these challenges by integrating cross-channel, multi-timestamp, and multi-granularity features, enhancing its capacity to capture subject-invariant representations. The model is evaluated in the subject-independent setup, which is more aligned with real-world applications, and demonstrates strong performance in this challenging scenario. The results confirm Medformer's capability to handle subject-independent challenges. The paper also presents ablation studies and additional experiments on human activity recognition datasets to demonstrate the model's learning ability on general time series with potential channel correlations. The conclusion highlights the effectiveness and robustness of Medformer in MedTS classification, with limitations including the need for careful tuning of patch lengths and the lack of specific mechanisms for subject-independent setups. Future work could explore automatic selection of patch lengths and decomposition of subject-specific features to enhance learning in subject-independent scenarios.
Reach us at info@futurestudyspace.com