The paper introduces Dual-3DM $ ^{3} $ -AD, a novel multi-modal fusion-based approach for early multi-class Alzheimer's diagnosis using MRI and PET scans. The model enhances image quality through preprocessing techniques like Quaternion Non-local Means Denoising Algorithm (QNLM), Morphology function, and Block Divider Model (BDM). It employs a Mixed-transformer with Furthered U-Net for semantic segmentation and feature extraction, followed by a multi-scale feature extraction module and Densely Connected Feature Aggregator Module (DCFAM) to aggregate features. A multi-head attention mechanism reduces feature dimensionality, and a softmax layer performs multi-class diagnosis. The model outperforms existing methods with 98% accuracy, 97.8% sensitivity, 97.5% specificity, and 98.2% f-measure. The framework combines MRI and PET data to provide a comprehensive analysis, enhancing diagnostic accuracy. The model is validated using ADNI data, demonstrating superior performance in classification and diagnostic accuracy. The study highlights the importance of multi-modal data fusion and advanced preprocessing techniques in improving Alzheimer's diagnosis.The paper introduces Dual-3DM $ ^{3} $ -AD, a novel multi-modal fusion-based approach for early multi-class Alzheimer's diagnosis using MRI and PET scans. The model enhances image quality through preprocessing techniques like Quaternion Non-local Means Denoising Algorithm (QNLM), Morphology function, and Block Divider Model (BDM). It employs a Mixed-transformer with Furthered U-Net for semantic segmentation and feature extraction, followed by a multi-scale feature extraction module and Densely Connected Feature Aggregator Module (DCFAM) to aggregate features. A multi-head attention mechanism reduces feature dimensionality, and a softmax layer performs multi-class diagnosis. The model outperforms existing methods with 98% accuracy, 97.8% sensitivity, 97.5% specificity, and 98.2% f-measure. The framework combines MRI and PET data to provide a comprehensive analysis, enhancing diagnostic accuracy. The model is validated using ADNI data, demonstrating superior performance in classification and diagnostic accuracy. The study highlights the importance of multi-modal data fusion and advanced preprocessing techniques in improving Alzheimer's diagnosis.