The paper introduces a novel multi-modal fusion-based approach named Dual-3DM³-AD for early and accurate diagnosis of Alzheimer's Disease (AD). The approach combines MRI and PET image scans to enhance the diagnostic accuracy. The preprocessing techniques, such as Quaternion Non-local Means Denoising Algorithm (QNLM), Morphology function, and Block Divider Model (BDM), are used to improve image quality. The Mixed-transformer with Furthered U-Net is employed for semantic segmentation to identify relevant regions in the images. The multi-scale feature extraction module extracts features from both segmented images, which are then aggregated using the Densely Connected Feature Aggregator Module (DCFAM). A multi-head attention mechanism reduces feature dimensionality, followed by a softmax layer for multi-class diagnosis. The proposed model outperforms existing methods with 98% accuracy, 97.5% sensitivity, 97.5% specificity, and 98.2% F-measure, as demonstrated through various performance metrics and ROC curves. The study highlights the importance of multi-modal fusion and advanced preprocessing techniques in improving the accuracy of AD diagnosis.The paper introduces a novel multi-modal fusion-based approach named Dual-3DM³-AD for early and accurate diagnosis of Alzheimer's Disease (AD). The approach combines MRI and PET image scans to enhance the diagnostic accuracy. The preprocessing techniques, such as Quaternion Non-local Means Denoising Algorithm (QNLM), Morphology function, and Block Divider Model (BDM), are used to improve image quality. The Mixed-transformer with Furthered U-Net is employed for semantic segmentation to identify relevant regions in the images. The multi-scale feature extraction module extracts features from both segmented images, which are then aggregated using the Densely Connected Feature Aggregator Module (DCFAM). A multi-head attention mechanism reduces feature dimensionality, followed by a softmax layer for multi-class diagnosis. The proposed model outperforms existing methods with 98% accuracy, 97.5% sensitivity, 97.5% specificity, and 98.2% F-measure, as demonstrated through various performance metrics and ROC curves. The study highlights the importance of multi-modal fusion and advanced preprocessing techniques in improving the accuracy of AD diagnosis.