The paper introduces melSTExp, a multimodal deep learning approach that predicts gene expression levels using whole-slide images (WSIs) stained with Hematoxylin and Eosin (H&E) and spatial transcriptomics (ST) data. The method integrates multimodal information from H&E images and ST data, leveraging the Transformer encoder and Densenet-121 encoder. Each spot detected by ST is treated as a "word," and its intrinsic features and spatial context are integrated through a self-attention mechanism. Contrastive learning is employed to enhance the model's predictive capability by fusing image features with spot features. The proposed method is evaluated on three datasets: HER2+, cSCC, and Alex+10x, demonstrating superior performance in predicting spatial gene expression compared to existing methods. melSTExp also shows promise in interpreting cancer-specific overexpressed genes, identifying immune-related genes, and preserving original gene expression patterns. The source code is available at <https://github.com/shizhiceng/melSTExp>.The paper introduces melSTExp, a multimodal deep learning approach that predicts gene expression levels using whole-slide images (WSIs) stained with Hematoxylin and Eosin (H&E) and spatial transcriptomics (ST) data. The method integrates multimodal information from H&E images and ST data, leveraging the Transformer encoder and Densenet-121 encoder. Each spot detected by ST is treated as a "word," and its intrinsic features and spatial context are integrated through a self-attention mechanism. Contrastive learning is employed to enhance the model's predictive capability by fusing image features with spot features. The proposed method is evaluated on three datasets: HER2+, cSCC, and Alex+10x, demonstrating superior performance in predicting spatial gene expression compared to existing methods. melSTExp also shows promise in interpreting cancer-specific overexpressed genes, identifying immune-related genes, and preserving original gene expression patterns. The source code is available at <https://github.com/shizhiceng/melSTExp>.