This paper introduces a novel multimodal contrastive pretraining framework for electrocardiogram (ECG) signals, called ECG Semantic Integrator (ESI), which aims to improve the quality and robustness of learned representations of 12-lead ECG signals. The framework consists of two key components: Cardio Query Assistant (CQA) and ESI. CQA integrates a retrieval-augmented generation (RAG) pipeline to leverage large language models (LLMs) and external medical knowledge to generate detailed textual descriptions of ECGs. The generated text is enriched with information about demographics and waveform patterns. ESI integrates both contrastive and captioning loss to pretrain ECG encoders for enhanced representations. The proposed approach is validated through various downstream tasks, including arrhythmia detection and ECG-based subject identification. Experimental results demonstrate substantial improvements over strong baselines in these tasks. The ESI framework shows the potential of combining multimodal pretraining to improve the analysis of ECG signals. The study also addresses the challenges of ECG signal processing, including the need for large-scale and high-quality annotated training samples, and the variability in terminology and detail across different ECG datasets and sources. The proposed framework utilizes a two-step multimodal contrastive pretraining approach to enhance the representations learned from ECG signals. The CQA framework generates standardized and enriched textual descriptions for ECGs by leveraging the capability of RAG to retrieve relevant information from ECG textbooks. The ESI framework aligns ECG signals with their corresponding text annotations to pretrain the encoders for an enhanced semantic understanding of ECG content. The contributions of the study include the introduction of a RAG-based ECG description generation pipeline CQA and the development of an ESI framework with both contrastive and captioning capability in pretraining to train an ECG foundation model on approximately 650,000 12-lead ECG signals. The results show that the proposed method achieves promising performances in arrhythmia detection and ECG-based user identification, with improvements in AUC scores and accuracy compared to prior methods. The study also highlights the benefits of multimodal learning for ECG analysis and the value of integrating captioning loss with contrastive pretraining.This paper introduces a novel multimodal contrastive pretraining framework for electrocardiogram (ECG) signals, called ECG Semantic Integrator (ESI), which aims to improve the quality and robustness of learned representations of 12-lead ECG signals. The framework consists of two key components: Cardio Query Assistant (CQA) and ESI. CQA integrates a retrieval-augmented generation (RAG) pipeline to leverage large language models (LLMs) and external medical knowledge to generate detailed textual descriptions of ECGs. The generated text is enriched with information about demographics and waveform patterns. ESI integrates both contrastive and captioning loss to pretrain ECG encoders for enhanced representations. The proposed approach is validated through various downstream tasks, including arrhythmia detection and ECG-based subject identification. Experimental results demonstrate substantial improvements over strong baselines in these tasks. The ESI framework shows the potential of combining multimodal pretraining to improve the analysis of ECG signals. The study also addresses the challenges of ECG signal processing, including the need for large-scale and high-quality annotated training samples, and the variability in terminology and detail across different ECG datasets and sources. The proposed framework utilizes a two-step multimodal contrastive pretraining approach to enhance the representations learned from ECG signals. The CQA framework generates standardized and enriched textual descriptions for ECGs by leveraging the capability of RAG to retrieve relevant information from ECG textbooks. The ESI framework aligns ECG signals with their corresponding text annotations to pretrain the encoders for an enhanced semantic understanding of ECG content. The contributions of the study include the introduction of a RAG-based ECG description generation pipeline CQA and the development of an ESI framework with both contrastive and captioning capability in pretraining to train an ECG foundation model on approximately 650,000 12-lead ECG signals. The results show that the proposed method achieves promising performances in arrhythmia detection and ECG-based user identification, with improvements in AUC scores and accuracy compared to prior methods. The study also highlights the benefits of multimodal learning for ECG analysis and the value of integrating captioning loss with contrastive pretraining.