This paper introduces a novel training-free method for CLIP-based adaptation using Gaussian Discriminant Analysis (GDA). GDA, a classical algorithm, assumes that features of each class follow Gaussian distributions with identical covariance. By leveraging Bayes' formula, the classifier can be expressed in terms of the class means and covariance, which can be estimated from the data without additional training. The method integrates GDA with the zero-shot classifier from CLIP to ensemble the visual and textual modalities. Extensive experiments on 17 datasets validate that the proposed method outperforms or achieves comparable results to state-of-the-art training-free and training-required methods in few-shot classification, imbalanced learning, and out-of-distribution generalization. Additionally, the method is extended to base-to-new generalization and unsupervised learning, demonstrating its effectiveness in these scenarios as well. The code for the method is publicly available at <https://github.com/mrflogs/ICLR24>.This paper introduces a novel training-free method for CLIP-based adaptation using Gaussian Discriminant Analysis (GDA). GDA, a classical algorithm, assumes that features of each class follow Gaussian distributions with identical covariance. By leveraging Bayes' formula, the classifier can be expressed in terms of the class means and covariance, which can be estimated from the data without additional training. The method integrates GDA with the zero-shot classifier from CLIP to ensemble the visual and textual modalities. Extensive experiments on 17 datasets validate that the proposed method outperforms or achieves comparable results to state-of-the-art training-free and training-required methods in few-shot classification, imbalanced learning, and out-of-distribution generalization. Additionally, the method is extended to base-to-new generalization and unsupervised learning, demonstrating its effectiveness in these scenarios as well. The code for the method is publicly available at <https://github.com/mrflogs/ICLR24>.