This paper analyzes the asymptotic behaviors of Support Vector Machines (SVMs) with the Gaussian (RBF) kernel when the hyperparameters C (penalty parameter) and σ² (kernel width) take extreme values. The study reveals how these hyperparameters influence the classifier's performance, particularly in terms of generalization error. The key findings include: 1. **Severe Underfitting**: Occurs when C approaches 0 or σ² approaches 0 or infinity. In these cases, the classifier tends to assign all data points to the majority class, leading to poor generalization. 2. **Severe Overfitting**: Happens when σ² approaches 0 and C is large. The classifier becomes overly sensitive to training examples, misclassifying small regions around minority class examples. 3. **Strict Separation**: When C approaches infinity and σ² is fixed, the classifier strictly separates the two classes, which can be overfitting if the data contains noise. 4. **Convergence to Linear SVM**: When σ² approaches infinity and C is proportional to σ², the Gaussian kernel SVM converges to a Linear SVM with a corresponding penalty parameter. The analysis also shows that if complete model selection using the Gaussian kernel is done, there is no need to consider linear SVMs. The paper proposes a heuristic method for model selection that leverages these asymptotic behaviors to efficiently search for hyperparameters with small generalization errors. This method is competitive with traditional cross-validation approaches and is more efficient, especially for large datasets. The results are illustrated with a figure showing the asymptotic behaviors in the (log C, log σ²) space. The paper concludes that understanding these asymptotic behaviors helps in selecting optimal hyperparameters for SVMs with Gaussian kernels.This paper analyzes the asymptotic behaviors of Support Vector Machines (SVMs) with the Gaussian (RBF) kernel when the hyperparameters C (penalty parameter) and σ² (kernel width) take extreme values. The study reveals how these hyperparameters influence the classifier's performance, particularly in terms of generalization error. The key findings include: 1. **Severe Underfitting**: Occurs when C approaches 0 or σ² approaches 0 or infinity. In these cases, the classifier tends to assign all data points to the majority class, leading to poor generalization. 2. **Severe Overfitting**: Happens when σ² approaches 0 and C is large. The classifier becomes overly sensitive to training examples, misclassifying small regions around minority class examples. 3. **Strict Separation**: When C approaches infinity and σ² is fixed, the classifier strictly separates the two classes, which can be overfitting if the data contains noise. 4. **Convergence to Linear SVM**: When σ² approaches infinity and C is proportional to σ², the Gaussian kernel SVM converges to a Linear SVM with a corresponding penalty parameter. The analysis also shows that if complete model selection using the Gaussian kernel is done, there is no need to consider linear SVMs. The paper proposes a heuristic method for model selection that leverages these asymptotic behaviors to efficiently search for hyperparameters with small generalization errors. This method is competitive with traditional cross-validation approaches and is more efficient, especially for large datasets. The results are illustrated with a figure showing the asymptotic behaviors in the (log C, log σ²) space. The paper concludes that understanding these asymptotic behaviors helps in selecting optimal hyperparameters for SVMs with Gaussian kernels.