Stochastic Gradient Descent (SGD) is a widely used optimization algorithm for training machine learning models, particularly in large-scale learning scenarios. This chapter discusses the theory and practical applications of SGD, its variants, and recommendations for its effective use. SGD is a stochastic approximation of gradient descent, where instead of computing the gradient over the entire dataset, it uses a single randomly selected example at each iteration. This makes SGD computationally efficient and suitable for large datasets. SGD is particularly effective when the training set is large, as it allows for faster computation and can directly optimize the expected risk by randomly sampling from the data distribution. The convergence of SGD depends on the learning rate schedule, with decreasing learning rates often used to ensure convergence. While SGD is asymptotically slower than batch gradient descent, it is more suitable for large-scale learning due to its lower memory and computational requirements. The chapter provides recommendations for using SGD, including data preprocessing, gradient checking, and learning rate selection. It also discusses the use of second-order methods like second-order stochastic gradient descent (2SGD), which can improve convergence but may not significantly reduce variance. The chapter also covers specific applications of SGD, such as training linear models with L2 regularization and handling sparse data. Experiments show that SGD and its variants, such as averaged stochastic gradient descent (ASGD), perform well on various machine learning tasks, including text classification and sequence labeling. The chapter concludes that SGD is a versatile and effective algorithm for large-scale learning, and its successful application depends on careful experimentation and attention to gradient computation.Stochastic Gradient Descent (SGD) is a widely used optimization algorithm for training machine learning models, particularly in large-scale learning scenarios. This chapter discusses the theory and practical applications of SGD, its variants, and recommendations for its effective use. SGD is a stochastic approximation of gradient descent, where instead of computing the gradient over the entire dataset, it uses a single randomly selected example at each iteration. This makes SGD computationally efficient and suitable for large datasets. SGD is particularly effective when the training set is large, as it allows for faster computation and can directly optimize the expected risk by randomly sampling from the data distribution. The convergence of SGD depends on the learning rate schedule, with decreasing learning rates often used to ensure convergence. While SGD is asymptotically slower than batch gradient descent, it is more suitable for large-scale learning due to its lower memory and computational requirements. The chapter provides recommendations for using SGD, including data preprocessing, gradient checking, and learning rate selection. It also discusses the use of second-order methods like second-order stochastic gradient descent (2SGD), which can improve convergence but may not significantly reduce variance. The chapter also covers specific applications of SGD, such as training linear models with L2 regularization and handling sparse data. Experiments show that SGD and its variants, such as averaged stochastic gradient descent (ASGD), perform well on various machine learning tasks, including text classification and sequence labeling. The chapter concludes that SGD is a versatile and effective algorithm for large-scale learning, and its successful application depends on careful experimentation and attention to gradient computation.