This chapter provides practical recommendations for gradient-based training of deep architectures, focusing on hyperparameters and optimization techniques. It discusses the importance of hyperparameters in deep learning, particularly in the context of backpropagation and gradient-based optimization. The chapter outlines the use of unsupervised pre-training and supervised fine-tuning for deep neural networks, emphasizing the role of gradient descent and stochastic optimization. It also covers the use of auto-encoders, including denoising and contractive auto-encoders, which help in learning robust representations. The chapter addresses the challenges of training deep architectures, including the need for careful hyperparameter tuning, the impact of mini-batch size, learning rate, and momentum on training performance. It also discusses the importance of regularization techniques such as weight decay and sparsity constraints in preventing overfitting. The chapter concludes with open questions regarding the training of deep architectures and the optimization of neural networks.This chapter provides practical recommendations for gradient-based training of deep architectures, focusing on hyperparameters and optimization techniques. It discusses the importance of hyperparameters in deep learning, particularly in the context of backpropagation and gradient-based optimization. The chapter outlines the use of unsupervised pre-training and supervised fine-tuning for deep neural networks, emphasizing the role of gradient descent and stochastic optimization. It also covers the use of auto-encoders, including denoising and contractive auto-encoders, which help in learning robust representations. The chapter addresses the challenges of training deep architectures, including the need for careful hyperparameter tuning, the impact of mini-batch size, learning rate, and momentum on training performance. It also discusses the importance of regularization techniques such as weight decay and sparsity constraints in preventing overfitting. The chapter concludes with open questions regarding the training of deep architectures and the optimization of neural networks.