This chapter provides practical recommendations for training deep neural networks, focusing on hyper-parameter optimization and gradient-based optimization techniques. It begins by discussing the revival of artificial neural networks research after the 2006 breakthrough in Deep Learning, which introduced greedy layer-wise unsupervised pre-training. The chapter covers various unsupervised representation learning algorithms, such as auto-encoders and denoising auto-encoders, which are crucial for learning internal representations in deep models. It also delves into the theoretical advantages of deep learning, including the efficiency of distributed representations and the effectiveness of unsupervised learning in semi-supervised and multi-task settings. The chapter then explores the practical aspects of training deep neural networks, including the choice of hyper-parameters, such as the learning rate, mini-batch size, and momentum, and the importance of early stopping to avoid overfitting. It emphasizes the role of automatic differentiation in computing gradients efficiently and the use of symbolic computation in libraries like Theano to simplify gradient calculations. The chapter also discusses the selection of model-specific hyper-parameters, such as the number of hidden units and regularization coefficients, and provides guidelines for tuning these parameters to achieve optimal performance. Finally, the chapter addresses open questions and challenges in training deep architectures, highlighting the need for further theoretical analysis and empirical validation to improve optimization methods and generalization performance.This chapter provides practical recommendations for training deep neural networks, focusing on hyper-parameter optimization and gradient-based optimization techniques. It begins by discussing the revival of artificial neural networks research after the 2006 breakthrough in Deep Learning, which introduced greedy layer-wise unsupervised pre-training. The chapter covers various unsupervised representation learning algorithms, such as auto-encoders and denoising auto-encoders, which are crucial for learning internal representations in deep models. It also delves into the theoretical advantages of deep learning, including the efficiency of distributed representations and the effectiveness of unsupervised learning in semi-supervised and multi-task settings. The chapter then explores the practical aspects of training deep neural networks, including the choice of hyper-parameters, such as the learning rate, mini-batch size, and momentum, and the importance of early stopping to avoid overfitting. It emphasizes the role of automatic differentiation in computing gradients efficiently and the use of symbolic computation in libraries like Theano to simplify gradient calculations. The chapter also discusses the selection of model-specific hyper-parameters, such as the number of hidden units and regularization coefficients, and provides guidelines for tuning these parameters to achieve optimal performance. Finally, the chapter addresses open questions and challenges in training deep architectures, highlighting the need for further theoretical analysis and empirical validation to improve optimization methods and generalization performance.