This paper provides a comprehensive survey of recent advances in Convolutional Neural Networks (CNNs). CNNs have become a cornerstone of deep learning, achieving state-of-the-art results in various tasks such as image recognition, speech recognition, and natural language processing. The paper discusses improvements in CNNs across multiple aspects, including layer design, activation functions, loss functions, regularization, and optimization. It also covers various applications of CNNs in computer vision, speech, and natural language processing. The paper begins with an introduction to CNNs, tracing their development from the early work of Hubel and Wiesel to the modern architectures like LeNet-5, AlexNet, and more recent ones such as ResNet. It then provides an overview of the basic components of CNNs, including convolutional, pooling, and fully-connected layers. The paper then delves into various improvements in CNNs, such as tiled convolution, transposed convolution, dilated convolution, and network-in-network. It also discusses recent advancements in pooling layers, including Lp pooling, mixed pooling, stochastic pooling, spectral pooling, and spatial pyramid pooling. The paper further explores activation functions, including ReLU, Leaky ReLU, Parametric ReLU, Randomized ReLU, ELU, and Maxout. It discusses loss functions such as hinge loss, softmax loss, contrastive loss, triplet loss, and Kullback-Leibler divergence. The paper also covers regularization techniques like ℓp-norm regularization, dropout, and dropconnect. Finally, it discusses optimization techniques, including data augmentation and weight initialization. Overall, the paper provides a detailed review of the latest advancements in CNNs, highlighting the key improvements and their impact on various tasks. It serves as a valuable resource for researchers and practitioners interested in the latest developments in deep learning and CNNs.This paper provides a comprehensive survey of recent advances in Convolutional Neural Networks (CNNs). CNNs have become a cornerstone of deep learning, achieving state-of-the-art results in various tasks such as image recognition, speech recognition, and natural language processing. The paper discusses improvements in CNNs across multiple aspects, including layer design, activation functions, loss functions, regularization, and optimization. It also covers various applications of CNNs in computer vision, speech, and natural language processing. The paper begins with an introduction to CNNs, tracing their development from the early work of Hubel and Wiesel to the modern architectures like LeNet-5, AlexNet, and more recent ones such as ResNet. It then provides an overview of the basic components of CNNs, including convolutional, pooling, and fully-connected layers. The paper then delves into various improvements in CNNs, such as tiled convolution, transposed convolution, dilated convolution, and network-in-network. It also discusses recent advancements in pooling layers, including Lp pooling, mixed pooling, stochastic pooling, spectral pooling, and spatial pyramid pooling. The paper further explores activation functions, including ReLU, Leaky ReLU, Parametric ReLU, Randomized ReLU, ELU, and Maxout. It discusses loss functions such as hinge loss, softmax loss, contrastive loss, triplet loss, and Kullback-Leibler divergence. The paper also covers regularization techniques like ℓp-norm regularization, dropout, and dropconnect. Finally, it discusses optimization techniques, including data augmentation and weight initialization. Overall, the paper provides a detailed review of the latest advancements in CNNs, highlighting the key improvements and their impact on various tasks. It serves as a valuable resource for researchers and practitioners interested in the latest developments in deep learning and CNNs.