This paper provides a comprehensive survey of recent advances in Convolutional Neural Networks (CNNs). It covers improvements in various aspects of CNNs, including layer design, activation functions, loss functions, regularization, optimization, and fast computation. The authors also discuss the application of CNNs in computer vision, speech, and natural language processing. The paper begins by introducing the basic components of CNNs, such as convolutional, pooling, and fully-connected layers. It then delves into recent advancements in these layers, including tiled convolution, transposed convolution, dilated convolution, network-in-network, and inception modules. The paper also explores different pooling methods, activation functions like ReLU, Leaky ReLU, Parametric ReLU, and ELU, and loss functions such as hinge loss, softmax loss, contrastive loss, and triplet loss. Additionally, it discusses regularization techniques like $\ell_p$-norm regularization, Dropout, and DropConnect, as well as optimization methods including data augmentation and weight initialization. The paper concludes with a discussion on the future directions and challenges in the field of CNNs.This paper provides a comprehensive survey of recent advances in Convolutional Neural Networks (CNNs). It covers improvements in various aspects of CNNs, including layer design, activation functions, loss functions, regularization, optimization, and fast computation. The authors also discuss the application of CNNs in computer vision, speech, and natural language processing. The paper begins by introducing the basic components of CNNs, such as convolutional, pooling, and fully-connected layers. It then delves into recent advancements in these layers, including tiled convolution, transposed convolution, dilated convolution, network-in-network, and inception modules. The paper also explores different pooling methods, activation functions like ReLU, Leaky ReLU, Parametric ReLU, and ELU, and loss functions such as hinge loss, softmax loss, contrastive loss, and triplet loss. Additionally, it discusses regularization techniques like $\ell_p$-norm regularization, Dropout, and DropConnect, as well as optimization methods including data augmentation and weight initialization. The paper concludes with a discussion on the future directions and challenges in the field of CNNs.