This paper provides a comprehensive survey of recent deep Convolutional Neural Network (CNN) architectures, focusing on their architectural innovations and applications. CNNs have demonstrated exceptional performance in various computer vision tasks such as image classification, segmentation, object detection, and video processing. The survey highlights the importance of multiple feature extraction stages and the role of large datasets and advanced hardware in accelerating CNN research. Key advancements include the use of different activation and loss functions, parameter optimization, regularization, and architectural innovations. Notably, the exploitation of spatial and channel information, depth and width of the architecture, and multi-path information processing have gained significant attention. The paper classifies recent innovations in CNN architectures into seven categories: spatial exploitation, depth, multi-path, width, feature-map exploitation, channel boosting, and attention. It also discusses the basic components of CNNs, current challenges, and applications, providing a historical perspective on the development of CNNs from their inception to the present. The survey aims to help readers understand the design principles of CNNs and inspire further architectural innovations.This paper provides a comprehensive survey of recent deep Convolutional Neural Network (CNN) architectures, focusing on their architectural innovations and applications. CNNs have demonstrated exceptional performance in various computer vision tasks such as image classification, segmentation, object detection, and video processing. The survey highlights the importance of multiple feature extraction stages and the role of large datasets and advanced hardware in accelerating CNN research. Key advancements include the use of different activation and loss functions, parameter optimization, regularization, and architectural innovations. Notably, the exploitation of spatial and channel information, depth and width of the architecture, and multi-path information processing have gained significant attention. The paper classifies recent innovations in CNN architectures into seven categories: spatial exploitation, depth, multi-path, width, feature-map exploitation, channel boosting, and attention. It also discusses the basic components of CNNs, current challenges, and applications, providing a historical perspective on the development of CNNs from their inception to the present. The survey aims to help readers understand the design principles of CNNs and inspire further architectural innovations.