This paper presents a comprehensive analysis of deep neural network (DNN) models submitted to the ImageNet challenge over the past four years, focusing on their computational requirements and accuracy. The authors compare various DNN architectures in terms of accuracy, memory footprint, parameters, operations count, inference time, and power consumption. Key findings include: 1. **Inefficiency of Fully Connected Layers**: Fully connected layers are inefficient for smaller batches of images. 2. **Hyperbolic Relationship Between Accuracy and Inference Time**: There is a hyperbolic relationship between accuracy and inference time, meaning a small increase in accuracy can significantly increase computational time. 3. **Energy Constraints**: Energy constraints set an upper bound on the maximum achievable accuracy and model complexity. 4. **Reliability of Operations Count**: The number of operations is a reliable estimate of inference time. The analysis highlights the importance of these metrics for designing efficient DNNs, particularly in practical applications where resource utilisation, power consumption, and latency are critical. The paper also demonstrates that GoogLeNet is the most efficient architecture in terms of parameter space utilisation, achieving up to 10× more information per parameter compared to AlexNet and 20× compared to VGG-16.This paper presents a comprehensive analysis of deep neural network (DNN) models submitted to the ImageNet challenge over the past four years, focusing on their computational requirements and accuracy. The authors compare various DNN architectures in terms of accuracy, memory footprint, parameters, operations count, inference time, and power consumption. Key findings include: 1. **Inefficiency of Fully Connected Layers**: Fully connected layers are inefficient for smaller batches of images. 2. **Hyperbolic Relationship Between Accuracy and Inference Time**: There is a hyperbolic relationship between accuracy and inference time, meaning a small increase in accuracy can significantly increase computational time. 3. **Energy Constraints**: Energy constraints set an upper bound on the maximum achievable accuracy and model complexity. 4. **Reliability of Operations Count**: The number of operations is a reliable estimate of inference time. The analysis highlights the importance of these metrics for designing efficient DNNs, particularly in practical applications where resource utilisation, power consumption, and latency are critical. The paper also demonstrates that GoogLeNet is the most efficient architecture in terms of parameter space utilisation, achieving up to 10× more information per parameter compared to AlexNet and 20× compared to VGG-16.