This paper explores whether deep neural networks are necessary for achieving high accuracy in tasks such as speech recognition and computer vision. The authors empirically demonstrate that shallow feed-forward nets can learn the same complex functions as deep nets and achieve similar or even better accuracies with fewer parameters. They achieve this by training shallow nets to mimic deep nets using model compression techniques. The study focuses on two datasets: TIMIT for phoneme recognition and CIFAR-10 for image recognition. On both datasets, shallow nets trained to mimic deep models perform similarly to or better than deep models, even when the shallow nets have fewer parameters. The paper also discusses the benefits of shallow nets, such as faster training and better scalability with parallel hardware, and suggests that the complexity of deep nets may not be essential for achieving high accuracy. The findings highlight the potential for developing better learning algorithms for shallow nets to match the performance of deep nets.This paper explores whether deep neural networks are necessary for achieving high accuracy in tasks such as speech recognition and computer vision. The authors empirically demonstrate that shallow feed-forward nets can learn the same complex functions as deep nets and achieve similar or even better accuracies with fewer parameters. They achieve this by training shallow nets to mimic deep nets using model compression techniques. The study focuses on two datasets: TIMIT for phoneme recognition and CIFAR-10 for image recognition. On both datasets, shallow nets trained to mimic deep models perform similarly to or better than deep models, even when the shallow nets have fewer parameters. The paper also discusses the benefits of shallow nets, such as faster training and better scalability with parallel hardware, and suggests that the complexity of deep nets may not be essential for achieving high accuracy. The findings highlight the potential for developing better learning algorithms for shallow nets to match the performance of deep nets.