This paper introduces the concept of "linear classifier probes" to better understand the dynamics and roles of intermediate layers in neural networks. The authors propose using linear classifiers, referred to as "probes," trained independently of the model to monitor and measure the features at each layer. This approach helps in developing a better intuition about the model's behavior and diagnosing potential problems. The probes are applied to popular models such as Inception v3 and ResNet-50, and it is observed that the linear separability of features increases monotonically with depth. The paper also discusses related work on understanding neural networks, including linear classification with kernel PCA, generalization and transferability of layers, relevance propagation, and SVCCA. The authors provide practical considerations for using probes, such as dimensionality reduction and handling large feature sets. They demonstrate the effectiveness of probes through experiments on MNIST, ResNet-50, and Inception v3, showing how they can identify problematic behaviors in models. The paper concludes by discussing future directions, including extending the concept to other models and exploring the implications of training models to discourage certain layers from being useful to linear classifiers.This paper introduces the concept of "linear classifier probes" to better understand the dynamics and roles of intermediate layers in neural networks. The authors propose using linear classifiers, referred to as "probes," trained independently of the model to monitor and measure the features at each layer. This approach helps in developing a better intuition about the model's behavior and diagnosing potential problems. The probes are applied to popular models such as Inception v3 and ResNet-50, and it is observed that the linear separability of features increases monotonically with depth. The paper also discusses related work on understanding neural networks, including linear classification with kernel PCA, generalization and transferability of layers, relevance propagation, and SVCCA. The authors provide practical considerations for using probes, such as dimensionality reduction and handling large feature sets. They demonstrate the effectiveness of probes through experiments on MNIST, ResNet-50, and Inception v3, showing how they can identify problematic behaviors in models. The paper concludes by discussing future directions, including extending the concept to other models and exploring the implications of training models to discourage certain layers from being useful to linear classifiers.