The paper introduces a novel convolutional auto-encoder (CAE) for unsupervised feature learning, which forms a convolutional neural network (CNN) when stacked. Each CAE is trained using conventional online gradient descent without additional regularization. The max-pooling layer is essential for learning biologically plausible features consistent with previous approaches. Initializing a CNN with filters from a trained CAE stack yields superior performance on benchmarks such as MNIST and CIFAR10. The CAE architecture shares weights across locations in the input, preserving spatial locality and learning non-trivial features. The max-pooling layer introduces sparsity in the hidden representation, enhancing filter selectivity and generalization. The paper demonstrates that pre-trained CNNs using CAE stacks outperform randomly initialized nets, with the best results on CIFAR10 for any unsupervised method trained on raw data.The paper introduces a novel convolutional auto-encoder (CAE) for unsupervised feature learning, which forms a convolutional neural network (CNN) when stacked. Each CAE is trained using conventional online gradient descent without additional regularization. The max-pooling layer is essential for learning biologically plausible features consistent with previous approaches. Initializing a CNN with filters from a trained CAE stack yields superior performance on benchmarks such as MNIST and CIFAR10. The CAE architecture shares weights across locations in the input, preserving spatial locality and learning non-trivial features. The max-pooling layer introduces sparsity in the hidden representation, enhancing filter selectivity and generalization. The paper demonstrates that pre-trained CNNs using CAE stacks outperform randomly initialized nets, with the best results on CIFAR10 for any unsupervised method trained on raw data.