The paper introduces FractalNet, a design strategy for neural network architecture based on self-similarity. Fractal networks are extremely deep networks whose structural layout is a truncated fractal, containing interacting subpaths of different lengths without pass-through connections. Unlike residual networks, FractalNet does not rely on residuals for success, achieving an error rate of 22.85% on CIFAR-100, matching the state-of-the-art held by residual networks. The authors demonstrate that residual representation is not essential for the success of ultra-deep convolutional neural networks. Fractal networks exhibit intriguing properties, such as being computationally efficient implicit unions of subnetworks of various depths. They also show the ability to extract high-performance fixed-depth subnetworks. To facilitate this, the authors develop drop-path, a regularization technique that prevents co-adaptation of subpaths, enabling anytime behavior where shallow subnetworks provide quick answers and deeper subnetworks offer more accurate results. The paper includes experimental comparisons with residual networks on CIFAR-10, CIFAR-100, and SVHN datasets, showing that FractalNet outperforms or matches the performance of ResNet without data augmentation.The paper introduces FractalNet, a design strategy for neural network architecture based on self-similarity. Fractal networks are extremely deep networks whose structural layout is a truncated fractal, containing interacting subpaths of different lengths without pass-through connections. Unlike residual networks, FractalNet does not rely on residuals for success, achieving an error rate of 22.85% on CIFAR-100, matching the state-of-the-art held by residual networks. The authors demonstrate that residual representation is not essential for the success of ultra-deep convolutional neural networks. Fractal networks exhibit intriguing properties, such as being computationally efficient implicit unions of subnetworks of various depths. They also show the ability to extract high-performance fixed-depth subnetworks. To facilitate this, the authors develop drop-path, a regularization technique that prevents co-adaptation of subpaths, enabling anytime behavior where shallow subnetworks provide quick answers and deeper subnetworks offer more accurate results. The paper includes experimental comparisons with residual networks on CIFAR-10, CIFAR-100, and SVHN datasets, showing that FractalNet outperforms or matches the performance of ResNet without data augmentation.