CoAtNet is a hybrid model that combines convolution and attention mechanisms to achieve state-of-the-art performance across various datasets and resource constraints. The model leverages two key insights: (1) depthwise convolution and self-attention can be naturally unified via simple relative attention; and (2) vertically stacking convolution and attention layers in a principled way improves generalization, capacity, and efficiency. Experiments show that CoAtNet achieves 86.0% ImageNet top-1 accuracy without extra data, 88.56% with pre-training on 13M images from ImageNet-21K, and 90.88% with pre-training on JFT-3B, outperforming previous models. The model's design balances the strengths of convolution and attention, achieving better generalization and capacity while maintaining efficiency. CoAtNet is evaluated on ImageNet-1K, ImageNet-21K, and JFT datasets, demonstrating superior performance under different data sizes and computational budgets. The model's architecture is designed to optimize both generalization and model capacity, with a multi-stage layout that includes convolution and attention blocks. The results show that CoAtNet achieves state-of-the-art performance in image classification, outperforming previous models in terms of accuracy and efficiency.CoAtNet is a hybrid model that combines convolution and attention mechanisms to achieve state-of-the-art performance across various datasets and resource constraints. The model leverages two key insights: (1) depthwise convolution and self-attention can be naturally unified via simple relative attention; and (2) vertically stacking convolution and attention layers in a principled way improves generalization, capacity, and efficiency. Experiments show that CoAtNet achieves 86.0% ImageNet top-1 accuracy without extra data, 88.56% with pre-training on 13M images from ImageNet-21K, and 90.88% with pre-training on JFT-3B, outperforming previous models. The model's design balances the strengths of convolution and attention, achieving better generalization and capacity while maintaining efficiency. CoAtNet is evaluated on ImageNet-1K, ImageNet-21K, and JFT datasets, demonstrating superior performance under different data sizes and computational budgets. The model's architecture is designed to optimize both generalization and model capacity, with a multi-stage layout that includes convolution and attention blocks. The results show that CoAtNet achieves state-of-the-art performance in image classification, outperforming previous models in terms of accuracy and efficiency.