Uformer: A General U-Shaped Transformer for Image Restoration

Uformer: A General U-Shaped Transformer for Image Restoration

25 Nov 2021 | Zhendong Wang, Xiaodong Cun, Jianmin Bao, Wengang Zhou, Jianzhuang Liu, Houqiang Li
Uformer is a novel Transformer-based architecture designed for image restoration tasks. It leverages the hierarchical encoder-decoder structure with Transformer blocks to capture both local and global dependencies in images. The key contributions of Uformer include: 1. **Locally-enhanced Window (LeWin) Transformer Block**: This block performs non-overlapping window-based self-attention, reducing computational complexity while capturing local context. It is more efficient than global self-attention on high-resolution feature maps. 2. **Learnable Multi-scale Restoration Modulator**: This modulator adjusts features in multiple layers of the decoder using a multi-scale spatial bias, enhancing the restoration quality for various image degradation types. Uformer is evaluated on several image restoration tasks, including denoising, motion deblurring, defocus deblurring, and deraining. Extensive experiments show that Uformer achieves state-of-the-art performance or comparable results compared to existing methods, demonstrating its effectiveness and efficiency. The code and models are available at <https://github.com/ZhendongWang6/Uformer>.Uformer is a novel Transformer-based architecture designed for image restoration tasks. It leverages the hierarchical encoder-decoder structure with Transformer blocks to capture both local and global dependencies in images. The key contributions of Uformer include: 1. **Locally-enhanced Window (LeWin) Transformer Block**: This block performs non-overlapping window-based self-attention, reducing computational complexity while capturing local context. It is more efficient than global self-attention on high-resolution feature maps. 2. **Learnable Multi-scale Restoration Modulator**: This modulator adjusts features in multiple layers of the decoder using a multi-scale spatial bias, enhancing the restoration quality for various image degradation types. Uformer is evaluated on several image restoration tasks, including denoising, motion deblurring, defocus deblurring, and deraining. Extensive experiments show that Uformer achieves state-of-the-art performance or comparable results compared to existing methods, demonstrating its effectiveness and efficiency. The code and models are available at <https://github.com/ZhendongWang6/Uformer>.
Reach us at info@study.space
[slides and audio] Uformer%3A A General U-Shaped Transformer for Image Restoration