Joint operation and attention block search for lightweight image restoration

作者:Shen, Hao; Zhao, Zhong-Qiu*; Liao, Wenrui; Tian, Weidong; Huang, De-Shuang
来源:Pattern Recognition, 2022, 132: 108909.
DOI:10.1016/j.patcog.2022.108909

摘要

Recently, block-based design methods have shown effectiveness in image restoration tasks, which are usually designed in a handcrafted manner and have computation and memory consumption challenges in practice. In this paper, we propose a joint operation and attention block search algorithm for im-age restoration, which focuses on searching for optimal combinations of operation blocks and atten-tion blocks. Specifically, we first construct two search spaces: operation block search space and atten-tion block search space. The former is used to explore the suitable operation of each layer and aims to construct a lightweight and effective operation search module (OSM). The latter is applied to dis-cover the optimal connection of various attention mechanisms and aims to enhance the feature expres-sion. The searched structure is called the attention search module (ASM). Then we combine OSM and ASM to construct a joint search module (JSM), which serves as the basic module to build the final net-work. Moreover, we propose a cross-scale fusion module (CSFM) to effectively integrate multiple hier-archical features from JSMs, which helps to mine feature corrections of intermediate layers. Extensive experiments on image super-resolution, gray image denoising, and JPEG image deblocking tasks demon-strate that our proposed network can achieve competitive performance. The source code is available on https://github.com/it-hao/JSNet .