| Citation: | HUANG S Y，HU H Y，YANG Y，et al. Image super-resolution reconstruction network based on expectation maximization self-attention residual[J]. Journal of Beijing University of Aeronautics and Astronautics，2024，50（2）：388-397 （in Chinese） doi: 10.13700/j.bh.1001-5965.2022.0401
|
In recent years, most deep learning-based image super-resolution (SR) reconstruction methods mainly improve the quality of image reconstruction by increasing the depth of the model, while also increasing the computational cost of the model. Additionally, a lot of networks have implemented the attention mechanism to enhance their capacity for feature extraction, but it is still challenging to properly understand the properties of various regions. In response to the above problems, this paper proposes a novel SR reconstruction network based on expectation maximization (EM) self-attention residual. The network constructs a feature-enhanced residual block by improving the basic residual block to better reuse the features extracted from the residual block. In order to increase the spatial correlation of the feature information, an EM self-attention residual block is constructed by introducing the EM self-attention mechanism, which is used to enhance the feature extraction capability of each module in the deep network model. Moreover, the feature extraction structure of the entire model is constructed by cascading EM self-attention residual blocks. Finally, a reconstructed high-resolution image is obtained through an up-sampling image reconstruction module.In order to verify the effectiveness of the proposed method, this paper has carried out comparison experiments with some mainstream methods. The experimental results show that the proposed method can achieve better subjective visual effects and better objective evaluation indicators on five popular widely used SR test datasets.
| [1] |
TAKEDA H, MILANFAR P, PROTTER M, et al. Super-resolution without explicit subpixel motion estimation[J]. IEEE Transactions on Image Processing, 2009, 18(9): 1958-1975. doi: 10.1109/TIP.2009.2023703
|
| [2] |
ZOU W W W, YUEN P C. Very low resolution face recognition problem[J]. IEEE Transactions on Image Processing, 2012, 21(1): 327-340.
|
| [3] |
SHI W, CABALLERO J, LEDIG C, et al. Cardiac image super-resolution with global correspondence using multi-atlas PatchMatch[C]//Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention. Berlin: Springer, 2013, 8151: 9-16.
|
| [4] |
ARUN P V, BUDDHIRAJU K M, PORWAL A, et al. CNN based spectral super-resolution of remote sensing images[J]. Signal Processing, 2020, 169: 107394. doi: 10.1016/j.sigpro.2019.107394
|
| [5] |
KEYS R. Cubic convolution interpolation for digital image processing[J]. IEEE Transactions on Acoustics, Speech, and Signal Processing, 1981, 29(6): 1153-1160. doi: 10.1109/TASSP.1981.1163711
|
| [6] |
MARQUIAN A, OSHER S J. Image super-resolution by TV-regularization and Bregman iteration[J]. Journal of Scientific Computing, 2008, 37(3): 367-382. doi: 10.1007/s10915-008-9214-8
|
| [7] |
DONG W, ZHANG L, SHI G, et al. Image deblurring and super-resolution by adaptive sparse domain selection and adaptive regularization[J]. IEEE Transactions on Image Processing, 2011, 20(7): 1838-1857. doi: 10.1109/TIP.2011.2108306
|
| [8] |
YANG J, WRIGHT J, HUANG T S, et al. Image super-resolution via sparse representation[J]. IEEE Transactions on Image Processing, 2010, 19(11): 2861-2873. doi: 10.1109/TIP.2010.2050625
|
| [9] |
ZEYDE R, ELAD M, PROTTER M. On single image scale-up using sparse-representations[C]//Proceedings of the International Conference on Curves and Surfaces. Berlin: Springer, 2010: 711-730.
|
| [10] |
HU Y, WANG N, TAO D, et al. SERF: A simple, effective, robust, and fast image super-resolver from cascaded linear regression[J]. IEEE Transactions on Image Processing, 2016, 25(9): 4091-4102. doi: 10.1109/TIP.2016.2580942
|
| [11] |
HUANG Y, LI J, GAO X, et al. Single image super-resolution via multiple mixture prior models[J]. IEEE Transactions on Image Processing, 2018, 27(12): 5904-5917. doi: 10.1109/TIP.2018.2860685
|
| [12] |
DONG C, LOY C C, HE K, et al. Image super-resolution using deep convolutional networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016, 38(2): 295-307. doi: 10.1109/TPAMI.2015.2439281
|
| [13] |
HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2016: 770-778.
|
| [14] |
KIM J, LEE J K, LEE K M. Accurate image super-resolution using very deep convolutional networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2016: 1646-1654.
|
| [15] |
KIM J, LEE J K, LEE K M. Deeply-recursive convolutional network for image super-resolution[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2016: 1637-1645.
|
| [16] |
LIM B, SON S, KIM H, et al. Enhanced deep residual networks for single image super-resolution[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2017: 1132-1140.
|
| [17] |
ZHANG Y L, LI K P, LI K, et al. Image super-resolution using very deep residual channel attention networks[C]//Proceedings of the European Conference on Computer Vision. Berlin: Springer, 2018, 11211: 286-310.
|
| [18] |
DAI T, CAI J, ZHANG Y, et al. Second-order attention network for single image super-resolution[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2019: 11057-11066.
|
| [19] |
HUI Z, WANG X, GAO X. Fast and accurate single image super-resolution via information distillation network[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2018: 723-731.
|
| [20] |
LI J, FANG F, MEI K, et al. Multi-scale residual network for image super-resolution[C]//Proceedings of the European Conference on Computer Vision. Berlin: Springer, 2018, 11212: 527-542.
|
| [21] |
AHN N, KANG B, SOHN K A. Fast, accurate, and lightweight super-resolution with cascading residual network[C]//Proceedings of the European Conference on Computer Vision. Berlin: Springer, 2018, 11214: 256-272.
|
| [22] |
LEDIG C, THEIS L, HUSZAR F, et al. Photo-realistic single image super-resolution using a generative adversarial network[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2017: 105-114.
|
| [23] |
WANG X, YU K, WU S, et al. ESRGAN: Enhanced super-resolution generative adversarial networks[C]//Proceedings of the European Conference on Computer Vision. Berlin: Springer, 2018, 11133: 63-79.
|
| [24] |
YADAV S, CHEN C, ROSS A. Relativistic discriminator: A one-class classifier for generalized iris presentation attack detection[C]//Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. Piscataway: IEEE Press, 2020: 2624-2633.
|
| [25] |
LEI J, LIU C, JIANG D. Fault diagnosis of wind turbine based on long short-term memory networks[J]. Renewable Energy, 2019, 133: 422-432. doi: 10.1016/j.renene.2018.10.031
|
| [26] |
TAN Z, WANG M, XIE J, et al. Deep semantic role labeling with self-attention[C]//Proceedings of the AAAI Conference on Artificial Intelligence. Palo Alto: AAAI, 2018.
|
| [27] |
WANG X, GIRSHICK R, GUPTA A, et al. Non-local neural networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2018: 7794-7803.
|
| [28] |
WOO S, PARK J, LEE J Y, et al. CBAM: Convolutional block attention module[C]//Proceedings of the European Conference on Computer Vision. Berlin: Springer, 2018, 11211: 3-19.
|
| [29] |
SANG H, ZHOU Q, ZHAO Y. PCANet: Pyramid convolutional attention network for semantic segmentation[J]. Image and Vision Computing, 2020, 103: 103997. doi: 10.1016/j.imavis.2020.103997
|
| [30] |
赵杨璐, 段丹丹, 胡饶敏, 等. 基于EM算法的混合模型中子总体个数的研究[J]. 数理统计与管理, 2020, 39(1): 35-50. doi: 10.13860/j.cnki.sltj.20190808-002
ZHAO Y L, DUAN D D, HU R M, et al. On the number of components in mixture model based on EM algorithm[J]. Journal of Applied Statistics and Management, 2020, 39(1): 35-50(in Chinese). doi: 10.13860/j.cnki.sltj.20190808-002
|
| [31] |
LI X, ZHONG Z, WU J, et al. Expectation-maximization attention networks for semantic segmentation[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. Piscataway: IEEE Press, 2019: 9166-9175.
|
| [32] |
AGUSTSSON E, TIMOFTE R. NTIRE 2017 challenge on sngle image super-resolution: Dataset and study[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2017: 1122-1131.
|
| [33] |
LAI W S, HUANG J B, AHUJA N, et al. Deep Laplacian pyramid networks for fast and accurate super-resolution[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2017: 5835-5843.
|
| [34] |
ZHANG K, ZOU W, ZHANG L. Learning a single convolutional super-resolution network for multiple degradations[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2018: 3262-3271.
|
| [35] |
TIAN C, ZHUGE R, WU Z, et al. Lightweight image super-resolution with enhanced CNN[J]. Knowledge-Based Systems, 2020, 205: 106235. doi: 10.1016/j.knosys.2020.106235
|
| [36] |
ZHAO H, KONG X, HE J, et al. Efficient image super-resolution using pixel attention[C]//Proceedings of the European Conference on Computer Vision. Berlin: Springer, 2020, 12537: 56-72.
|
| [37] |
WANG L G, DONG X Y, WANG Y Q, et al. Exploring sparsity in image super-resolution for efficient inference[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2021: 4915-4924.
|
| [38] |
BEVILACQUA M, ROUMY A, GUILLEMOT C, et al. Low-complexity single-image super-resolution based on nonnegative neighbor embedding[C]//Procedings of the British Machine Vision Conference. Berlin: Springer, 2012: 135.1-135.10.
|
| [39] |
MARTIN D, FOWLKES C, TAL D, et al. A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics[C]//Proceedings of the 8th IEEE International Conference on Computer Vision. Piscataway: IEEE Press, 2001, 2: 416-423.
|
| [40] |
HUANG J B, SINGH A, AHUJA N. Single image super-resolution from transformed self-exemplars[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2015: 5197-5206.
|
| [41] |
MATSUI Y, ITO K, ARAMAKI Y, et al. Sketch-based manga retrieval using Manga109 dataset[J]. Multimedia Tools and Applications, 2017, 76(20): 21811-21838. doi: 10.1007/s11042-016-4020-z
|
Figures(6) / Tables(4)
Copyright © Journal of Beijing University of Aeronautics and Astronautics
Address: Editorial Department of Journal of Beijing University of Aeronautics and Astronautics, 37 Xueyuan Road, Haidian District, Beijing Post Code: 100191 Email: jbuaa@buaa.edu.cn
Supported by:
Beijing Renhe Information Technology Co., Ltd.
DownLoad:
