| Citation: | YANG Sichen, WANG Huafeng, WANG Yuehai, et al. Super-resolution reconstruction algorithm based on deep learning mechanism and wavelet fusion[J]. Journal of Beijing University of Aeronautics and Astronautics, 2020, 46(1): 189-197. doi: 10.13700/j.bh.1001-5965.2019.0146(in Chinese)
|
Deep learning technology has developed rapidly in the field of super-resolution reconstruction. In order to further improve the quality and visual effect of reconstructed images, this paper proposes a super-resolution reconstruction based on wavelet transform and generative adversarial networks (GAN) for the unnatural problem of texture reconstruction based on the super-resolution reconstruction algorithm of GAN. In this paper, each component of the wavelet decomposition in the GAN is trained in separate subnets to realize the prediction of wavelet coefficients by the network. Effectively reconstruct high-resolution images with rich global information and local texture details. The experimental results show that the peak signal-to-noise ratio (PSNR) and structural similarity of the objective evaluation index of the reconstructed image can be improved by at least 0.99 dB and 0.031, respectively, based on the algorithm of GAN.
| [1] |
曹杨.基于学习的超分辨率复原技术研究[D].北京: 北京工业大学, 2009: 3. http://www.wanfangdata.com.cn/details/detail.do?_type=degree&id=Y1569683
CAO Y.Research on learning-based super-resolution restoration technology[D].Beijing: Beijing University of Technology, 2009: 3(in Chinese). http://www.wanfangdata.com.cn/details/detail.do?_type=degree&id=Y1569683
|
| [2] |
TRINH D H, LUONG M, DIBOS F, et al.Novel example-based method for super-resolution and denoising of medical images[J].IEEE Transactions on Image Processing, 2014, 23(4):1882-1895. doi: 10.1109/TIP.2014.2308422
|
| [3] |
LEI Z, YANG J, XUE B, et al.Super-resolution reconstruction of Chang'e-1 satellite CCD stereo camera images[J].Infrared and Laser Engineering, 2012, 41(2):404-408.
|
| [4] |
SHEN H, NG M K, LI P, et al.Super-resolution reconstruction algorithm to MODIS remote sensing images[J].Computer Journal, 2009, 52(1):90-100. http://cn.bing.com/academic/profile?id=499974c692e27f810c774c0356d3502f&encoded=0&v=paper_preview&mkt=zh-cn
|
| [5] |
石晓勃, 高树辉.基于警用图像处理系统对超分辨率图像重建分析[J].中国人民公安大学学报, 2016, 22(4):14-17. http://d.old.wanfangdata.com.cn/Periodical/gadxxb-zrkxb201604004
SHI X B, GAO B.Analysis of super-resolution image reconstruction based on police image processing system[J].Journal of Chinese People's Public Security University, 2016, 22(4):14-17(in Chinese). http://d.old.wanfangdata.com.cn/Periodical/gadxxb-zrkxb201604004
|
| [6] |
杨宇翔.图像超分辨率算法研究[D].北京: 中国科学技术大学, 2013: 14-28.
YANG Y X.Image super resolution algorithm research[D].Beijing: University of Science and Technology of China, 2013: 14-28(in Chinese).
|
| [7] |
PARKER J A, KENYON R V, TROXEL D E.Comparison of interpolating methods for image resampling[J].IEEE Transactions on Medical Imaging, 1983, 2(1):31-39. doi: 10.1109/TMI.1983.4307610
|
| [8] |
HOU H S, ANDREWS H.Cubic splines for image interpolation and digital filtering[J].IEEE Transactions on Acoustics Speech & Signal Processing, 1978, 26(6):508-517. doi: 10.1109-TASSP.1978.1163154/
|
| [9] |
LI X, ORCHARD M T.New edge-directed interpolation[J].IEEE Transactions on Image Processing, 2001, 10(10):1521-1527. doi: 10.1109/83.951537
|
| [10] |
OZKAN M K, TEKALP A M, SEZAN M I.POCS-based restoration of space-varying blurred images[J].IEEE Transactions on Image Processing, 1994, 3(4):450-454. doi: 10.1109/83.298398
|
| [11] |
IRANI M, PELEG S.Improving resolution by image registration[J].CVGIP Graphical Models & Image Processing, 1991, 53(3):231-239. doi: 10.1017-S1041610209990342/
|
| [12] |
SCHULTZ R R, STEVENSON R L.A Bayesian approach to image expansion for improved definition[J].IEEE Transactions on Image Processing, 1994, 3(3):233-242. http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=c4dbb80e6666be9055d6cde3254f2682
|
| [13] |
YANG J, WANG Z, LIN Z, et al.Coupled dictionary training for image super-resolution[J].IEEE Transactions on Image Processing, 2012, 21(8):3467-3478. doi: 10.1109/TIP.2012.2192127
|
| [14] |
DONG C, LO C C, HE K, et al.Learning a deep convolutional network for image super-resolution[C]//European Conference on Computer Vision.Berlin: Springer, 2014: 184-199. doi: 10.1007/978-3-319-10593-2_13
|
| [15] |
KIM J, LEE J K, LEE K M.Deeply-recursive convolutional network for image super-resolution[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Piscataway, NJ: IEEE Press, 2016: 1637-1645.
|
| [16] |
SHI W, CABALLERO J, HUSZAR F, et al.Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Piscataway, NJ: IEEE Press, 2016: 1874-1883. https://www.researchgate.net/publication/307925703_Real-Time_Single_Image_and_Video_Super-Resolution_Using_an_Efficient_Sub-Pixel_Convolutional_Neural_Network
|
| [17] |
TONG T, LI G, LIU X, et al.Image super-resolution using dense skip connections[C]//Proceedings of the IEEE International Conference on Computer Vision.Piscataway, NJ: IEEE Press, 2017: 4809-4817. https://www.researchgate.net/publication/322057608_Image_Super-Resolution_Using_Dense_Skip_Connections
|
| [18] |
LEDIG C, WANG Z, SHI W, et al.Photo-realistic single image super-resolution using a generative adversarial network[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Piscataway, NJ: IEEE Press, 2017: 4681-4690. https://www.researchgate.net/publication/308152499_Photo-Realistic_Single_Image_Super-Resolution_Using_a_Generative_Adversarial_Network
|
| [19] |
LAI W S, HUANG J B, AHUJA N, et al.Deep Laplacian pyramid networks for fast and accurate super-resolution[J].IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 1:5835-5843. http://cn.bing.com/academic/profile?id=87defcb6cbf436c9e84391087df5f972&encoded=0&v=paper_preview&mkt=zh-cn
|
| [20] |
GUO L, HE Z M.A projection on convex sets super-resolution algorithm using wavelet transform[C]//2008 9th International Conference on Signal Processing.Piscataway, NJ: IEEE Press, 2008: 1039-1041. https://www.researchgate.net/publication/224355735_A_projection_on_Convex_Sets_super-resolution_algorithm_using_wavelet_transform
|
| [21] |
AKBARZADEH S, HASSAN G, FAEZE V.An efficient single image super resolution algorithm based on wavelet transforms[C]//2015 9th Iranian Conference on Machine Vision and Image Processing (MVIP).Piscataway, NJ: IEEE Press, 2015: 111-114. https://www.researchgate.net/publication/284174229_An_efficient_single_image_super_resolution_algorithm_based_on_wavelet_transforms
|
| [22] |
KUMAR N, VERMA R, SETHI A.Convolutional neural networks for wavelet domain super resolution[J].Pattern Recognition Letters, 2017, 90:65-71. doi: 10.1016/j.patrec.2017.03.014
|
| [23] |
HUANG H, HE R, SUN Z, et al.Wavelet-SRNet: A wavelet-based CNN for multi-scale face super resolution[C]//Proceedings of the IEEE International Conference on Computer Vision.Piscataway, NJ: IEEE Press, 2017: 1689-1697. https://www.researchgate.net/publication/322058760_Wavelet-SRNet_A_Wavelet-Based_CNN_for_Multi-scale_Face_Super_Resolution
|
| [24] |
HE K, ZHANG X, REN S, et al.Deep residual learning for image recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Piscataway, NJ: IEEE Press, 2016: 770-778. https://www.researchgate.net/publication/307964703_Deep_Residual_Learning
|
| [25] |
LIM B, SON S, KIM H, et al.Enhanced deep residual networks for single image super-resolution[C]//2011 IEEE Computer Vision and Pattern Recognition Workshops.Piscataway, NJ: IEEE Press, 2017: 1132-1140. https://www.researchgate.net/publication/318337451_Enhanced_Deep_Residual_Networks_for_Single_Image_Super-Resolution
|
| [26] |
TIMOFTE R, DE SMET V, VAN GOOL L.Anchored neighborhood regression for fast example-based super-resolution[C]//Proceedings of the IEEE International Conference on Computer Vision. Piscataway, NJ: IEEE Press, 2013: 1920-1927. https://www.researchgate.net/publication/262323537_Anchored_Neighborhood_Regression_for_Fast_Example-Based_Super-Resolution
|
| [27] |
RUSSAKOVSKY O, DENG J, SU H, et al.ImageNet large scale visual recognition challenge[J].International Journal of Computer Vision, 2014, 115(3):211-252. http://d.old.wanfangdata.com.cn/NSTLHY/NSTL_HYCC0214533907/
|
| [28] |
BEVILACQUA M, ROUMY A, GUILLEMOT C, et al.Neighbor embedding based single-image super-resolution using semi-nonnegative matrix factorization[C]//2012 IEEE International Conference on Acoustics, Speech and Signal Processing(ICASSP).Piscataway, NJ: IEEE Press, 2012: 1289-1292. https://www.researchgate.net/publication/261466481_Neighbor_embedding_based_single-image_super-resolution_using_Semi-Nonnegative_Matrix_Factorization
|
| [29] |
ZEYDE R, ELAD M, PROTTER M.On single image scale-up using sparse-representations[C]//International Conference on Curves and Surfaces.Berlin: Springer, 2010: 711-730.
|
| [30] |
MARTIN D, FOWLKES C, TAL D, et al.A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics[C]//IEEE Conference on Computer Vision.Piscataway, NJ: IEEE Press, 2001: 416-423. https://www.researchgate.net/publication/3906161_A_Database_of_Human_Segmented_Natural_Images_and_its_Application_to_Evaluating_Segmentation_Algorithms_and_Measuring_Ecological_Statistics
|
| [31] |
WANG Z, BOVIK A C, SHEIKH H R, et al.Image quality assessment:From error visibility to structural similarity[J].IEEE Transactions on Image Processing, 2004, 13(4):600-612. doi: 10.1109/TIP.2003.819861
|
| [32] |
ZHANG L, ZHANG L, MOU X, et al.FSIM:A feature similarity index for image quality assessment[J].IEEE Transactions on Image Processing, 2011, 20(8):2378-2386. doi: 10.1109/TIP.2011.2109730
|
| [33] |
WANG Z, BOVIK A C.A universal image quality index[J].IEEE Signal Processing Letters, 2002, 9(3):81-84. http://d.old.wanfangdata.com.cn/OAPaper/oai_doaj-articles_9471bc783cfb47b797e41099fa1c8cf0
|
| [34] |
OJALA T, PIETILAINEN M, HARWOOD D.Performance evaluation of texture measures with classification based on Kullback discrimination of distributions[C]//Proceedings of 12th International Conference on Pattern Recognition.Piscataway, NJ: IEEE Press, 1994, 1: 582-585. https://www.researchgate.net/publication/3684008_Performance_evaluation_of_texture_measures_with_classification_based_on_Kullback_discrimination_of_distributions
|
Figures(6) / Tables(3)
Copyright © Journal of Beijing University of Aeronautics and Astronautics
Address: Editorial Department of Journal of Beijing University of Aeronautics and Astronautics, 37 Xueyuan Road, Haidian District, Beijing Post Code: 100191 Email: jbuaa@buaa.edu.cn
Supported by:
Beijing Renhe Information Technology Co., Ltd.
DownLoad:
