深度学习机制与小波融合的超分辨率重建算法

doi:10.13700/j.bh.1001-5965.2019.0146

北京航空航天大学学报 ›› 2020, Vol. 46 ›› Issue (1): 189-197.doi: 10.13700/j.bh.1001-5965.2019.0146

深度学习机制与小波融合的超分辨率重建算法

杨思晨¹, 王华锋², 王月海¹, 李锦涛², 王赟豪²

1. 北方工业大学信息学院, 北京 100144;
2. 北京航空航天大学软件学院, 北京 100083

收稿日期:2019-04-03 发布日期:2020-01-21
通讯作者: 王华锋 E-mail:wanghuafeng@buaa.edu.cn
作者简介:杨思晨,女,硕士研究生。主要研究方向:图像处理;王华锋,男,博士,副教授,硕士生导师。主要研究方向:计算机视觉。

Super-resolution reconstruction algorithm based on deep learning mechanism and wavelet fusion

YANG Sichen¹, WANG Huafeng², WANG Yuehai¹, LI Jintao², WANG Yunhao²

1. School of Information, North China University of Technology, Beijing 100144, China;
2. School of Software, Beihang University, Beijing 100083, China

Received:2019-04-03 Published:2020-01-21

摘要/Abstract

摘要： 深度学习技术在超分辨率重建领域中发展迅速。为了进一步提升重建图像的质量和视觉效果，针对基于生成对抗网络（GAN）的超分辨率重建算法重建图像的纹理放大后不自然的问题，提出了一种结合小波变换和生成对抗网络的超分辨率重建算法。所提算法在生成对抗网络中将小波分解的每个分量在各自独立的子网中进行训练，实现网络对小波系数的预测，有效地重建出具有丰富的全局信息和局部纹理细节信息的高分辨率图像。实验结果表明，对比基于生成对抗网络的算法，所提算法重建图像的客观评价指标峰值信噪比（PSNR）和结构相似性分别能提高至少0.99 dB和0.031。

关键词: 小波变换, 生成对抗网络(GAN), 超分辨率重建, 深度学习, 多分辨分析

Abstract: Deep learning technology has developed rapidly in the field of super-resolution reconstruction. In order to further improve the quality and visual effect of reconstructed images, this paper proposes a super-resolution reconstruction based on wavelet transform and generative adversarial networks (GAN) for the unnatural problem of texture reconstruction based on the super-resolution reconstruction algorithm of GAN. In this paper, each component of the wavelet decomposition in the GAN is trained in separate subnets to realize the prediction of wavelet coefficients by the network. Effectively reconstruct high-resolution images with rich global information and local texture details. The experimental results show that the peak signal-to-noise ratio (PSNR) and structural similarity of the objective evaluation index of the reconstructed image can be improved by at least 0.99 dB and 0.031, respectively, based on the algorithm of GAN.

Key words: wavelet transform, generative adversarial network (GAN), super-resolution reconstruction, deep learning, mult-iresolution analysis

中图分类号:

TP183
TP751

杨思晨, 王华锋, 王月海, 李锦涛, 王赟豪. 深度学习机制与小波融合的超分辨率重建算法[J]. 北京航空航天大学学报, 2020, 46(1): 189-197.

YANG Sichen, WANG Huafeng, WANG Yuehai, LI Jintao, WANG Yunhao. Super-resolution reconstruction algorithm based on deep learning mechanism and wavelet fusion[J]. Journal of Beijing University of Aeronautics and Astronautics, 2020, 46(1): 189-197.

参考文献

[1] 曹杨.基于学习的超分辨率复原技术研究[D].北京:北京工业大学, 2009:3.CAO Y.Research on learning-based super-resolution restoration technology[D].Beijing:Beijing University of Technology,2009:3(in Chinese).
[2] TRINH D H,LUONG M,DIBOS F,et al.Novel example-based method for super-resolution and denoising of medical images[J].IEEE Transactions on Image Processing,2014,23(4):1882-1895.
[3] LEI Z,YANG J,XUE B,et al.Super-resolution reconstruction of Chang'e-1 satellite CCD stereo camera images[J].Infrared and Laser Engineering,2012,41(2):404-408.
[4] SHEN H,NG M K,LI P,et al.Super-resolution reconstruction algorithm to MODIS remote sensing images[J].Computer Journal,2009,52(1):90-100.
[5] 石晓勃,高树辉.基于警用图像处理系统对超分辨率图像重建分析[J].中国人民公安大学学报,2016,22(4):14-17.SHI X B, GAO B.Analysis of super-resolution image reconstruction based on police image processing system[J].Journal of Chinese People's Public Security University,2016,22(4):14-17(in Chinese).
[6] 杨宇翔.图像超分辨率算法研究[D].北京:中国科学技术大学,2013:14-28.YANG Y X.Image super resolution algorithm research[D].Beijing:University of Science and Technology of China,2013:14-28(in Chinese).
[7] PARKER J A,KENYON R V,TROXEL D E.Comparison of interpolating methods for image resampling[J].IEEE Transactions on Medical Imaging,1983,2(1):31-39.
[8] HOU H S, ANDREWS H.Cubic splines for image interpolation and digital filtering[J].IEEE Transactions on Acoustics Speech & Signal Processing,1978,26(6):508-517.
[9] LI X,ORCHARD M T.New edge-directed interpolation[J].IEEE Transactions on Image Processing,2001,10(10):1521-1527.
[10] OZKAN M K,TEKALP A M,SEZAN M I.POCS-based restoration of space-varying blurred images[J].IEEE Transactions on Image Processing,1994,3(4):450-454.
[11] IRANI M,PELEG S.Improving resolution by image registration[J].CVGIP Graphical Models & Image Processing,1991,53(3):231-239.
[12] SCHULTZ R R,STEVENSON R L.A Bayesian approach to image expansion for improved definition[J].IEEE Transactions on Image Processing,1994,3(3):233-242.
[13] YANG J,WANG Z,LIN Z,et al.Coupled dictionary training for image super-resolution[J].IEEE Transactions on Image Processing,2012,21(8):3467-3478.
[14] DONG C,LO C C,HE K,et al.Learning a deep convolutional network for image super-resolution[C]//European Conference on Computer Vision.Berlin:Springer,2014:184-199.
[15] KIM J,LEE J K,LEE K M.Deeply-recursive convolutional network for image super-resolution[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Piscataway,NJ:IEEE Press,2016:1637-1645.
[16] SHI W,CABALLERO J,HUSZAR F,et al.Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Piscataway,NJ:IEEE Press,2016:1874-1883.
[17] TONG T,LI G,LIU X,et al.Image super-resolution using dense skip connections[C]//Proceedings of the IEEE International Conference on Computer Vision.Piscataway,NJ:IEEE Press,2017:4809-4817.
[18] LEDIG C,WANG Z,SHI W,et al.Photo-realistic single image super-resolution using a generative adversarial network[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Piscataway,NJ:IEEE Press,2017:4681-4690.
[19] LAI W S,HUANG J B,AHUJA N,et al.Deep Laplacian pyramid networks for fast and accurate super-resolution[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2017,1:5835-5843.
[20] GUO L, HE Z M.A projection on convex sets super-resolution algorithm using wavelet transform[C]//2008 9th International Conference on Signal Processing.Piscataway,NJ:IEEE Press,2008:1039-1041.
[21] AKBARZADEH S,HASSAN G,FAEZE V.An efficient single image super resolution algorithm based on wavelet transforms[C]//2015 9th Iranian Conference on Machine Vision and Image Processing (MVIP).Piscataway,NJ:IEEE Press,2015:111-114.
[22] KUMAR N,VERMA R,SETHI A.Convolutional neural networks for wavelet domain super resolution[J].Pattern Recognition Letters,2017,90:65-71.
[23] HUANG H,HE R,SUN Z,et al.Wavelet-SRNet:A wavelet-based CNN for multi-scale face super resolution[C]//Proceedings of the IEEE International Conference on Computer Vision.Piscataway,NJ:IEEE Press,2017:1689-1697.
[24] HE K,ZHANG X,REN S,et al.Deep residual learning for image recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Piscataway,NJ:IEEE Press, 2016:770-778.
[25] LIM B,SON S,KIM H,et al.Enhanced deep residual networks for single image super-resolution[C]//2011 IEEE Computer Vision and Pattern Recognition Workshops.Piscataway,NJ:IEEE Press,2017:1132-1140.
[26] TIMOFTE R,DE SMET V,VAN GOOL L.Anchored neighborhood regression for fast example-based super-resolution[C]//Proceedings of the IEEE International Conference on Computer Vision. Piscataway,NJ:IEEE Press,2013:1920-1927.
[27] RUSSAKOVSKY O,DENG J,SU H,et al.ImageNet large scale visual recognition challenge[J].International Journal of Computer Vision, 2014,115(3):211-252.
[28] BEVILACQUA M,ROUMY A,GUILLEMOT C,et al.Neighbor embedding based single-image super-resolution using semi-nonnegative matrix factorization[C]//2012 IEEE International Conference on Acoustics,Speech and Signal Processing(ICASSP).Piscataway,NJ:IEEE Press,2012:1289-1292.
[29] ZEYDE R,ELAD M,PROTTER M.On single image scale-up using sparse-representations[C]//International Conference on Curves and Surfaces.Berlin:Springer,2010:711-730.
[30] MARTIN D,FOWLKES C,TAL D,et al.A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics[C]//IEEE Conference on Computer Vision.Piscataway,NJ:IEEE Press,2001:416-423.
[31] WANG Z,BOVIK A C,SHEIKH H R,et al.Image quality assessment:From error visibility to structural similarity[J].IEEE Transactions on Image Processing,2004,13(4):600-612.
[32] ZHANG L,ZHANG L,MOU X,et al.FSIM:A feature similarity index for image quality assessment[J].IEEE Transactions on Image Processing,2011,20(8):2378-2386.
[33] WANG Z,BOVIK A C.A universal image quality index[J].IEEE Signal Processing Letters,2002,9(3):81-84.
[34] OJALA T,PIETILAINEN M,HARWOOD D.Performance evaluation of texture measures with classification based on Kullback discrimination of distributions[C]//Proceedings of 12th International Conference on Pattern Recognition.Piscataway,NJ:IEEE Press,1994,1:582-585.

深度学习机制与小波融合的超分辨率重建算法

Super-resolution reconstruction algorithm based on deep learning mechanism and wavelet fusion

RichHTML

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

本文评价

[1]	谢颖贤, 武迎春, 王玉梅, 赵贤凌, 王安红. 基于小波域清晰度评价的光场全聚焦图像融合[J]. 北京航空航天大学学报, 2019, 45(9): 1848-1854.
[2]	张克明, 蔡远文, 任元. 基于生成对抗网络的航天异常事件检测方法[J]. 北京航空航天大学学报, 2019, 45(7): 1329-1336.
[3]	魏宏喜, 张越. 基于生成对抗网络的零样本图像分类[J]. 北京航空航天大学学报, 2019, 45(12): 2345-2350.
[4]	张新峰, 郭宇桐, 蔡轶珩, 孙萌. 基于DCNN和全连接CRF的舌图像分割算法[J]. 北京航空航天大学学报, 2019, 45(12): 2364-2374.
[5]	赵国, 秦世引. PMMWI与VI优势互补的人体隐蔽违禁物检测与定位[J]. 北京航空航天大学学报, 2019, 45(10): 2011-2025.
[6]	郭琳, 秦世引. 遥感图像飞机目标高效搜检深度学习优化算法[J]. 北京航空航天大学学报, 2019, 45(1): 159-173.
[7]	袁梅, 商富凯, 董韶鹏. 基于经验小波变换的复合材料板声发射源定位[J]. 北京航空航天大学学报, 2018, 44(7): 1395-1401.
[8]	韩京冶, 许福, 陈志泊, 刘辉. 一种基于深度学习的交互式电话号码识别方法[J]. 北京航空航天大学学报, 2018, 44(5): 1074-1080.
[9]	王鑫, 吴际, 刘超, 杨海燕, 杜艳丽, 牛文生. 基于LSTM循环神经网络的故障时间序列预测[J]. 北京航空航天大学学报, 2018, 44(4): 772-784.
[10]	车畅畅, 王华伟, 倪晓梅, 洪骥宇. 基于深度学习的航空发动机故障融合诊断[J]. 北京航空航天大学学报, 2018, 44(3): 621-628.
[11]	许轲, 吴凤鸽, 赵军锁. 基于深度强化学习的软件定义卫星姿态控制算法[J]. 北京航空航天大学学报, 2018, 44(12): 2651-2659.
[12]	王鑫, 侯志强, 余旺盛, 戴铂, 金泽芬芬. 基于深度稀疏学习的鲁棒视觉跟踪[J]. 北京航空航天大学学报, 2017, 43(12): 2554-2563.
[13]	石泉, 李景文, 杨威, 曾虹程, 张豪杰. 基于小波变换的多方位角SAR图像融合方法[J]. 北京航空航天大学学报, 2017, 43(10): 2135-2142.
[14]	王召斌, 符赛, 尚尚, 翟国富. 弹用电磁继电器贮存退化试验及其寿命预测方法[J]. 北京航空航天大学学报, 2016, 42(12): 2610-2619.
[15]	陈浩, 郭军海, 齐巍. 基于经验小波变换的目标加速度估计算法[J]. 北京航空航天大学学报, 2015, 41(1): 154-159.