基于注意机制的轻量化稠密连接网络单幅图像去雨

柴国强; 王大为; 芦宾; 李竹

doi:10.13700/j.bh.1001-5965.2021.0294

基于注意机制的轻量化稠密连接网络单幅图像去雨

doi: 10.13700/j.bh.1001-5965.2021.0294

山西师范大学物理与信息工程学院, 临汾 041000

基金项目:

国家自然科学基金 62201333

国家自然科学基金 62004119

山西省基础研究计划 20210302124647

详细信息

通讯作者:
芦宾, E-mail: lubinsxnu@sina.cn

中图分类号: TP391
计量
- 文章访问数: 284
- HTML全文浏览量: 113
- PDF下载量: 33
- 被引次数: 0
出版历程
- 收稿日期: 2021-06-03
- 录用日期: 2021-08-27
- 网络出版日期: 2021-09-15
- 整期出版日期: 2022-11-20

Lightweight densely connected network based on attention mechanism for single-image deraining

College of Physics and Information Engineering, Shanxi Normal University, Linfen 041000, China

Funds:

National Natural Science Foundation of China 62201333

National Natural Science Foundation of China 62004119

Basic Research Program of Shanxi Province 20210302124647

More Information

Corresponding author: LU Bin, E-mail: lubinsxnu@sina.cn

摘要

摘要:
图像中附着的雨条纹对背景造成的破坏严重影响了对图像信息的分析和后续研究。为了恢复被雨条纹破坏的背景纹理特征, 提出一种基于注意机制的轻量化稠密连接网络针对单幅图像进行去雨。注意机制有利于网络准确定位降雨区域, 稠密连接网络的使用增强了特征的复用, 缓解了梯度消失和模型退化问题。利用多尺度通道混洗深度可分离卷积实现网络轻量化设计, 降低了网络参数规模, 提升了网络运行效率。在合成数据集和真实数据集上的去雨结果表明, 所提算法在定量指标和定性分析上都优于现有算法。
- 注意机制 /
- 稠密连接网络 /
- 轻量化设计 /
- 图像去雨 /
- 深度学习
Abstract:
The rain streaks attached to the image seriously affect the analysis and follow-up research of the image information. To restore the background texture damaged by rain streaks, this paper proposes a lightweight densely connected network based on attention mechanism to remove rain from a single image. The attention mechanism makes the network locate in the rain streaks area accurately, and the densely connected network enhances the feature reuse, alleviates the gradient disappearance and model degradation problems. The utilization of multi-scale mix channel depthwise separable convolutions realizes lightweight design by reducing the scale of network parameters and enhancing the efficiency of network operation. Both qualitative and quantitative validations on synthetic and real-world datasets demonstrate that the proposed approach can achieve competitive performance in comparison with the state-of-the-art methods.
- attention mechanism /
- densely connected network /
- lightweight design /
- image deraining /
- deep learning

HTML全文

图 1 基于注意机制的改进稠密连接去雨网络结构

Figure 1. Structure of modified densely connected rain removal network based on attention mechanism

下载: 全尺寸图片幻灯片

图 2 多尺度聚集自适应雨条纹检测模块结构

Figure 2. Structure of multi-scale aggregation self-adapting rain streaks detection module

下载: 全尺寸图片幻灯片

图 3 注意图可视化

Figure 3. Visualization of attention map

下载: 全尺寸图片幻灯片

图 4 稠密连接网络

Figure 4. Densely connected network

下载: 全尺寸图片幻灯片

图 5 多尺度通道混洗的深度可分离卷积

Figure 5. Depthwise separable convolution with multi-scale channel shuffle

下载: 全尺寸图片幻灯片

图 6 不同算法合成数据集去雨结果可视化对比

Figure 6. Visualized comparison of different rain removal agorithms on synthetic dataset

下载: 全尺寸图片幻灯片

图 7 不同算法真实数据集去雨结果可视化对比

Figure 7. Visualized comparison of different rain removal algorithms on real dataset

下载: 全尺寸图片幻灯片

表 1 不同去雨算法PSNR和SSIM对比

Table 1. PSNR and SSIM comparison of different rain removal algorithms

数据集	算法	PSNR	SSIM
Rain100H	GMM	14.262	0.544
	DSC	15.661	0.423
	RESCAN	26.451	0.846
	ReHEN	27.972	0.864
	SPANet	26.762	0.836
	UMRL	26.840	0.854
	本文	29.362	0.898
Rain100L	GMM	29.111	0.880
	DSC	24.162	0.870
	RESCAN	35.672	0.967
	ReHEN	37.412	0.980
	SPANet	35.473	0.967
	UMRL	32.305	0.966
	本文	37.524	0.985
Rain800	GMM	20.462	0.730
	DSC	18.561	0.599
	RESCAN	24.091	0.841
	ReHEN	26.961	0.854
	SPANet	25.960	0.866
	UMRL	25.155	0.879
	本文	27.155	0.884

下载: 导出CSV

表 2 不同方法参数量与测试时间对比

Table 2. Comparison of parameters number and average running time

算法	参数量/10⁶	测试时间/s
GMM		423.511
DSC		146.761
RESCAN	0.298	0.443
ReHEN	0.150	0.522
SPANet	0.219	0.145
UMRL	0.984	0.162
本文	0.112	0.122

下载: 导出CSV

表 3 阶段数量对去雨效果的影响

Table 3. Effect of stage number on rain removal

阶段数量	PSNR	SSIM
1	26.156	0.756
2	27.256	0.812
3	28.116	0.866
4	28.596	0.878
5	28.702	0.895
6	29.362	0.898
7	29.362	0.899

下载: 导出CSV

参考文献(28)

[1]	LIU S, HUANG D, WANG Y. Receptive field block net for accurate and fast object detection[C]//European Conference on Computer Vision (ECCV). Berlin: Springer, 2018: 385-400.
[2]	SALEH F, ALIAKBARIAN M, SALZMANN M, et al. Effective use of synthetic data for urban scene semantic segmentation[C]//European Conference on Computer Vision (ECCV). Berlin: Springer, 2018: 86-103.
[3]	VO B N, VO B T, PHUNG D. Labeled random finite sets and the Bayes multi-target tracking filter[J]. IEEE Transactions on Signal Processing, 2014, 62(24): 6554-6567. doi: 10.1109/TSP.2014.2364014
[4]	LI Y, TAN R T, GUO X, et al. Rain streak removal using layer priors[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR). Piscataway: IEEE Press, 2016: 2736-2744.
[5]	KIM J, LEE C, SIM J, et al. Single-image deraining using an adaptive nonlocal means filter[C]//IEEE International Conference on Image Processing. Piscataway: IEEE Press, 2013: 914-917.
[6]	LUO Y, XU Y, JI H. Removing rain from a single image via discriminative sparse coding[C]//International Conference on Computer Vision. Piscataway: IEEE Press, 2015: 3397-3405.
[7]	WANG Y, LIU S, CHEN C, et al. A hierarchical approach for rain or snow removing in a single color image[J]. IEEE Transactions on Image Processing, 2017, 26(8): 3936-3950. doi: 10.1109/TIP.2017.2708502
[8]	YANG W, TAN R, FENG J, et al. Joint rain detection and removal from a single image with contextualized deep networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 42(6): 1377-1393. doi: 10.1109/TPAMI.2019.2895793
[9]	WANG T Y, YANG X, XU K, et al. Spatial attentive single-image deraining with a high quality real rain dataset[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR). Piscataway: IEEE Press, 2019: 12262-12271.
[10]	REN D, ZUO W, HU Q, et al. Progressive image deraining networks: A better and simpler baseline[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR). Piscataway: IEEE Press, 2019: 3932-3941.
[11]	ZHANG D, HE J, ZHAO Y. Learning to perform joint image super-resolution and rain removal via a single-convolutional neural network[J]. Journal of Electronic Imaging, 2018, 27(6): 063024.1-063024.11.
[12]	XIA H, ZHUGE R, LI H, et al. Single image rain removal via a simplified residual dense network[J]. IEEE Access, 2018, 6: 66522-66535. doi: 10.1109/ACCESS.2018.2879330
[13]	王聪. 图像去雨的深度模型[D]. 大连: 大连理工大学, 2020: 5-8. WANG C. Deep models for single image deraining[D]. Dalian: Dalian University of Technology, 2020: 5-8(in Chinese).
[14]	WANG C, XING X, WU Y, et al. DCSFN: Deep cross-scale fusion network for single image rain removal[C]//Proceedings of the 28th ACM International Conference on Multimedia. New York: ACM, 2020: 1643-1651.
[15]	WANG C, WU Y, SU Z, et al. Joint self-attention and scale-aggregation for self-calibrated deraining network[C]//Proceedings of the 28th ACM International Conference on Multimedia. New York: ACM, 2020: 2517-2525.
[16]	HUANG G, LIU Z, MAATEN L, et al. Densely connected con-volutional networks[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR). Piscataway: IEEE Press, 2017: 2261-2269.
[17]	CHAI G Q, WANG Z B, GUO G D, et al. Recursive modified dense network for single image deraining[J]. Journal of Electronic Imaging, 2020, 29(3): 33006.
[18]	LIU R, JIANG D, ZHANG L, et al. Deep depthwise separable convolutional network for change detection in optical aerial images[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2020, 13: 1109-1118. doi: 10.1109/JSTARS.2020.2974276
[19]	庞帆. 激光雷达惯导耦合的里程计与建图方法研究[D]. 北京: 北京建筑大学, 2020: 2-6. PANG F. Odometer and mapping method of lidar inertial navigation coupling[D]. Beijing: Beijing University of Civil Engineering and Architecture, 2020: 2-6(in Chinese).
[20]	ZHANG X, ZHOU X, LIN M, et al. ShuffleNet: An extremely efficient convolutional neural network for mobile devices[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR). Piscataway: IEEE Press, 2018: 6848-6856.
[21]	HUANG H, HE R, SUN Z, et al. Wavelet-SRNet: A wavelet-based CNN for multi-scale face super resolution[C]//International Conference on Computer Vision. Piscataway: IEEE Press, 2017: 1698-1706.
[22]	JOHNSON J, ALAHI A, LI F. Perceptual losses for real-time style transfer and super-resolution[C]//European Conference on Computer Vision(ECCV). Berlin: Springer, 2016: 694-711.
[23]	YANG C, MA C, YANG M. Single-image super-resolution: A benchmark[C]//European Conference on Computer Vision(ECCV). Berlin: Springer, 2014: 372-386.
[24]	ZHANG H, SINDAGI V, PATEL V. Image deraining using a conditional generative adversarial network[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2020, 30(11): 3943-3956. doi: 10.1109/TCSVT.2019.2920407
[25]	YANG W, TAN R, FENG J, et al. Deep joint rain detection and removal from a singled image[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR). Piscataway: IEEE Press, 2017: 1685-1694.
[26]	LI X, WU J, LIN Z, et al. Recurrent squeeze-and-excitation context aggregation net for single image deraining[C]//European Conference on Computer Vision(ECCV). Berlin: Springer, 2018: 262-277.
[27]	YANG Y, LU H. Single image deraining via recurrent hierarchy enhancement network[C]//Proceedings of the 27th ACM International Conference on Multimedia. New York: ACM, 2019: 1814-1822.
[28]	YASARLA R, PATEL V M. Uncertainty guided multi-scale residual learning-using a cycle spinning CNN for single image deraining[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR). Piscataway: IEEE Press, 2019: 8397-8406.