Improved transfer learning based dual-branch convolutional neural network image dehazing
-
摘要:
针对现有图像去雾算法存在去雾不彻底和图像颜色失真的问题，提出一种迁移学习子网络和残差注意力子网络相结合的图像去雾模型。采用迁移学习子网络的预训练模型增强样本的特征属性；构建双分支网络结构，并利用残差注意力子网络辅助迁移学习子网络训练网络模型的参数；利用尾部集成学习的方法融合双网络的特征，得到去雾图像的模型参数，完成图像恢复任务。实验结果表明：所提算法在RESIDE数据集和O-HAZE数据集上PSNR指标比GCANet分别提高了1.87 dB和4.22 dB，在O-HAZE数据集上SSIM指标比GCANet提高了6.7%。
Abstract:To address the problems of incomplete dehazing and image color distortion in the existing image dehazing algorithms, a dehazing network combining transfer learning sub-net and residual attention sub-net is proposed. First, the pre-trained model of the transfer learning subnet is adopted to enhance the feature attributes of the samples. Second, the structure of the dual-branch network is constructed, and the residual attention sub-network is used to assist the transfer learning sub-network to train the parameters of the network model. Finally, the method of tail ensemble learning is used to fuse the features of the dual network to obtain the model parameters of the dehazed image, so as to complete the image restoration task.The experimental results show that the algorithm proposed in the paper improves the PSNR index by 1.87 dB and 4.22 dB on the RESIDE dataset and the O-HAZE dataset respectively compared to GCANet, and the SSIM index on the O-HAZE dataset by 6.7% compared to GCANet.
-
图 5 不同去雾算法在SOTS数据集上的去雾效果
Figure 5. Dehazing effect of different dehazing algorithms on SOTS dataset
图 6 不同去雾算法在O-HAZE数据集上的去雾效果
Figure 6. Dehazing effect of different dehazing algorithms on O-HAZE dataset
表 1 不同去雾算法在SOTS数据集上的SSIM、PSNR值
Table 1. SSIM and PSNR values of SOTS dataset under different dehazing algorithms
算法 PSNR/dB SSIM DCP 16.62 0.817 CAP 19.05 0.840 DehazeNet 20.64 0.800 AODNet 19.06 0.850 GCANet 30.23 0.975 本文算法 32.10 0.94 
下载: 导出CSV
表 2 不同去雾算法在O-HAZE数据集上的SSIM、PSNR值
Table 2. SSIM and PSNR values of O-HAZE dataset under different dehazing algorithms
算法 PSNR/dB SSIM DCP 12.92 0.505 CAP 14.55 0.567 DehazeNet 16.27 0.604 AODNet 17.69 0.616 GCANet 18.41 0.680 本文算法 22.63 0.747
下载: 导出CSV
表 3 不同去雾算法对性能的影响
Table 3. Effects of different dehazing algorithms on performance
算法 PSNR/dB SSIM TLN（NP） 19.87 0.661 TLN 20.56 0.706 RAN 17.34 0.589 TLN（NP）+RAN 20.92 0.716 本文 21.94 0.736
下载: 导出CSV
表 4 不同模型对性能的影响
Table 4. Effects of different models on performance
模型 PSNR/dB SSIM Model 1 21.36 0.612 Model 2 21.50 0.700 Model 3 21.52 0.650 Model 4 22.63 0.747
下载: 导出CSV
表 5 不同加权系数对性能的影响
Table 5. Effect of different weighting coefficients on performance
方法 $ {\lambda _{\rm{s}}} $ $ {\lambda _{\rm{p}}} $ $ {\lambda _{\rm{m}}} $ 损失函数 PRNR/dB SSIM Method 1 1 0 0 $L = {L_{\rm{s}}}$ 20.90 0.607 Method 2 1 0.05 0 $L = {L_{\rm{s}}} + 0.05 {L_{\rm{p}}}$ 21.06 0.703 Method 3 1 0.01 0 $L = {L_{\rm{s}}} + 0.01 {L_{\rm{p}}}$ 21.33 0.717 Method 4 1 0 0.5 $L = {L_{\rm{s}}} + 0.5 {L_{\rm{m}}}$ 20.81 0.713 Method 5 1 0 0.8 $L = {L_{\rm{s}}} + 0.8 {L_{\rm{m}}}$ 20.75 0.727 Method 6 1 0.05 0.5 $L = {L_{\rm{s}}} + 0.05 {L_{\rm{p}}} + 0.5 {L_{\rm{m}}}$ 21.35 0.733 Method 7 1 0.05 0.8 $L = {L_{\rm{s}}} + 0.05 {L_{\rm{p}}} + 0.8 {L_{\rm{m}}}$ 21.20 0.730 Method 8 1 0.01 0.5 $L = {L_{\rm{s} } } + 0.01 {L_{\rm{p} } } + 0.5 {L_{\rm{m}}}$ 22.63 0.747 Method 9 1 0.01 0.8 $L = {L_{\rm{s}}} + 0.01 {L_{\rm{p}}} + 0.8 {L_{\rm{m}}}$ 22.38 0.730
下载: 导出CSV
-
[1] 禹晶, 徐东彬, 廖庆敏. 图像去雾技术研究进展[J]. 中国图象图形学报, 2011, 16(9): 1561-1576.YU J, XU D B, LIAO Q M. Imagedefogging: A survey[J]. Journal of Image and Graphics, 2011, 16(9): 1561-1576(in Chinese). [2] HE K, SUN J, TANG X. Single image haze removal using dark channel prior[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2010, 33(12): 2341-2353. [3] ZHU Q, MAI J, SHAO L. A fast single image haze removal algorithm using color attenuation prior[J]. IEEE Transactions on Image Processing, 2015, 24(11): 3522-3533. doi: 10.1109/TIP.2015.2446191 [4] TAN R T. Visibility in bad weather from a single image[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2008: 1-8. [5] 杨燕, 张金龙, 张浩文. 基于区间估计与透射率自适应约束的去雾算法[J]. 北京航空航天大学学报, 2022, 48(1): 15-26. doi: 10.13700/j.bh.1001-5965.2020.0547YANG Y, ZHANG J L, ZHANG H W. Dehazing algorithm based on interval estimation and adaptive constraints of transmittance[J]. Journal of Beijing University of Aeronautics and Astronautics, 2022, 48(1): 15-26(in Chinese). doi: 10.13700/j.bh.1001-5965.2020.0547 [6] CAI B, XU X, JIA K, et al. DehazeNet: An end-to-end system for single image haze removal[J]. IEEE Transactions on Image Processing, 2016, 25(11): 5187-5198. doi: 10.1109/TIP.2016.2598681 [7] REN W, LIU S, ZHANG H, et al. Single image dehazing via multi-scale convolutional neural networks[C]//Proceedings of the European conference on Computer Vision. Berlin: Springer, 2016: 154-169. [8] LI B, PENG X, WANG Z, et al. AOD-Net: All-in-one dehazing network[C]//Proceedings of the IEEE International Conference on Computer Vision. Piscataway: IEEE Press, 2017: 4770-4778. [9] LIU X, MA Y, SHI Z, et al. GridDehazeNet: Attention-based multi-scale network for image dehazing[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. Piscataway: IEEE Press, 2019: 7314-7323. [10] CHEN D, HE M, FAN Q, et al. Gated context aggregation network for image dehazing and deraining[C]//Proceedings of the IEEE Winter Conference on Applications of Computer Vision. Piscataway: IEEE Press, 2019: 1375-1383. [11] 王殿伟, 李顺利, 韩鹏飞. 基于特征约束CycleGAN的单幅图像去雾算法研究[J]. 激光与光电子学进展, 2021, 58(14): 272-278.WANG D W, LI S L, HANG P F. Feature constraint CycleGAN for single image dehazing algorithm[J]. Laser & Optoelectronics Progress, 2021, 58(14): 272-278(in Chinese). [12] HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2016: 770-778. [13] NIU Z, ZHONG G, YU H. A review on the attention mechanism of deep learning[J]. Neurocomputing, 2021, 452: 48-62. doi: 10.1016/j.neucom.2021.03.091 [14] ZHU J Y, PARK T, ISOLA P, et al. Unpaired image-to-image translation using cycle-consistent adversarial networks[C]//Proceedings of the IEEE International Conference on Computer Vision. Piscataway: IEEE Press, 2017: 2223-2232. [15] LI B, REN W, FU D, et al. Benchmarking single-image dehazing and beyond[J]. IEEE Transactions on Image Processing, 2019, 28(1): 492-505. [16] ANCUTI C O, ANCUTI C, TIMOFTE R, et al. O-HAZE: A dehazing benchmark with real hazy and haze-free outdoor images[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . Piscataway: IEEE Press, 2018: 754-762. -
下载:
点击查看大图
计量
- 文章访问数: 1070
- HTML全文浏览量: 97
- PDF下载量: 45
- 被引次数: 0
