北京航空航天大学学报 ›› 2021, Vol. 47 ›› Issue (6): 1105-1114.doi: 10.13700/j.bh.1001-5965.2020.0178

• 论文 • 上一篇    下一篇

基于DLatLRR与VGG Net的红外与可见光图像融合

沈瑜, 陈小朋   

  1. 兰州交通大学 电子与信息工程学院, 兰州 730070
  • 收稿日期:2020-05-08 发布日期:2021-07-06
  • 通讯作者: 沈瑜 E-mail:18609311366@163.com
  • 基金资助:
    国家自然科学基金(61861025,61562057,61761027,51669010);长江学者和创新团队发展计划(IRT_16R36);光电技术与智能控制教育部重点实验室(兰州交通大学)开放课题(KFKT2018-9);兰州市人才创新创业项目(2018-RC-117);甘肃省教育厅高等学校科研项目(216130);兰州交通大学青年基金(2015005)

Infrared and visible image fusion based on latent low-rank representation decomposition and VGG Net

SHEN Yu, CHEN Xiaopeng   

  1. School of Electronic and Information Engineering, Lanzhou Jiaotong University, Lanzhou 730070, China
  • Received:2020-05-08 Published:2021-07-06
  • Supported by:
    National Natural Science Foundation of China (61861025,61562057,61761027,51669010); Program for Changjiang Scholars and Innovative Research Team in University (IRT_16R36); Key Laboratory of Opt-electronic Technology and Intelligent Control of Ministry of Education (Lanzhou Jiaotong University) Open Object (KFKT2018-9); Lanzhou Talent Innovation and Entrepreneurship Project (2018-RC-117); Research Project of Institutions of Higher Learning of Education Department of Gansu Province (216130); Lanzhou Jiaotong University Youth Fund (2015005)

摘要: 针对红外与可见光图像融合中特征损失严重、显著目标不突出的问题,提出了一种低秩表示分解与深度神经网络相结合的图像融合算法。首先,对源图像进行潜在低秩表示分解(DLatLRR),得到相应的低秩部分、显著部分及稀疏噪声。然后,分别采用16层的VGG Net模型和联合特征加权算法对低秩部分与显著部分进行融合,舍弃二者的稀疏噪声。最后,对融合得到的低秩部分和显著部分进行图像重建,得到最终的融合图像。实验结果表明:与其他算法进行比较,所提算法能够对图像的深层次细节特征进行融合,突出场景中的感兴趣区域,且融合图像的相关差异和、结构相似性、线性相关度等多种客观指标均有所提升,提升最大值分别为0.73、0.15、0.11,噪声产生率的最大缩减值为0.041 2。

关键词: 图像处理, 图像融合, 潜在低秩表示分解(DLatLRR), VGG Net, 联合特征加权

Abstract: An image fusion algorithm combining low-rank representation decomposition and deep neural network is proposed to solve the problem of serious feature loss and non-prominent target in infrared and visible image fusion. First, Latent Low-Rank Representation Decomposition (DLatLRR) was performed on the source image to obtain the corresponding low-rank part, saliency part and sparse noise. Then, the VGG Net model and the joint feature weighting algorithm were used to fuse the low-rank part and the saliency part respectively, and the sparse noise of two parts were discarded. Finally, image reconstruction was carried out on the low-rank part and saliency part of the fusion to obtain the final fusion image. Compared with other methods, the experimental results show that the algorithm can fuse the deep details of the image and highlight the "interested" area in the scene. The objective indexes of the fused image including the sum of the correlations of differences, structure similarity index measure, correlation coefficient all improve, with the maximum values of 0.73, 0.15 and 0.11 respectively, and the maximum reduction value of noise generation rate is 0.041 2.

Key words: image processing, image fusion, Latent Low-Rank Representation Decomposition (DLatLRR), VGG Net, joint feature weighting

中图分类号: 


版权所有 © 《北京航空航天大学学报》编辑部
通讯地址:北京市海淀区学院路37号 北京航空航天大学学报编辑部 邮编:100191 E-mail:jbuaa@buaa.edu.cn
本系统由北京玛格泰克科技发展有限公司设计开发