| Citation: | YANG Y C，LI X M，DANG J W，et al. Infrared and visible image fusion based on GEMD and improved PCNN[J]. Journal of Beijing University of Aeronautics and Astronautics，2023，49（9）：2317-2329 （in Chinese） doi: 10.13700/j.bh.1001-5965.2022.0756
|
Because of the limitations of decomposition tools in traditional image fusion methods, artifacts, decreased brightness, and contrast appear on fused image edges. A gradient edge-preserving multi-level decomposition(GEMD)-based approach for infrared and visible image fusion, as well as an enhanced pulse-coupled neural network(PCNN), are described. The gradient bilateral filter(GBF) is proposed based on the bilateral filter and the gradient filter(GF), which can preserve the edge structure, brightness, and contrast information while smoothing the detail information. Firstly, the source images are divided into three layers of feature maps and a base layer, and then a multi-level decomposition model is built using a gradient bilateral filter and gradient filter. Each layer of feature maps has two different structures, thin and thick. Then, according to the characteristics of the information contained in each feature map, the PCNN, which introduces an improved Laplacian operator in the input stimulus to enhance weak details captured in the image, regional energy, and contrast saliency are respectively adopted to obtain the fusion images of each sub-feature and the fusion image of the base layer as the fusion rules. Finally, the sub-fusion images are superimposed to obtain the final fusion image. Through experimental verification, the proposed method has improved both visual effect and quantitative evaluation and improved the brightness and contrast information of infrared and visible fusion images.
| [1] |
沈瑜, 陈小朋. 基于DLatLRR与VGG Net的红外与可见光图像融合[J]. 北京航空航天大学学报, 2021, 47(6): 1105-1114.
SHEN Y, CHEN X P. Infrared and visible image fusion based on latent low-rank representation decomposition and VGG Net[J]. Journal of Beijing University of Aeronautics and Astronautics, 2021, 47(6): 1105-1114(in Chinese).
|
| [2] |
LI G F, LIN Y J, QU X D. An infrared and visible image fusion method based on multi-scale transformation and norm optimization[J]. Information Fusion, 2021, 71: 109-129. doi: 10.1016/j.inffus.2021.02.008
|
| [3] |
朱雯青, 汤心溢, 张瑞, 等. 基于边缘保持和注意力生成对抗网络的红外与可见光图像融合[J]. 红外与毫米波学报, 2021, 40(5): 696-708.
ZHU W Q, TANG X Y, ZHANG R, et al. Infrared and visible image fusion based on edge-preserving and attention generative adversarial network[J]. Journal of Infrared and Millimeter Waves, 2021, 40(5): 696-708(in Chinese).
|
| [4] |
刘佳妮, 金伟其, 李力, 等. 自适应参考图像的可见光与热红外彩色图像融合算法[J]. 光谱学与光谱分析, 2016, 36(12): 3907-3914.
LIU J N, JIN W Q, LI L, et al. Visible and infrared thermal image fusion algorithm based on self-adaptive reference image[J]. Spectroscopy and Spectral Analysis, 2016, 36(12): 3907-3914(in Chinese).
|
| [5] |
LUO Y Y, HE K J, XU D, et al. Infrared and visible image fusion based on visibility enhancement and hybrid multiscale decomposition[J]. Optik, 2022, 258: 168914. doi: 10.1016/j.ijleo.2022.168914
|
| [6] |
GUO Z Y, YU X T, DU Q L. Infrared and visible image fusion based on saliency and fast guided filtering[J]. Infrared Physics & Technology, 2022, 123: 104178.
|
| [7] |
MA J Y, ZHOU Y. Infrared and visible image fusion via gradientlet filter[J]. Computer Vision and Image Understanding, 2020, 197-198: 103016. doi: 10.1016/j.cviu.2020.103016
|
| [8] |
王满利, 王晓龙, 张长森. 基于动态范围压缩增强和NSST的红外与可见光图像融合算法[J]. 光子学报, 2022, 51(9): 277-291.
WANG M L, WANG X L, ZHANG C S. Infrared and visible image fusion algorithm based on dynamic range compression enhancement and NSST[J]. Acta Photonica Sinica, 2022, 51(9): 277-291(in Chinese).
|
| [9] |
ZHOU Z Q, WANG B, LI S, et al. Perceptual fusion of infrared and visible images through a hybrid multi-scale decomposition with Gaussian and bilateral filters[J]. Information Fusion, 2016, 30: 15-26. doi: 10.1016/j.inffus.2015.11.003
|
| [10] |
LI X S, ZHOU F Q, TAN H S, et al. Multimodal medical image fusion based on joint bilateral filter and local gradient energy[J]. Information Sciences, 2021, 569: 302-325. doi: 10.1016/j.ins.2021.04.052
|
| [11] |
HU J W, LI S T. The multiscale directional bilateral filter and its application to multisensor image fusion[J]. Information Fusion, 2012, 13(3): 196-206. doi: 10.1016/j.inffus.2011.01.002
|
| [12] |
周志强, 汪渤, 李立广, 等. 基于双边与高斯滤波混合分解的图像融合方法[J]. 系统工程与电子技术, 2016, 38(1): 8-13.
ZHOU Z Q, WANG B, LI L G, et al. Image fusion based on a hybrid decomposition via bilateral and Gaussian filters[J]. Systems Engineering and Electronics, 2016, 38(1): 8-13(in Chinese).
|
| [13] |
刘峰, 沈同圣, 马新星. 交叉双边滤波和视觉权重信息的图像融合[J]. 仪器仪表学报, 2017, 38(4): 1005-1013. doi: 10.3969/j.issn.0254-3087.2017.04.027
LIU F, SHEN T S, MA X X. Image fusion via cross bilateral filter and visual weight information[J]. Chinese Journal of Scientific Instrument, 2017, 38(4): 1005-1013(in Chinese). doi: 10.3969/j.issn.0254-3087.2017.04.027
|
| [14] |
戴进墩, 刘亚东, 毛先胤, 等. 基于FDST和双通道PCNN的红外与可见光图像融合[J]. 红外与激光工程, 2019, 48(2): 67-74.
DAI J D, LIU Y D, MAO X Y, et al. Infrared and visible image fusion based on FDST and dual-channel PCNN[J]. Infrared and Laser Engineering, 2019, 48(2): 67-74(in Chinese).
|
| [15] |
MA J L, ZHOU Z Q, WANG B, et al. Infrared and visible image fusion based on visual saliency map and weighted least square optimization[J]. Infrared Physics & Technology, 2017, 82: 8-17.
|
| [16] |
LIU Y, CHEN X, CHENG J A, et al. Infrared and visible image fusion with convolutional neural networks[J]. International Journal of Wavelets, Multiresolution and Information Processing, 2018, 16(3): 1850018. doi: 10.1142/S0219691318500182
|
| [17] |
LIU Y, CHEN X, WARD R K, et al. Medical image fusion via convolutional sparsity based morphological component analysis[J]. IEEE Signal Processing Letters, 2019, 26(3): 485-489. doi: 10.1109/LSP.2019.2895749
|
| [18] |
ZHANG Y, ZHANG L J, BAI X Z, et al. Infrared and visual image fusion through infrared feature extraction and visual information preservation[J]. Infrared Physics & Technology, 2017, 83: 227-237.
|
| [19] |
TANG L F, YUAN J T, ZHANG H, et al. PIAFusion: A progressive infrared and visible image fusion network based on illumination aware[J]. Information Fusion, 2022, 83-84: 79-92. doi: 10.1016/j.inffus.2022.03.007
|
| [20] |
张小利, 李雄飞, 李军. 融合图像质量评价指标的相关性分析及性能评估[J]. 自动化学报, 2014, 40(2): 306-315.
ZHANG X L, LI X F, LI J. Validation and correlation analysis of metrics for evaluating performance of image fusion[J]. Acta Automatica Sinica, 2014, 40(2): 306-315(in Chinese).
|
| [21] |
王晓文, 赵宗贵, 汤磊. 一种新的红外与可见光图像融合评价方法[J]. 系统工程与电子技术, 2012, 34(5): 871-875.
WANG X W, ZHAO Z G, TANG L. Novel quality metric for infrared and visible image fusion[J]. Systems Engineering and Electronics, 2012, 34(5): 871-875(in Chinese).
|
Figures(11) / Tables(1)
Copyright © Journal of Beijing University of Aeronautics and Astronautics
Address: Editorial Department of Journal of Beijing University of Aeronautics and Astronautics, 37 Xueyuan Road, Haidian District, Beijing Post Code: 100191 Email: jbuaa@buaa.edu.cn
Supported by:
Beijing Renhe Information Technology Co., Ltd.
DownLoad:
