颜色一致性的无人机航拍低光照图像增强算法

王殿伟; 刘旺; 房杰; 许志杰

doi:10.13700/j.bh.1001-5965.2023.0172

颜色一致性的无人机航拍低光照图像增强算法

doi: 10.13700/j.bh.1001-5965.2023.0172

王殿伟^1, ,,
刘旺¹,
房杰¹,
许志杰²

1.
西安邮电大学通信与信息工程学院，西安 710121
2.
哈德斯菲尔德大学计算机与工程学院，哈德斯菲尔德HD1 3DH

基金项目: 国家自然科学基金青年基金(62201454)；陕西省国际科技合作计划(2023-GHYB-04)

详细信息

通讯作者:
E-mail：wangdianwei@xupt.edu.cn

中图分类号: TP391.41
计量
- 文章访问数: 429
- HTML全文浏览量: 108
- PDF下载量: 12
- 被引次数: 0
出版历程
- 收稿日期: 2023-04-10
- 录用日期: 2023-05-19
- 网络出版日期: 2023-06-27
- 整期出版日期: 2025-04-30

Low illumination image enhancement algorithm for UAV aerial photography with color consistency

1.
School of Communications and Information Engineering，Xi’an University of Posts & Telecommunications，Xi’an 710121，China
2.
School of Computing and Engineering，University of Huddersfield，Huddersfield HD1 3DH，UK

Funds: Youth Fund of the National Natural Science Foundation of China (62201454); Shaanxi Provincial International Science and Technology Cooperation Plan (2023-GHYB-04)

More Information

Corresponding author: E-mail：wangdianwei@xupt.edu.cn

摘要

摘要:
针对无人机(UAV)在低光照条件下拍摄的图像存在亮度低、视觉效果差等问题，建立了无人机航拍低光照数据集，提出了一种颜色一致性的无人机航拍低光照图像增强算法。在亮度增强阶段，构建亮度增强网络(BENet)来增强图像的亮度，利用颜色学习网络(CNet)模块和金字塔颜色嵌入(PCE)模块将图像的颜色特征和内容特征相结合，避免增强后的图像出现颜色失真；在图像校正阶段，构建基于域转移的校正网络，使用自建的数据集训练网络，借助光照良好图像来校正亮度增强阶段增强后的图像，减少噪声对图像的影响，得到增强后的图像。实验结果表明：所提算法可提升图像亮度，避免出现噪声放大和颜色失真的问题，在客观指标上，整体优于对比算法，同时还能提升目标检测算法在夜间的检测性能。
- 低光照图像增强 /
- 无人机航拍图像 /
- 亮度增强 /
- 颜色一致 /
- 图像去噪
Abstract:
To address the issue of low brightness and poor visual effect of unmanned aerial vehicles (UAVs) in low illumination conditions, this paper established a low illumination dataset of UAV aerial photography and proposed a low illumination image enhancement algorithm for UAV aerial photography with color consistency. Firstly, in the brightness enhancement stage, this paper constructed a brightness enhancement network (BENet) to enhance the brightness of images and used the color network (CNet) module and the pyramid color embedding (PCE) module to combine the color features and content features of the images, which avoided color distortion in the enhanced image. In the image correction stage, this paper constructed a correction network based on domain transmission, trained the network with the self-built dataset, corrected the enhanced image after the first stage with the help of well-illuminated images, reduced the effect of noise on the image, and finally obtained the enhanced image. The experimental results show that the algorithm effectively avoids the problems of color distortion and noise amplification while enhancing the image brightness, and it outperforms other advanced algorithms in terms of objective indicators and improves the detection performance of the target detection algorithm at night.
- low illumination image enhancement /
- unmanned aerial vehicle aerial photography image /
- brightness enhancement /
- color consistency /
- image denoising

HTML全文

图 1 本文网络结构

Figure 1. Structure of the proposed network

下载: 全尺寸图片幻灯片

图 2 PCE模块结构

Figure 2. Structure of PCE module

下载: 全尺寸图片幻灯片

图 3 数据集图像

Figure 3. Dataset images

下载: 全尺寸图片幻灯片

图 4 不同算法在建筑场景下的增强结果

Figure 4. Enhancement results of different algorithms in architectural scenarios

下载: 全尺寸图片幻灯片

图 5 不同算法在街道场景下的增强结果

Figure 5. Enhancement results of different algorithms in street scenarios

下载: 全尺寸图片幻灯片

图 6 不同算法在野外场景下的增强结果

Figure 6. Enhancement results of different algorithms in field scenarios

下载: 全尺寸图片幻灯片

图 7 不同算法在公开数据集场景下的增强结果

Figure 7. Enhancement results of different algorithms on public datasets

下载: 全尺寸图片幻灯片

图 8 颜色一致性消融实验对比

Figure 8. Comparison of color consistency ablation experiments

下载: 全尺寸图片幻灯片

图 9 图像校正去噪消融实验对比

Figure 9. Comparison of image correction ablation experiments

下载: 全尺寸图片幻灯片

图 10 检测算法结果

Figure 10. Detection algorithm results

下载: 全尺寸图片幻灯片

表 1 无人机航拍低光照数据集

Table 1. Low illumination dataset of UAV photography

场景	图像数量/张
建筑	4000
街道	4000
野外	2000

下载: 导出CSV

表 2 不同算法在自建数据集上的客观评价指标

Table 2. Objective evaluation indicators for different algorithms on self-built dataset

算法	LOE	BRISQUE	NIQE	处理时间/s
RetinexNet^[5]	702.80	31.47	3.663	0.0270
DRBN^[21]	462.32	28.69	3.250	0.0241
Zero-DCE++^[22]	433.86	15.37	3.357	0.0012
EnlightenGAN^[12]	408.01	17.63	3.247	0.0150
KinD++^[7]	609.83	23.05	3.813	0.0220
URetinex-Net^[23]	151.21	31.50	3.444	0.0281
SCI^[24]	222.67	37.73	3.568	0.0067
本文算法	213.78	19.45	3.205	0.0232

下载: 导出CSV

表 3 不同算法在公开数据集上的NIQE值

Table 3. NIQE of different methods on public datasets

算法	LIME	NPE	MEF	DICM
RetinexNet^[5]	4.907	4.080	4.904	4.674
DRBN^[21]	4.133	4.112	3.356	3.431
Zero-DCE++^[22]	3.762	4.253	3.283	3.362
EnlightenGAN^[12]	3.608	3.425	3.121	2.846
KinD++^[7]	4.206	3.551	3.374	2.965
URetinex-Net^[23]	4.096	4.041	3.318	3.235
SCI^[24]	4.138	4.160	3.433	3.709
本文算法	3.557	3.363	3.065	2.622

下载: 导出CSV

表 4 消融实验客观评价指标

Table 4. Objective evaluation indicators for ablation experiments

算法	PSNR/dB	SSIM
无颜色一致性	20.138	0.769
加入BM3D	19.893	0.778
本文算法	21.221	0.791

下载: 导出CSV

表 5 图像增强前后算法检测结果平均精度对比

Table 5. Comparison of AP values of algorithm detection results before and after image enhancement

算法	主干网络	Person		Bicycle		Car		Boat
算法	主干网络	增强前	增强后	增强前	增强后	增强前	增强后	增强前	增强后
SSD300	VGG-16	0.355	0.361	0.344	0.343	0.351	0.346	0.364	0.375
Deformable DETR	ResNet-50	0.382	0.393	0.365	0.375	0.358	0.360	0.381	0.399

下载: 导出CSV

参考文献(31)

[1]	李可夫, 钟汇才, 高兴宇, 等. 显著性引导的低光照人脸检测[J]. 北京航空航天大学学报, 2021, 47(3): 572-584. LI K F, ZHONG H C, GAO X Y, et al. Saliency guided low-light face detection[J]. Journal of Beijing University of Aeronautics and Astronautics, 2021, 47(3): 572-584(in Chinese).
[2]	PIZER S M, AMBURN E P, AUSTIN J D, et al. Adaptive histogram equalization and its variations[J]. Computer Vision, Graphics, and Image Processing, 1987, 39(3): 355-368. doi: 10.1016/S0734-189X(87)80186-X
[3]	LAND E H. The Retinex theory of color vision[J]. Scientific American, 1977, 237(6): 108-128. doi: 10.1038/scientificamerican1277-108
[4]	GUO X J, LI Y, LING H B. LIME: low-light image enhancement via illumination map estimation[J]. IEEE Transactions on Image Processing, 2017, 26(2): 982-993.
[5]	WEI C, WANG W J, YANG W H, et al. Deep Retinex decomposition for low-light enhancement[EB/OL]. (2018-08-14)[2023-04-01]. https://arxiv.org/abs/1808.04560v1.
[6]	ZHANG Y H, ZHANG J W, GUO X J. Kindling the darkness: a practical low-light image enhancer[C]//Proceedings of the 27th ACM International Conference on Multimedia. New York: ACM, 2019: 1632-1640.
[7]	ZHANG Y H, GUO X J, MA J Y, et al. Beyond brightening low-light images[J]. International Journal of Computer Vision, 2021, 129(4): 1013-1037. doi: 10.1007/s11263-020-01407-x
[8]	LV F F, LU F, WU J H, et al. MBLLEN: low-light image/video enhancement using CNNs[C]//Proceedings of the British Machine Vision Conference. [S. l. ]: BMVC, 2018.
[9]	孙帮勇, 赵兴运, 吴思远, 等. 基于移位窗口多头自注意力U型网络的低照度图像增强方法[J]. 电子与信息学报, 2022, 44(10): 3399-3408. SUN B Y, ZHAO X Y, WU S Y, et al. Low-light image enhancement method based on shifted window multi-head self-attention U-shaped network[J]. Journal of Electronics & Information Technology, 2022, 44(10): 3399-3408(in Chinese).
[10]	XU K, YANG X, YIN B C, et al. Learning to restore low-light images via decomposition-and-enhancement[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2020: 2278-2287.
[11]	张雅荔, 李文元, 李昌禄, 等. 融合注意力引导的多尺度低照度图像增强方法[J]. 西安电子科技大学学报, 2023, 50(1): 129-136. ZHANG Y L, LI W Y, LI C L, et al. Method for enhancement of the multi-scale low-light image by combining an attention guidance[J]. Journal of Xidian University, 2023, 50(1): 129-136(in Chinese).
[12]	JIANG Y F, GONG X Y, LIU D, et al. EnlightenGAN: deep light enhancement without paired supervision[J]. IEEE Transactions on Image Processing, 2021, 30: 2340-2349. doi: 10.1109/TIP.2021.3051462
[13]	向森, 王应锋, 邓慧萍, 等. 基于双重迭代的零样本低照度图像增强[J]. 电子与信息学报, 2022, 44(10): 3379-3388. XIANG S, WANG Y F, DENG H P, et al. Zero-shot learning for low-light image enhancement based on dual iteration[J]. Journal of Electronics & Information Technology, 2022, 44(10): 3379-3388(in Chinese).
[14]	王殿伟, 刘旺, 房杰, 等. 受生物视觉启发的无人机航拍低照度图像增强算法[J]. 西北工业大学学报, 2023, 41(1): 144-152. doi: 10.1051/jnwpu/20234110144 WANG D W, LIU W, FANG J, et al. Enhancement algorithm of low illumination image for UAV images inspired by biological vision[J]. Journal of Northwestern Polytechnical University, 2023, 41(1): 144-152(in Chinese). doi: 10.1051/jnwpu/20234110144
[15]	JIN Y Y, YANG W H, TAN R T. Unsupervised night image enhancement: when layer decomposition meets light-effects suppression[EB/OL]. (2022-09-17)[2023-04-01]. https://arxiv.org/abs/2207.10564v2.
[16]	RONNEBERGER O, FISCHER P, BROX T. U-Net: convolutional networks for biomedical image segmentation[C]//Proceedings of the Medical Image Computing and Computer-Assisted Intervention. Berlin: Springer, 2015: 234-241.
[17]	ZHANG Z, ZHENG H, HONG R C, et al. Deep color consistent network for low-light image enhancement[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2022: 1889-1898.
[18]	ZHANG F, SHAO Y J, SUN Y S, et al. Unsupervised low-light image enhancement via histogram equalization prior[EB/OL]. (2021-11-03)[2023-04-01]. https://arxiv.org/abs/2112.01766v1.
[19]	DU W C, CHEN H, YANG H Y. Learning invariant representation for unsupervised image restoration[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2020: 14471-14480.
[20]	WANG W J, WEI C, YANG W H, et al. GLADNet: low-light enhancement network with global awareness[C]//Proceedings of the 13th IEEE International Conference on Automatic Face & Gesture Recognition. Piscataway: IEEE Press, 2018: 751-755.
[21]	YANG W H, WANG S Q, FANG Y M, et al. From fidelity to perceptual quality: a semi-supervised approach for low-light image enhancement[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2020: 3060-3069.
[22]	LI C Y, GUO C L, LOY C C. Learning to enhance low-light image via zero-reference deep curve estimation[EB/OL]. (2021-03-01)[2023-04-01]. https://arxiv.org/abs/2103.00860v1.
[23]	WU W H, WENG J, ZHANG P P, et al. URetinex-Net: Retinex-based deep unfolding network for low-light image enhancement[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2022: 5891-5900.
[24]	MA L, MA T Y, LIU R S, et al. Toward fast, flexible, and robust low-light image enhancement[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2022: 5627-5636.
[25]	AHN H, KEUM B, KIM D, et al. Adaptive local tone mapping based on Retinex for high dynamic range images[C]//Proceedings of the IEEE International Conference on Consumer Electronics. Piscataway: IEEE Press, 2013: 153-156.
[26]	MITTAL A, MOORTHY A K, BOVIK A C. No-reference image quality assessment in the spatial domain[J]. IEEE Transactions on Image Processing, 2012, 21(12): 4695-4708. doi: 10.1109/TIP.2012.2214050
[27]	MITTAL A, SOUNDARARAJAN R, BOVIK A C. Making a “completely blind” image quality analyzer[J]. IEEE Signal Processing Letters, 2012, 20(3): 209-212.
[28]	DABOV K, FOI A, KATKOVNIK V, et al. Image denoising by sparse 3-D transform-domain collaborative filtering[J]. IEEE Transactions on Image Processing, 2007, 16(8): 2080-2095. doi: 10.1109/TIP.2007.901238
[29]	LIU W, ANGUELOV D, ERHAN D, et al. SSD: single shot multibox detector[C]//Proceedings of the Computer Vision. Berlin: Springer, 2016: 21-37.
[30]	ZHU X Z, SU W J, LU L W, et al. Deformable DETR: deformable Transformers for end-to-end object detection[EB/OL]. (2021-03-18)[2023-04-01]. https://arxiv.org/abs/2010.04159v4.
[31]	LOH Y P, CHAN C S. Getting to know low-light images with the exclusively dark dataset[J]. Computer Vision and Image Understanding, 2019, 178: 30-42. doi: 10.1016/j.cviu.2018.10.010