基于收缩场学习的Retinex低照度图像增强

吴庆波; 王蕊; 任文琦

doi:10.13700/j.bh.1001-5965.2020.0064

基于收缩场学习的Retinex低照度图像增强

doi: 10.13700/j.bh.1001-5965.2020.0064

吴庆波^{1, 2},
王蕊^{1, 2},
任文琦^{1, 2, ,}

1.
中国科学院信息工程研究所信息安全国家重点实验室, 北京 100193
2.
中国科学院大学网络空间安全学院, 北京 100049

基金项目:

国家自然科学基金 U1605252

国家自然科学基金 U1803264

国家自然科学基金 61802403

国家重点研发计划 2019YFB1406500

北京市自然科学基金 L182057

北京市自然科学基金 KZ201910005007

北京市自然科学基金 L182057

详细信息

作者简介:
吴庆波  男, 博士研究生。主要研究方向:图像处理

王蕊  女, 博士, 研究员, 博士生导师。主要研究方向:多媒体信息智能化处理、计算机视觉等

任文琦  男, 博士, 副研究员。主要研究方向:图像处理和机器学习

通讯作者:
任文琦, E-mail: renwenqi@iie.ac.cn

中图分类号: TP391
计量
- 文章访问数: 669
- HTML全文浏览量: 132
- PDF下载量: 127
- 被引次数: 0
出版历程
- 收稿日期: 2020-03-02
- 录用日期: 2020-03-20
- 网络出版日期: 2020-09-20

Learning shrinkage fields for low-light image enhancement via Retinex

WU Qingbo^{1, 2},
WANG Rui^{1, 2},
REN Wenqi^{1, 2
, ,}

1.
State Key Laboratory of Information Security, Institute of Information Engineering, CAS, Beijing 100193, China
2.
School of Cyber Security, University of Chinese Academy of Sciences, Beijing 100049, China

Funds:

National Natural Science Foundation of China U1605252

National Natural Science Foundation of China U1803264

National Natural Science Foundation of China 61802403

National Key R & D Program of China 2019YFB1406500

Beijing Natural Science Foundation L182057

Beijing Natural Science Foundation KZ201910005007

Beijing Natural Science Foundation L182057

More Information

Corresponding author: REN Wenqi, E-mail: renwenqi@iie.ac.cn

摘要

摘要:
为增强低照度图像和抑制噪声，提出了一种通过学习收缩场（SF）改进Retinex分解的图像增强方法。首先，构造新的目标函数，在正则项中引入2组不同的高阶滤波器分别约束未知的反射图和照明图。高阶滤波器可以学习到多种激活模式，有利于在恢复反射图的同时抑制噪声污染。然后，在优化目标函数时通过求解收缩场更新隐变量，参数化的压缩函数可以自适应地调整相应滤波器在反射图和照明图上的响应。最后，在每个级联内更新照明图之前，嵌入一个辅助的收缩场，以抑制噪声和不良伪影的传播，从而更精确地估计照明图。实验结果表明，所提方法取得的峰值信噪比（PSNR）和结构相似性（SSIM）均高于当前最新的低照度图像增强方法。
- 低照度图像 /
- Retinex模型 /
- 图像增强 /
- 图像去噪 /
- 收缩场(SF)模型
Abstract:
To enhance a low-light image and mitigate noises simultaneously, we propose an image enhancement algorithm by learning Shrinkage Field (SF) to improve the reflectance image and the illumination map in Retinex model. To this end, first, we design a novel objective function by constraining the latent reflectance image and the illumination map with two different groups of high-order filters in regularization terms. These filters can be learnt to possess various activation patterns and thus facilitate recovering the reflectance image and suppressing noises at the same time. Then, we update the latent variables in the objective function optimization by calculating SFs, where the parameterized shrinkage functions are capable to scale the responses of the corresponding high-order filters convolving the reflectance image and the illumination map. Finally, we embed an auxiliary SF model before the update of the illumination map in each cascade to suppress the propagation of noises and undesirable artifacts and further to refine the estimation of illumination map. Experimental results demonstrate that the proposed algorithm outperforms the state-of-the-art low-light image enhancement methods in terms of Peak Signal to Noise Ratio(PSNR) and Structural Similarity (SSIM).
- low-light image /
- Retinex model /
- image enhancement /
- image denoising /
- model of Shrinkage Field(SF)

HTML全文

图 1 低照度图像增强方法流程

Figure 1. Flowchart of low-light image enhancement method

下载: 全尺寸图片幻灯片

图 2 训练数据示例

Figure 2. Examples from training data

下载: 全尺寸图片幻灯片

图 3 不同方法在合成数据上的比较

Figure 3. Comparison of different methods on synthetic data

下载: 全尺寸图片幻灯片

图 4 不同方法在真实数据上的比较

Figure 4. Comparison of different methods on real-world data

下载: 全尺寸图片幻灯片

图 5 第i=1级联中学习到的收缩场Ψⁱ、和的滤波器和相应的压缩函数

Figure 5. Filters and corresponding shrinkage functions in SF models Ψⁱ, and at cascade i=1

下载: 全尺寸图片幻灯片

图 6 辅助反射图的有效性

Figure 6. Effectiveness of auxiliary reflectance image

下载: 全尺寸图片幻灯片

图 7 本文方法随迭代次数增加的增强结果

Figure 7. Enhancement results of proposed method with increasing iteration times

下载: 全尺寸图片幻灯片

图 8 高阶滤波器个数对实验结果的影响

Figure 8. Influence of high-order filter number on experimental results

下载: 全尺寸图片幻灯片

表 1 不同方法在含有多种噪声的测试集上增强结果的平均峰值信噪比和结构相似性

Table 1. Average PSNRs and SSIMs of results enhanced by different methods on test dataset with various noises

方法	mPSNR/dB			mSSIM
方法	σ²=0%	σ²=3%	σ²=5%	σ²=0%	σ²=3%	σ²=5%
LIME^[7]	19.33	18.62	17.89	0.852 6	0.824 9	0.791 3
DHN^[20]	19.87	19.07	18.03	0.892 2	0.855 6	0.816 8
SR^[9]	20.88	19.46	18.21	0.903 5	0.862 4	0.830 4
本文方法	21.73	20.55	19.24	0.911 6	0.871 3	0.852 0

下载: 导出CSV

表 2 在本文测试集上评估辅助反射图的有效性

Table 2. Effectiveness evaluation of auxiliary reflectance image on our test dataset

方法	mPSNR/dB	mSSIM
本文方法(不含 )	16.38	0.760 4
本文方法(包含 )	19.24	0.852 0

下载: 导出CSV

参考文献(31)

[1]	WANG R, LIANG D, ZHANG W, et al.MatchDR: Image correspondence by leveraging distance ratio constraint[C]//ACM International Conference on Multimedia.New York: ACM, 2016: 606-610.
[2]	GUO Y F, CAO X C, ZHANG W, et al.Fake colorized image detection[J].IEEE Transactions on Information Forensics and Security, 2018, 13(8):1932-1944. doi: 10.1109/TIFS.2018.2806926
[3]	GUO Y F, AU O C, WANG R, et al.Half image watermarking by content aware double-sided embedding error diffusion[J].IEEE Transaction on Image Processing, 2018, 27(7):3387-3402. doi: 10.1109/TIP.2018.2815181
[4]	PIZER S M, JOHNSTON R E, ERICKSEN J P, et al.Contrast-limited adaptive histogram equalization: Speed and effectiveness[C]//Proceedings of 1st Conference on Visualization in Biomedical Computing.Piscataway: IEEE Press, 1990: 22-25.
[5]	ARICI T, DIKBAS S, ALTUNBASAK Y.A histogram modification framework and its application for image contrast enhancement[J].IEEE Transactions on Image Processing, 2009, 18(9):1921-1935. doi: 10.1109/TIP.2009.2021548
[6]	LAND E H, MCCANN J J.Lightness and Retinex theory[J].Journal of the Optical Society of America, 1971, 61(1):1-11. doi: 10.1364/JOSA.61.000001
[7]	GUO X J, LI Y, LING H B.LIME:Low-light image enhancement via illumination map estimation[J].IEEE Transactions on Image Processing, 2017, 26(2):982-993. doi: 10.1109/TIP.2016.2639450
[8]	FU X Y, LIAO Y H, ZENG D L, et al.A probabilistic method for image enhancement with simultaneous illustration and reflectance estimation[J].IEEE Transactions on Image Processing, 2015, 24(12):4965-4977. doi: 10.1109/TIP.2015.2474701
[9]	LI M D, LIU J Y, YANG W H, et al.Structure-revealing low-light image enhancement via robust Retinex model[J].IEEE Transactions on Image Processing, 2018, 27(6):2828-2841. doi: 10.1109/TIP.2018.2810539
[10]	YUE H J, YANG J Y, SUN X Y, et al.Contrast enhancement based on intrinsic image decomposition[J].IEEE Transactions on Image Processing, 2017, 26(8):3981-3994. doi: 10.1109/TIP.2017.2703078
[11]	BANIC N, LONCARIC S.Light random sprays Retinex:Exploring the noisy illumination estimation[J].IEEE Signal Processing Letters, 2013, 20(12):1240-1243. doi: 10.1109/LSP.2013.2285960
[12]	PARK S, YU S, MOON B, et al.Low-light image enhancement using variational optimization-based Retinex model[J].IEEE Transactions on Consumer Electronics, 2017, 63(2):178-184. doi: 10.1109/TCE.2017.014847
[13]	NG M K, WANG W.A total variation model for Retinex[J].Society for Industrial and Applied Mathemetics, 2011, 4(1):345-365. http://dl.acm.org/citation.cfm?id=2078709
[14]	FU X Y, ZENG D, HUANG Y, et al.A weighted variational model for simultaneous reflectance and illumination estimation[C]//IEEE Conference on Computer Vision and Pattern Recognition.Piscataway: IEEE Press, 2016: 2782-2790.
[15]	DABOV V K K, FOI A, EGIAZARIAN K O.Image denoising by sparse 3-D transform-domain collaborative filtering[J].IEEE Transactions on Image Processing, 2007, 16(8):2080-2095. doi: 10.1109/TIP.2007.901238
[16]	WEI C, WANG W J, YANG W H, et al.Deep Retinex decomposition for low-light enhancement[C]//British Machine Vision Conference, 2018: 155-166.
[17]	SHEN L, YUE Z H, FENG F, et al.MSR-net: Low-light image enhancement using deep convolustional network[EB/OL].(2017-11-07)[2020-03-01].https://arxiv.org/abs/1711.02488.
[18]	JOBSON D J, RAHMAN Z U, WOODELL G A.A multiscale Retinex for bridging the gap between color images and the human observation of scenes[J].IEEE Transactions on Image Processing, 1997, 6(7):965-976. doi: 10.1109/83.597272
[19]	PARK S, YU S, KIM M, et al.Dual autoencoder network for Retinex-based low-light image enhancement[J].IEEE Access, 2019, 6:22084-22093. http://ieeexplore.ieee.org/document/8307190/
[20]	REN W Q, LIU S F, MA L, et al.Low-light image enhancement via a deep hybrid network[J].IEEE Transactions on Image Processing, 2019, 28(9):4364-4375. doi: 10.1109/TIP.2019.2910412
[21]	CAI J R, GU S H, ZHANG L.Learning a deep single image contrast enhancer from multi-exposure images[J].IEEE Transactions on Image Processing, 2018, 27(4):2049-2062. doi: 10.1109/TIP.2018.2794218
[22]	ZHU M F, PAN P B, CHEN W, et al.EEMEFN: Low-light image enhancement via edge-enhanced multi-exposure fusion network[C]//Proceedings of the 34th AAAI Conference on Artificial Intelligence, 2020: 13106-13113.
[23]	CHEN C, CHEN Q F, XU J, et al.Learning to see in the dark[C]//IEEE Conference on Computer Vision and Pattern Recognition.Piscataway: IEEE Press, 2018: 3291-3300.
[24]	LONG J, SHELHAMER E, DARRELL T.Fully convolutional networks for semantic segmentation[C]//IEEE Conference on Computer Vision and Pattern Recognition.Piscataway: IEEE Press, 2015: 3431-3440.
[25]	SCHMIDT U, ROTH S.Shrinkage fields for effective image restoration[C]//IEEE Conference on Computer Vision and Pattern Recognition.Piscataway: IEEE Press, 2014, 1: 2774-2781.
[26]	XIAO L, WANG J, HEIDRICH W, et al.Learning high-order filters for efficient blind deconvolution of document photographs[C]//IEEE Conference on Computer Vision and Pattern Recognition.Piscataway: IEEE Press, 2016: 734-749.
[27]	WU Q B, REN W Q, CAO X C.Learning interleaved cascade of shrinkage fields for joint image dehazing and denoising[J].IEEE Transactions on Image Processing, 2020, 29:1788-1801. doi: 10.1109/TIP.2019.2942504
[28]	WEN B, KAMILOV U, LIU D, et al.DeepCASD: An end-to-end approach for multi-spectral image super-resolution[C]//Proceedings of International Conference on Acoustics, Speech, and Signal Processing.Piscataway: IEEE Press, 2018: 6503-6505.
[29]	SAAD Y, SCHULTZ M H.GMRES:A generalized minimal residual algorithm for solving nonsymmetric linear systems[J].Society for Industrial and Applied Mathemetics, 1986, 7(3):856-869. http://cn.bing.com/academic/profile?id=3d78f4ae4ddb2205fc9f8f8a59481dfb&encoded=0&v=paper_preview&mkt=zh-cn
[30]	WANG Y, CAO Y, ZHA Z J.Progressive Retinex: Mutually reinforced illustration-noise perception network for low light image enhancement[C]//ACM International Conference on Multimedia.New York: ACM, 2019: 2015-2023.
[31]	SCHMIDT M.minFunc: Unconstrained differentiable multivariate optimization in Matlab[EB/OL].[2020-03-01].https://www.cs.ubc.ca/~schmidtm/Software/minFunc.html,2005.