基于边缘和结构的无参考屏幕内容图像质量评估

魏乐松; 陈俊豪; 牛玉贞

doi:10.13700/j.bh.1001-5965.2019.0367

基于边缘和结构的无参考屏幕内容图像质量评估

doi: 10.13700/j.bh.1001-5965.2019.0367

福州大学数学与计算机科学学院, 福州 350100

基金项目:

国家自然科学基金 61672158

福建省自然科学基金 2019J2006

详细信息

作者简介:
魏乐松  男, 硕士研究生。主要研究方向:多媒体

陈俊豪  男, 博士研究生。主要研究方向:多媒体、计算机图像学和人工智能

牛玉贞  女, 博士, 教授, 博士生导师。主要研究方向:计算机视觉、多媒体和计算机图像学

通讯作者:
牛玉贞. E-mail: yuzhenniu@gmail.com

中图分类号: TP399
计量
- 文章访问数: 860
- HTML全文浏览量: 102
- PDF下载量: 351
- 被引次数: 0
出版历程
- 收稿日期: 2019-07-09
- 录用日期: 2019-08-03
- 网络出版日期: 2019-12-20

Blind quality assessment for screen content images based on edge and structure

College of Mathematics and Computer Science, Fuzhou University, Fuzhou 350100, China

Funds:

National Natural Science Foundation of China 61672158

Natural Science Foundation of Fujian Province 2019J2006

More Information

Corresponding author: NIU Yuzhen. E-mail: yuzhenniu@gmail.com

摘要

摘要:
屏幕内容图像（SCI）是一种与传统自然图像不同的图像，具有更多的文本、图形以及特殊的布局。考虑文本、图形、图像和布局对屏幕内容图像质量的影响，提出了针对屏幕内容图像的基于边缘和结构的无参考质量评估（BES）算法。文本、图形和图像具有大量边缘，并且人类视觉系统对边缘高度敏感，因此BES算法首先使用Gabor滤波器的虚部提取边缘并计算每张屏幕内容图像的边缘特征。其次，提取一个结构特征来表示屏幕内容图像的布局。具体而言，利用Scharr滤波器计算得到一个局部二值模式（LBP）图，接着利用LBP图计算得到结构特征。最后，应用随机森林回归算法将边缘和结构特征映射为主观分数。实验结果表明，在数据库SIQAD和SCID上，所提出BES算法性能的皮尔森线性相关系数（PLCC）相对于对比算法中最先进的无参考算法，分别提高了2.63%和11.22%，甚至高于一些全参考算法。
- 屏幕内容图像 /
- 无参考质量评估 /
- 随机森林回归 /
- 边缘特征 /
- 结构特征
Abstract:
A screen content image (SCI) has great difference compared with a natural image, and an SCI contains more text, graphic, and special layout. Considering the influences of texts, graphics, pictures, and layout on quality of an SCI, a blind quality assessment metric for SCIs based on edge and structure (BES) has been proposed. Since texts, graphics, and pictures have a large number of edges and the human visual system is highly sensitive to edges, the BES metric first extracts edges using the imaginary part of the Gabor filter and computes an edge feature for each SCI. Second, a structure feature is extracted to represent the layout of an SCI. Specifically, the Scharr filter is exploited to calculate a local binary pattern (LBP) map which is used to compute a structure feature. Finally, a random forest regression algorithm is applied to map the edge and structure features to subjective scores. The experimental results show that in the database SIQAD and SCID, the Pearson linear correlation coefficient (PLCC) of the performance of the proposed BES index is 2.63% and 11.22% higher than the latest no reference index in the comparison respectively, and even higher than some full reference indexes.
- screen content image /
- blind quality assessment /
- random forest regression /
- edge feature /
- structure feature

HTML全文

图 1 失真屏幕内容图像以及对应的特征图和特征直方图示例

Figure 1. Examples of distorted screen content images, their feature maps, and feature histograms

下载: 全尺寸图片幻灯片

表 1 SIQAD数据库上的实验结果

Table 1. Experimental results on SIQAD database

指标	失真类型	全参考								无参考
指标	失真类型	PSNR	SSIM	MAD	GMSD	SPQA	SQI	ESIM	GFM	NIOE	BRISQUE	GWH-GLBP	NRLT	BES
PLCC	GN	0.905 3	0.880 6	0.885 2	0.895 6	0.892 1	0.882 9	0.889 1	0.899 0	0.863 4	0.904 5	0.849 3	0.898 3	0.909 0
	GB	0.850 3	0.910 4	0.912 0	0.909 4	0.905 8	0.920 2	0.923 4	0.914 3	0.756 0	0.890 9	0.909 8	0.882 6	0.922 4
	MB	0.704 4	0.806 0	0.836 1	0.843 6	0.831 5	0.878 9	0.888 6	0.866 2	0.548 7	0.857 1	0.832 0	0.861 1	0.869 4
	CC	0.740 1	0.743 5	0.393 3	0.782 7	0.799 2	0.772 4	0.764 1	0.810 7	0.355 5	0.541 7	0.410 8	0.787 8	0.822 4
	JPEG	0.754 5	0.748 7	0.766 2	0.774 6	0.769 6	0.821 8	0.799 9	0.839 8	0.598 0	0.288 7	0.572 9	0.781 2	0.757 6
	J2K	0.789 3	0.774 9	0.834 4	0.850 9	0.825 2	0.827 1	0.788 8	0.848 6	0.516 5	0.350 9	0.716 3	0.748 9	0.818 2
	LSC	0.780 5	0.730 7	0.818 4	0.855 9	0.795 8	0.831 0	0.791 5	0.828 8	0.586 9	0.286 5	0.551 2	0.730 9	0.757 5
	ALL	0.585 8	0.756 1	0.546 7	0.729 1	0.858 4	0.864 4	0.878 8	0.882 8	0.426 1	0.359 1	0.787 4	0.848 2	0.870 5
SRCC	GN	0.879 0	0.869 4	0.872 1	0.885 6	0.882 3	0.860 2	0.875 7	0.879 5	0.842 9	0.887 5	0.828 7	0.874 7	0.885 1
	GB	0.857 3	0.892 1	0.908 7	0.911 9	0.901 7	0.924 4	0.923 9	0.913 2	0.649 4	0.871 5	0.894 1	0.865 6	0.904 5
	MB	0.713	0.804 1	0.835 7	0.844 1	0.825 5	0.88 1	0.893 8	0.869 9	0.427 2	0.865 3	0.822 9	0.843 2	0.857 1
	CC	0.682 8	0.640 5	0.390 7	0.637 8	0.615 4	0.667 7	0.610 8	0.703 8	0.132 4	0.313 5	0.246 0	0.599 6	0.658 2
	JPEG	0.756 8	0.757 6	0.767 4	0.771 2	0.767 3	0.818 9	0.798 9	0.843 4	0.510 9	0.220 8	0.480 3	0.738 6	0.707 2
	J2K	0.774 6	0.760 3	0.838 2	0.843 6	0.815 2	0.816 9	0.782 7	0.844 4	0.323 8	0.301 8	0.701 7	0.731 5	0.793 9
	LSC	0.793	0.737 1	0.815 4	0.859 2	0.800 3	0.843 2	0.795 8	0.844 5	0.394 4	0.184 4	0.504 9	0.667 5	0.729 1
	ALL	0.557 0	0.756 6	0.583 1	0.730 5	0.841 6	0.854 8	0.863 2	0.873 5	0.382 7	0.662	0.719 4	0.824 5	0.848 8
RMSE	GN	6.337 2	7.704 4	6.939 1	6.635 4	6.739 4		6.827 2	6.683 5	7.408 3	6.013 7	7.682 9	6.310 0	6.106 2
	GB	7.737 6	6.361 9	6.226 9	6.981 6	6.430 1		5.827 0	6.145 9	9.802 6	6.479 2	6.146 6	6.689 2	5.597 4
	MB	9.228 7	7.060 0	7.132 29	6.981 6	7.222 3		5.963 9	6.518 4	10.613 6	6.198 2	6.934 7	6.428 4	6.432 3
	CC	8.459 1	6.818 4	11.565	7.829 7	7.618 4		8.114 1	7.363 8	11.467 8	9.966	11.198	7.560 1	6.713 5
	JPEG	6.116 5	5.640 6	6.038	5.942 7	6.000 0		5.640 1	5.100 9	7.286 9	8.603 7	7.323 8	5.712 1	5.988 0
	J2K	6.381 9	6.180 4	5.727 6	5.459 1	5.870 6		6.387 7	5.498 5	8.331 8	9.094	6.775 0	6.505 8	5.704 8
	LSC	5.333 6	4.937 9	4.902 5	4.412 1	5.166 4		5.215 0	4.773 6	6.815 9	7.871 2	7.028 9	5.783 6	5.390 8
	ALL	11.601	10.855	11.986	9.797 2	7.342 1	7.198 2	6.831 0	6.723 4	12.794 1	11.717	8.672 6	7.444 7	6.914 7

下载: 导出CSV

表 2 SCID数据库上的实验结果

Table 2. Experimental results on SCID database

指标	失真类型	全参考					无参考
指标	失真类型	PSNR	SSIM	MAD	IFC	GSIM	NIQE	BRISQUE	GWH-GLBP	NRLT	BES
PLCC	GN	0.953 0	0.935 4	0.931 5	0.889 7	0.917 0	0.832 4	0.964 5	0.861 9	0.968 1	0.939 8
	GB	0.777 2	0.871 1	0.855 9	0.840 6	0.844 9	0.591 2	0.584 7	0.714 4	0.667 2	0.855 3
	MB	0.761 5	0.879 4	0.836 2	0.337 2	0.838 3	0.539 0	0.643 4	0.673 6	0.590 6	0.931 4
	CC	0.743 5	0.690 3	0.498 7	0.119 8	0.867 5	0.284 0	0.473 4	0.252 0	0.499 3	0.812 2
	JPEG	0.839 3	0.858 1	0.925 1	0.876 2	0.937 3	0.682 4	0.587 9	0.769 4	0.851 2	0.820 5
	J2K	0.917 6	0.858 6	0.938 1	0.857 0	0.944 1	0.709 9	0.551 5	0.664 5	0.832 6	0.795 4
	CSC	0.062 2	0.089 0	0.129 6	0.076 4	0.056 0	0.219 6	0.187 9	0.213 4	0.196 3	0.311 0
	HEVC-SCC	0.799 1	0.863 5	0.895 3	0.791 8	0.883 5	0.428 9	0.441 8	0.614 2	0.533 4	0.587 5
	CQD	0.921 0	0.866 8	0.901 4	0.765 5	0.897 4	0.578 7	0.714 6	0.603 4	0.725 0	0.830 3
	ALL	0.762 2	0.757 9	0.771 9	0.628 5	0.704 2	0.339 2	0.630 3	0.664 7	0.706 0	0.785 2
SRCC	GN	0.942 4	0.917 1	0.926 2	0.887 7	0.911 2	0.850 5	0.987 1	0.866 9	0.961 2	0.933 3
	GB	0.702	0.869 8	0.860 3	0.835 1	0.842 0	0.346 2	0.516 2	0.701 9	0.625 0	0.847 9
	MB	0.737 5	0.858 8	0.829 6	0.447 7	0.819 4	0.369 1	0.626 5	0.641 9	0.512 3	0.788 3
	CC	0.726 5	0.656 4	0.478 4	0.119 8	0.830 4	0.100 4	0.309 6	0.231 4	0.303 1	0.477 3
	JPEG	0.823 1	0.849 0	0.924 2	0.877 0	0.936 6	0.629 1	0.546 9	0.743 0	0.838 2	0.795 7
	J2K	0.907 4	0.843 9	0.933 0	0.845 7	0.934 9	0.654 0	0.499 7	0.639 8	0.791 6	0.763 4
	CSC	0.090 8	0.096 3	0.144 0	0.052 1	0.121 4	0.040 1	0.014 3	0.098 6	0.100 9	0.109 8
	HEVC-SCC	0.807 4	0.826 3	0.877 1	0.786 9	0.873 0	0.392 1	0.244 4	0.485 1	0.452 4	0.465 4
	CQD	0.908 0	0.776 6	0.902 4	0.736 8	0.863 4	0.439 7	0.558 0	0.504 6	0.592 3	0.754 8
	ALL	0.751 2	0.714 6	0.757 6	0.579 9	0.699 3	0.272 7	0.606 6	0.621 1	0.686 6	0.761 3
RMSE	GN	3.809 3	4.445 8	4.571 4	5.738 0	5.012 7	6.960 2	3.236 9	6.362 3	3.167 9	4.516 2
	GB	6.663 3	5.199 8	5.447 5	5.735 4	5.664 8	8.442 2	8.476 7	7.311 0	7.756 7	5.436 5
	MB	7.084 3	5.204 4	5.994 7	10.290 5	5.960 7	8.984 9	8.243 7	7.931 1	8.697 9	6.425 3
	CC	5.986 7	6.476 7	7.759 0	8.887 6	4.452 4	8.482 1	7.717 8	8.569 8	7.689 1	5.183 6
	JPEG	8.171 8	7.717 9	5.707 6	7.243 1	5.236 9	10.900 1	12.138 6	9.494 8	7.889 7	8.634 2
	J2K	6.322 2	8.156 2	5.510 3	8.198 6	5.246 2	11.167 3	13.125 7	11.586 3	8.714 8	9.254 0
	CSC	9.820 3	9.800 3	9.756 4	9.810 6	9.823 9	9.348 9	9.496 6	9.523 4	9.535 6	9.265 4
	HEVC-SCC	8.400 9	8.503 7	6.198 8	8.496 9	6.517 6	12.369 5	12.354 5	10.827 0	11.556 2	11.138 2
	CQD	4.981 4	7.985 5	5.535 4	8.226 9	5.640 6	10.376 3	8.708 3	10.002 5	8.850 8	7.101 5
	ALL	9.168 2	9.613 3	8.973 9	11.015 7	10.055 2	13.327 1	10.996 7	10.520 5	9.981 1	8.831 9

下载: 导出CSV

参考文献(22)

[1]	WANG Z, BOVIK A C, SHEIKH H R, et al.Image quality assessment:from error visibility to structural similarity[J].IEEE Transactions on Image Processing, 2004, 13(4):600-612. doi: 10.1089-fpd.2009.0394/
[2]	XUE W, ZHANG L, MOU X, et al.Gradient magnitude similarity deviation:A highly efficient perceptual image quality index[J].IEEE Transactions on Image Processing, 2014, 23(2):684-695. http://d.old.wanfangdata.com.cn/OAPaper/oai_arXiv.org_1308.3052
[3]	MITTAL A, SOUNDARARAJAN R, BOVIK A C.Making a 'completely blind' image quality analyzer[J].IEEE Signal Processing Letters, 2013, 20(3):209-212. doi: 10.1109/LSP.2012.2227726
[4]	MITTAL A, MOORTHY A K, BOVIK A C. No-reference image quality assessment in the spatial domain[J]. IEEE Transactions on Image Processing, 2012, 21(12):4695-4708. doi: 10.1109/TIP.2012.2214050
[5]	YANG H, FANG Y, LIN W.Perceptual quality assessment of screen content images[J].IEEE Transactions on Image Processing, 2015, 24(11):4408-4421. doi: 10.1109/TIP.2015.2465145
[6]	WANG S, GU K, ZENG K, et al.Objective quality assessment and perceptual compression of screen content images[J].IEEE Computer Graphics and Applications, 2016, 38(1):47-58. http://cn.bing.com/academic/profile?id=afb530fcb8d57edf81be66a52bddb021&encoded=0&v=paper_preview&mkt=zh-cn
[7]	WANG S, GU K, ZHANG X, et al.Reduced-reference quality assessment of screen content images[J].IEEE Transactions on Circuits and Systems for Video Technology, 2016, 28(1):1-14. http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=3b4d028baf5733721ef44cf665499817
[8]	SHAO F, GAO Y, LI F, et al.Toward a blind quality predictor for screen content images[J].IEEE Transactions on Systems, Man, and Cybernetics:Systems, 2017, 48(9):1-10. http://cn.bing.com/academic/profile?id=2459d4882e48e7d12f4bde49d15c8aa2&encoded=0&v=paper_preview&mkt=zh-cn
[9]	FANG Y, YAN J, LI L, et al.No reference quality assessment for screen content images with both local and global feature representation[J].IEEE Transactions on Image Processing, 2018, 27(4):1600-1610. http://cn.bing.com/academic/profile?id=b45dfd75d33067d502084c9f002b9728&encoded=0&v=paper_preview&mkt=zh-cn
[10]	GUO X, HUANG L, GU K, et al.Naturalization of screen content images for enhanced quality evaluation[J].IEICE Transactions on Information and Systems, 2017, 100(3):574-577. http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=J-STAGE_2199740
[11]	NI Z, ZENG H, MA L, et al.A Gabor feature-based quality assessment model for the screen content Images[J].IEEE Transactions on Image Processing, 2018, 27(9):4516-4528. doi: 10.1109/TIP.2018.2839890
[12]	DAUGMAN J G.Uncertainty relation for resolution in space, spatial frequency, and orientation optimized by two-dimensional visual cortical filters[J].Journal of the Optical Society of America A, 1985, 2(7):1160-1169. doi: 10.1364/JOSAA.2.001160
[13]	JONES J P, PALMER L A.An evaluation of the two-dimensional Gabor filter model of simple receptive fields in cat striate cortex[J].Journal of Neurophysiology, 1987, 58(6):1233-1258. doi: 10.1152/jn.1987.58.6.1233
[14]	GEUSEBROEK J M, VAN DEBOOMGAARD R, SMEULDERS A W M, et al.Color invariance[J].IEEE Transactions on Pattern Analysis and Machine Intelligence, 2001, 23(12):1338-1350. doi: 10.1109/34.977559
[15]	GEUSEBROEK J M, VAN DEBOOMGAARD R, SMEULDERS A W M, et al.Color and scale: The spatial structure of color images[C]//Proceedings of the European Conference on Computer Vision.Berlin: Springer, 2000: 331-341.
[16]	OJALA T, PIETIKAINEN M, MAENPAA T.Multiresolution gray-scale and rotation invariant texture classification with local binary patterns[J].Classification with Local Binary Patterns, 2002, 24(7):971-987. http://cn.bing.com/academic/profile?id=bbab6ef5dd30d698066c963f5602e424&encoded=0&v=paper_preview&mkt=zh-cn
[17]	LI Q, LIN W, FANG Y.No-reference quality assessment for multiply-distorted images in gradient domain[J].IEEE Signal Processing Letters, 2016, 23(4):541-545. doi: 10.1109/LSP.2016.2537321
[18]	HUGHES H C, NOZAWA G, KITTERLE F.Global precedence, spatial frequency channels, and the statistics of natural images[J].Journal of Cognitive Neuroscience, 1996, 8(3):197-230. doi: 10.1162/jocn.1996.8.3.197
[19]	NI Z, MA L, ZENG H, et al.ESIM:Edge similarity for screen content image quality assessment[J].IEEE Transactions on Image Processing, 2017, 26(10):4818-4831. doi: 10.1109/TIP.2017.2718185
[20]	ERIC C L, DAMON M C.Most apparent distortion:Full-reference image quality assessment and the role of strategy[J].Journal of Electronic Imaging, 2010, 19(1):011006. doi: 10.1117/1.3267105
[21]	SHEIKH H R, BOVIK A C, DE VECIANA G, et al.An information fidelity criterion for image quality assessment using natural scene statistics[J].IEEE Transactions on Image Processing, 2005, 14(12):2117-2128. doi: 10.1109/TIP.2005.859389
[22]	LIU A, LIN W, NARWARIA M.Image quality assessment based on gradient similarity[J].IEEE Transactions on Image Processing, 2012, 21(4):1500-1512. doi: 10.1109/TIP.2011.2175935