| Citation: | WU Zhishan, ZHANG Shuai, NIU Yuzhenet al. Retargeted image quality assessment based on multi-scale distortion-aware feature[J]. Journal of Beijing University of Aeronautics and Astronautics, 2019, 45(12): 2487-2494. doi: 10.13700/j.bh.1001-5965.2019.0368(in Chinese)
|
The subjective visual experience of viewing images using a variety of display devices is usually affected by image retargeting operation. To improve the consistency between subjective perception and objective assessment for the retargeted images, we present an objective retargeted image quality assessment (RIQA) method based on multi-scale distortion-aware (MSDA) features. Because semantic distortion and detail distortion appear on different scales of the image, we propose to extract distortion-aware features from multiple scales of the image. Specifically, we present an accurate measurement for the aspect ratio similarity (ARS) between the original and retargeted images. Furthermore, we use a fused visual attention map to simulate the subjective attention of the human visual system to the image. The experimental results on the two benchmark databases show that the Kendall rank correlation coefficient (KRCC), Pearson linear correlation coefficient (PLCC), and Spearman rank-order correlation coefficient (SRCC) indicators of the proposed MSDA method are 4.1%, 1.8%, and 4.5% higher than the optimal method in the comparative methods, respectively.
| [1] |
RUBINSTEIN M, SHAMIR A, AVIDAN S.Multi-operator media retargeting[J].ACM Transactions on Graphics, 2009, 28(3):23.
|
| [2] |
AVIDAN S, SHAMIR A.Seam carving for content-aware image resizing[J].ACM Transactions on Graphics, 2007, 26(3):10. doi: 10.1145/1276377.1276390
|
| [3] |
PRITCH Y, KAV-VENAKI E, PELEG S.Shift-map image editing[C]//IEEE 12th International Conference on Computer Vision.Piscataway, NJ: IEEE Press, 2009: 151-158.
|
| [4] |
WANG Y S, TAI C L, SORKINE O, et al.Optimized scale-and-stretch for image resizing[J].ACM Transactions on Graphics, 2008, 27(5):118.
|
| [5] |
KRÄHENBVHL P, LANG M, HORNUNG A, et al.A system for retargeting of streaming video[J].ACM Transactions on Graphics, 2008, 28(5):126. doi: 10.1038-clpt.2011.14/
|
| [6] |
WOLF L, GUTTMANN M, COHEN-OR D.Non-homogeneous content-driven video-retargeting[C]//IEEE 11th International Conference on Computer Vision.Piscataway, NJ: IEEE Press, 2007: 1-6.
|
| [7] |
RUBINSTEIN M, GUTIERREZ D, SORKINE O, et al.A comparative study of image retargeting[J].ACM Transactions on Graphics, 2010, 29(6):160.
|
| [8] |
MA L, LIN W, DENG C, et al.Image retargeting quality assessment:A study of subjective scores and objective metrics[J].IEEE Journal of Selected Topics in Signal Processing, 2012, 6(6):626-639. doi: 10.1109/JSTSP.2012.2211996
|
| [9] |
SIMAKOV D, CASPI Y, SHECHTMAN E, et al.Summarizing visual data using bidirectional similarity[C]//IEEE Conference on Computer Vision and Pattern Recognition.Piscataway, NJ: IEEE Press, 2008: 1-8.
|
| [10] |
LIU Y J, LUO X, XUAN Y M, et al.Image retargeting quality assessment[J].Computer Graphics Forum, 2011, 30(2):583-592. doi: 10.1111/j.1467-8659.2011.01881.x
|
| [11] |
ZHANG Y, FANG Y, LIN W, et al.Backward registration based aspect ratio similarity for image retargeting quality assessment[J].IEEE Transactions on Image Processing, 2016, 25(9):4286-4297. doi: 10.1109/TIP.2016.2585884
|
| [12] |
ZHANG Y, LIN W, LI Q, et al.Multiple-level feature-based measure for retargeted image quality[J].IEEE Transactions on Image Processing, 2018, 27(1):451-463.
|
| [13] |
ZHANG S, NIU Y Z, LIN J W, et al.Visual attention fusion framework for image retargeting quality assessment[J].IOP Conference Series:Earth and Environmental Science, 2019, 234(1):12-64.
|
| [14] |
CHEN Z, LIN J, LIAO N, et al.Full reference quality assessment for image retargeting based on natural scene statistics modeling and bi-directional saliency similarity[J].IEEE Transactions on Image Processing, 2017, 26(11):5138-5148. doi: 10.1109/TIP.2017.2736422
|
| [15] |
FANG Y, CHEN Z, LIN W, et al.Saliency detection in the compressed domain for adaptive image retargeting[J].IEEE Transactions on Image Processing, 2012, 21(9):3888-3901. doi: 10.1109/TIP.2012.2199126
|
| [16] |
QIN Y, LU H, XU Y, et al.Saliency detection via cellular automata[C]//IEEE Conference on Computer Vision and Pattern Recognition.Piscataway, NJ: IEEE Press, 2015: 110-119.
|
| [17] |
JOACHIMS T.Training linear SVMs in linear time[C]//ACM SIGKDD International Conference on Knowledge Discovery & Data Mining.New York: ACM, 2006: 217-226.
|
| [18] |
LIU C, YUEN J, TORRALBA A.SIFT flow:Dense correspondence across scenes and its applications[J].IEEE Transactions on Pattern Analysis & Machine Intelligence, 2010, 33(5):978-994.
|
| [19] |
PELE O, WERMAN M.Fast and robust earth mover's distances[C]//IEEE 12th International Conference on Computer Vision.Piscataway, NJ: IEEE Press, 2009: 460-467.
|
| [20] |
HSU C C, LIN C W, FANG Y, et al.Objective quality assessment for image retargeting based on perceptual geometric distortion and information loss[J].IEEE Journal of Selected Topics in Signal Processing, 2014, 8(3):377-389. doi: 10.1109/JSTSP.2014.2311884
|
| [21] |
KENDALL M G.A new measure of rank correlation[J].Biometrika, 1938, 30(1):81-93.
|
| [22] |
DONG W, ZHOU N, PAUL J C, et al.Optimized image resizing using seam carving and scaling[J].ACM Transactions on Graphics, 2009, 28(5):125.
|
| [23] |
KARNI Z, FREEDMAN D, GOTSMAN C.Energy-based image deformation[J].Computer Graphics Forum, 2009, 28(5):1257-1268. doi: 10.1111/j.1467-8659.2009.01503.x
|
| [24] |
WANG Z, LU L, BOVIK A C.Video quality assessment based on structural distortion measurement[J].Signal Processing:Image Communication, 2004, 19(2):121-132. doi: 10.1016/S0923-5965(03)00076-6
|
| [25] |
SHEIKH H R, SABIR M F, BOVIK A C.A statistical evaluation of recent full reference image quality assessment algorithms[J].IEEE Transactions on Image Processing, 2006, 15(11):3440-3451. doi: 10.1109/TIP.2006.881959
|
| [26] |
WU Z, SU L, HUANG Q M.Cascaded partial decoder for fast and accurate salient object detection[EB/OL].(2018-04-18)[2019-07-09].
|
| [1] | ZHAO C L，ZHU J，HU Y J，et al. Aero-engine defect detection by fusing attention and multi-scale features[J]. Journal of Beijing University of Aeronautics and Astronautics，2025，51（3）：892-903 （in Chinese）. doi: 10.13700/j.bh.1001-5965.2023.0147. |
| [2] | ZHANG K，LIU Y，HU K. RAW image reconstruction based on multi-scale attention mechanism[J]. Journal of Beijing University of Aeronautics and Astronautics，2025，51（1）：257-264 （in Chinese）. doi: 10.13700/j.bh.1001-5965.2022.0959. |
| [3] | WU N，MU C P，HE Y，et al. Multi-scale infrared and visible image fusion based on nest connection[J]. Journal of Beijing University of Aeronautics and Astronautics，2025，51（2）：683-691 （in Chinese）. doi: 10.13700/j.bh.1001-5965.2023.0077. |
| [4] | ZHANG DongPing, ZHANG Yu, LIU Zhi-Yong, REN Ye, TANG Si-Liang, HE Shu-Ji, WEI Yang-Yue. Domain Adaptation and Multi-Scale Feature Fusion for Cross-Domain Few-Shot Object Detection[J]. Journal of Beijing University of Aeronautics and Astronautics. doi: 10.13700/j.bh.1001-5965.2024.0751 |
| [5] | LIU Zhenjiang, ZHANG Huijuan, JI Miaoxin, LU Tao, ZHANG Chi, YU Yuanjin. Lightweight multi-scale feature fusion enhancement algorithm for spatial non-cooperative small target detection[J]. Journal of Beijing University of Aeronautics and Astronautics. doi: 10.13700/j.bh.1001-5965.2024.0509 |
| [6] | SHEN Yu, LI Yangyang, LI Bohao, GAO Baoqu, WEI Ziyi, BAI Shan. Small Target Vehicle Detection Based On Perceptual Enhancement And Multi Scale Fusion[J]. Journal of Beijing University of Aeronautics and Astronautics. doi: 10.13700/j.bh.1001-5965.2024.0124 |
| [7] | YANG Zhangang, ZHOU Qingyu. Multi-timescale lifespan evaluation of SiC power modules in aviation converters[J]. Journal of Beijing University of Aeronautics and Astronautics. doi: 10.13700/j.bh.1001-5965.2024.0691 |
| [8] | LIANG H，WANG H R，WANG D. Cross-modal hashing network based on self-attention similarity transfer[J]. Journal of Beijing University of Aeronautics and Astronautics，2024，50（2）：615-622 （in Chinese）. doi: 10.13700/j.bh.1001-5965.2022.0402. |
| [9] | HUO Jiuyuan, LI Xin, CHANG Chen, LI Yufeng, ZHANG Yaonan. Roll bearing life prediction based on multi-scale feature fusion[J]. Journal of Beijing University of Aeronautics and Astronautics. doi: 10.13700/j.bh.1001-5965.2024.0161 |
| [10] | LIU Sheng, JIN Xuepeng, GAO Feng, GAN Yanhai. Sparse Attention and Deformable Feature Cross Fusion-Based Multi-source Remote Sensing Image Classification Method[J]. Journal of Beijing University of Aeronautics and Astronautics. doi: 10.13700/j.bh.1001-5965.2024.0480 |
| [11] | ZHANG Yun-zuo, YANG Yue-hui, DONG Xu, KANG Wei-li, BAI Jing. Dual-stream Deep Network Infrared Gait Recognition Based on Residual Multi-scale Fusion[J]. Journal of Beijing University of Aeronautics and Astronautics. doi: 10.13700/j.bh.1001-5965.2023.0605 |
| [12] | FENG X，WEI X K，LIU C H，et al. Contraband classification method for X-ray security images considering sample imbalance[J]. Journal of Beijing University of Aeronautics and Astronautics，2023，49（12）：3215-3221 （in Chinese）. doi: 10.13700/j.bh.1001-5965.2022.0095. |
| [13] | XU H B，FAN J，NI M，et al. Molecular dynamics study on dry friction damper with temperature influence[J]. Journal of Beijing University of Aeronautics and Astronautics，2023，49（11）：3031-3038 （in Chinese）. doi: 10.13700/j.bh.1001-5965.2022.0045. |
| [14] | ZHANG Zhi, YI Hua-hui, ZHENG Jin. Few-Shot Object Detection of Aerial Image Based on Language Guidance Vision[J]. Journal of Beijing University of Aeronautics and Astronautics. doi: 10.13700/j.bh.1001-5965.2023.0491 |
| [15] | CHENG De-qiang, WANG Pei-jie, DONG Yan-qiang, KOU Qi-qi, JIANG He. Image super-resolution reconstruction based on multi-scale spatial attention guidance network[J]. Journal of Beijing University of Aeronautics and Astronautics. doi: 10.13700/j.bh.1001-5965.2023.0547 |
| [16] | CHEN Y，CHEN J，TAO M F. Mural inpainting with generative adversarial networks based on multi-scale feature and attention fusion[J]. Journal of Beijing University of Aeronautics and Astronautics，2023，49（2）：254-264 （in Chinese）. doi: 10.13700/j.bh.1001-5965.2021.0242. |
| [17] | LI Chenxuan, GU Jiaojiao, WANG Lei, QIAN Kun, FENG Zeqin. Warship's vital parts detection algorithm based on lightweight Anchor-Free network with multi-scale feature fusion[J]. Journal of Beijing University of Aeronautics and Astronautics, 2022, 48(10): 2006-2019. doi: 10.13700/j.bh.1001-5965.2021.0050 |
| [18] | SU Kaiqi, YAN Weiqing, XU Jindong. 3D object detection based on multi-path feature pyramid network for stereo images[J]. Journal of Beijing University of Aeronautics and Astronautics, 2022, 48(8): 1487-1494. doi: 10.13700/j.bh.1001-5965.2021.0525 |
| [19] | CHENG Keyang, RONG Lan, JIANG Senlin, ZHAN Yongzhao. Double drive adaptive super-resolution reconstruction method of remote sensing images for object detection[J]. Journal of Beijing University of Aeronautics and Astronautics, 2022, 48(8): 1343-1352. doi: 10.13700/j.bh.1001-5965.2021.0517 |
| [20] | SONG Yuqin, GAO Shijie, ZENG Hedong, XIONG Gaoqiang. Multi-scale depthwise separable convolution facial expression recognition embedded in attention mechanism[J]. Journal of Beijing University of Aeronautics and Astronautics, 2022, 48(12): 2381-2387. doi: 10.13700/j.bh.1001-5965.2021.0114 |
| 1. | 胡波，谢国庆，李雷达，李静，杨嘉琛，路文，高新波. 图像重定向质量评价的研究进展. 中国图象图形学报. 2024(01): 22-44 .  | |
| 2. | 孙令翠，冯辉宗. 基于编码量化参数调节的图像清晰化处理. 计算机仿真. 2022(04): 180-184 . | |
| 3. | 赵莉，张克旺. 基于傅里叶描述子的实装战技图像典型特征相似度检测方法. 南京理工大学学报. 2022(04): 406-411 . |
Figures(6) / Tables(8)
Copyright © Journal of Beijing University of Aeronautics and Astronautics
Address: Editorial Department of Journal of Beijing University of Aeronautics and Astronautics, 37 Xueyuan Road, Haidian District, Beijing Post Code: 100191 Email: jbuaa@buaa.edu.cn
Supported by:
Beijing Renhe Information Technology Co., Ltd.
Zheng Feng, Ai Jun, Shang Jingwei, et al. Application of fuzzy discrimination in reliability accelerated testing[J]. Journal of Beijing University of Aeronautics and Astronautics, 2013, 39(4): 498-502. (in Chinese)
DownLoad:
