-
摘要:
3D视频网络服务的关键在于提高用户的体验质量（QoE），而体验质量往往会由于网络环境的变化及视频内容的不同而受到影响。传统的2D视频传输可以采用基于HTTP的自适应流媒体（HAS）速率自适应机制有效地利用网络带宽，提高用户体验质量。因此对于如何利用动态自适应流媒体技术实现至少需要传输两路视频流的3D网络视频服务已经越来越被关注。HAS技术的关键在于媒体质量级别的动态转换策略，主要研究了3D视频中不同视点比特率的变化对用户观看体验质量的影响。首先，建立一个主观数据库探讨块级客观质量与3D视频的视觉体验质量之间的关系，块级客观质量将随着比特率的变化而变化。其次，提出了一种基于卷积神经网络（CNN）的QoE模型，该模型可以通过块级客观质量有效地评估QoE，模型预测值和平均意见分（MOS）的皮尔森线性相关系数（PLCC）为0.906，可在自适应流媒体应用中为3D视频传输中不同视点的码率调整提供指导。
-
关键词:
- 3D视频 /
- 体验质量 /
- 视频质量评价 /
- 卷积神经网络(CNN) /
- HTTP自适应流媒体
Abstract:The key for 3D video network service is to improve the quality of experience (QoE) of users, which can be, however, affected by mutable network conditions and video contents. For conventional 2D videos, the HTTP adaptive streaming (HAS) technique has demonstrated its significance in improving user QoE by utilizing dynamically switched bitrates, while for 3D video transmission with at least two video streams, this technique has not yet been extensively explored. Dynamic conversion policy of the video quality level is the core of HAS technique. In this work, we investigate the impact on user QoE when introducing dynamic bitrates to different 3D videos. A subjective database is built to illustrate the connection between block-level objective quality, which changes with bitrates, and the QoE of 3D vision. Through this, we propose a convolutional neural network (CNN) based QoE model that effectively assesses the QoE by block-level objective quality. The Pearson linear correlation coefficient (PLCC) of the model predictive value and the mean opinion score (MOS) is 0.906.The proposed framework can provide guidance to inter-view bitrate balancing of HAS for 3D video transmission.
-
图 2 受试者人数上升导致的MOS数据饱和
Figure 2. MOS data saturation caused by increased number of subjects
分辨率 序列名称 空间信息 时间信息 1 024×768 KD 40.07 13.96 LB 57.83 4.05 BN 37.48 8.39 BM 56.15 9.07 1 920×1 080 JL 42.56 24.83 DB 35.72 2.56 AG 78.73 17.88 WS 54.57 17.27 1 920×1 088 CP 65.11 5.98 SK 23.65 15.05 ST 57.58 8.47 GF 51.96 16.11 
下载: 导出CSV
表 2 网络质量波动类型
Table 2. Network quality fluctuation types
质量变化 视点 编号 前5 s视频质量
(左/右)/(kbit·s-1)后5 s视频质量
(左/右)/(kbit·s-1)上升 单视点 1 50/1 000 1 000/1 000 2 100/1 000 1 000/1 000 3 200/1 000 1 000/1 000 双视点 4 50/50 1 000/1 000 5 100/100 1 000/1 000 6 200/200 1 000/1 000 下降 单视点 7 1 000/1 000 50/1 000 8 1 000/1 000 100/1 000 9 1 000/1 000 200/1 000 双视点 10 1 000/1 000 50/50 11 1 000/1 000 100/100 12 1 000/1 000 200/200
下载: 导出CSV
表 3 视频质量波动类型的MOS均值
Table 3. Average MOS of video quality fluctuation types
视频质量类型 单视点升 单视点降 双视点升 双视点降 低质量 3.39 3.11 2.00 1.83 中质量 3.88 3.55 2.87 2.78 高质量 4.17 4.11 3.61 3.47
下载: 导出CSV
-
[1] MILLER K, QUACCHIO E, GENNARI G, et al.Adaptation algorithm for adaptive streaming over HTTP[C]//Proceedings of the 19th International Packet Video Workshop.Piscataway, NJ: IEEE Press, 2012: 173-178. [2] ZHANG X, TONI L, FROSSARD P, et al.Adaptive streaming in interactive multiview video systems[J].IEEE Transactions on Circuits and Systems for Video Technology, 2019, 29(4):1130-1144. doi: 10.1109/TCSVT.2018.2819804 [3] ZHAO S, MEDHI D.SDN-Assisted adaptive streaming framework for tile-based immersive content using MPEG-DASH[C]//Proceedings of the IEEE Conference on Network Function Virtualization and Software Defined Networks.Piscataway, NJ: IEEE Press, 2017: 1-6. [4] PARK G, LEE J, LEE G, et al.Efficient 3D adaptive HTTP streaming scheme over internet TV[C]//Proceedings of the IEEE International Symposium on Broadband Multimedia Systems and Broadcasting.Piscataway, NJ: IEEE Press, 2012: 1-6. [5] GARCIA M N, SIMONE F D, TAVAKOLI S, et al.Quality of Experience and HTTP adaptive streaming: A review of subjective studies[C]//Proceedings of the 6th International Workshop on Quality of Multimedia Experience, QoMEX.Piscataway, NJ: IEEE Press, 2014: 141-146 [6] MOK R, LUO X, CHAN E, et al.QDASH: A QoE-aware DASH system[C]//Proceedings of the 3rd Multimedia Systems Conference.New York: ACM Press, 2012: 11-22. [7] TAVAKOLI S, GUTIERREZ J, GARCIA N.Subjective quality study of adaptive streaming of monoscopic and stereoscopic video[J].IEEE Journal on Selected Areas in Communications, 2014, 32(4):684-692. doi: 10.1109/JSAC.2014.140402 [8] SHAO F, LIN W, GU S, et al.Perceptual full-reference quality assessment of stereoscopic images by considering binocular visual characteristics[J].IEEE Transactions on Image Processing, 2013, 22(5):1940-1953. doi: 10.1109/TIP.2013.2240003 [9] LIN Y H, WU J L.Quality assessment of stereoscopic 3D image compression by binocular integration behaviors[J].IEEE Transactions on Image Processing, 2014, 23(4):1527-1542. http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=c49707e5d9668bd777d4e6776fcb177c [10] ZHANG Y, CHANDLER D M.3D-MAD:A full reference stereoscopic image quality estimator based on binocular lightness and contrast perception[J].IEEE Transactions on Image Processing, 2015, 24(11):3810-3825. doi: 10.1109/TIP.2015.2456414 [11] BATTISTI F, CARLI M, L E CALLET P, et al.Toward the assessment of quality of experience for asymmetric encoding in immersive media[J].IEEE Transactions on Broadcasting, 2018, 64(2):392-406. doi: 10.1109/TBC.2018.2828607 [12] RUSANOVSKYY D, MVLLER K, VETRO A.Common test conditions of 3DV core experiments[EB/OL].(2013-08-02)[2019-07-06]. https://phenix.int-evry.fr/jct2/doc_end_user/current_document.php?id=1344. [13] AVS Video Group.General test conditions of stereoscopic video coding[EB/OL]. (2010-12-16)[2019-07-06]. http://www.avs.org.cn/FileList.asp?meetingid=53&filetype=output. [14] ITU-Recommendation.BT.1788.Methodology for the subjective assessment of video quality in multimedia applications[S].Geneva: ITU, 2007. [15] ITU-Recommendation.BT.500-11.Methodology for the subjective assessment of the quality of television pictures[S].Geneva: ITU, 2012. [16] ZHANG W, LIU H.Toward a reliable collection of eye-tracking data for image quality research:Challenges, solutions, and applications[J].IEEE Transactions on Image Processing, 2017, 26(5):2424-2437. doi: 10.1109/TIP.2017.2681424 [17] ZHAO Y, CHEN Z, ZHU C, et al.Binocular just-noticeable-difference model for stereoscopic images[J].IEEE Signal Processing Letters, 2010, 18(1):19-22. http://cn.bing.com/academic/profile?id=f1eedbfa8ade2a0f4dea5dc1224e6ceb&encoded=0&v=paper_preview&mkt=zh-cn [18] SHAO F, JIANG G, WANG X, et al.Stereoscopic video coding with asymmetric luminance and chrominance qualities[J].IEEE Transactions on Consumer Electronics, 2011, 56(4):2460-2468. http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=bf798162da11ad193a1a02391fe6af1c [19] HUANG Y H, HUANG Y H, LIU W C.Quality-efficient upsampling method for asymmetric resolution stereoscopic video coding with interview motion compensation and error compensation[J].IEEE Transactions on Circuits and Systems for Video Technology, 2014, 24(3):430-442. doi: 10.1109/TCSVT.2013.2276877 [20] SAYGILI G, GURLER C G, TEKALP A M.Evaluation of asymmetric stereo video coding and rate scaling for adaptive 3D video streaming[J].IEEE Transactions on Broadcasting, 2011, 57(2):593-601. doi: 10.1109/TBC.2011.2131450 [21] CHEN C, CHOI L K, DE VECIANA G, et al.Modeling the time-varying subjective quality of HTTP video streams with rate adaptations[J].IEEE Transactions on Image Processing, 2014, 23(5):2206-2221. doi: 10.1109/TIP.2014.2312613 [22] ZHENGFANG D M, MA K D, WANG Z.Quality-of-experience for adaptive streaming videos:An expectation confirmation theory motivated approach[J].IEEE Transactions on Image Processing, 2018, 27(12):6135-6146. doi: 10.1109/TIP.2018.2855403 [23] WANG Z, BOVIK A C, SHEIKH H R, et al.Image quality assessment:from error visibility to structural similarity[J].IEEE Transactions on Image Processing, 2004, 13(4):600-612. doi: 10.1089-fpd.2009.0394/ [24] ZHANG W, QU C, MA L, et al.Learning structure of stereoscopic image for no-reference quality assessment with convolutional neural network[J].Pattern Recognition, 2016, 59:176-187. doi: 10.1016/j.patcog.2016.01.034 -
下载:
点击查看大图
计量
- 文章访问数: 793
- HTML全文浏览量: 196
- PDF下载量: 367
- 被引次数: 0
