HEVC对偶编码单元划分优化算法

刘美琴; 徐晨铭; 姚超; 林春雨; 赵耀

doi:10.13700/j.bh.1001-5965.2021.0528

HEVC对偶编码单元划分优化算法

doi: 10.13700/j.bh.1001-5965.2021.0528

刘美琴^{1, 2},
徐晨铭^{1, 2},
姚超^3, ,,
林春雨^{1, 2},
赵耀^{1, 2}

1.
北京交通大学信息科学研究所，北京 100044
2.
现代信息科学与网络技术北京市重点实验室，北京 100044
3.
北京科技大学计算机与通信工程学院，北京 100083

基金项目:

国家自然科学基金 61972028

国家自然科学基金 61902022

国家自然科学基金 62120106009

中央高校基本科研业务费专项资金 2019JBM018

中央高校基本科研业务费专项资金 FRF-TP-19-015A1

详细信息

通讯作者:
姚超, E-mail: yaochao@ustb.edu.cn

中图分类号: TP391
计量
- 文章访问数: 285
- HTML全文浏览量: 105
- PDF下载量: 44
- 被引次数: 0
出版历程
- 收稿日期: 2021-09-06
- 录用日期: 2021-09-17
- 网络出版日期: 2021-10-11
- 整期出版日期: 2022-08-20

Dual coding unit partition optimization algorithm of HEVC

LIU Meiqin^{1, 2},
XU Chenming^{1, 2},
YAO Chao^{3
, ,},
LIN Chunyu^{1, 2},
ZHAO Yao^{1, 2}

1.
Institute of Information Science, Beijing Jiaotong University, Beijing 100044, China
2.
Beijing Key Laboratory of Modern Information Science and Network Technology, Beijing 100044, China
3.
School of Computer and Communication Engineering, University of Science and Technology Beijing, Beijing 100083, China

Funds:

National Natural Science Foundation of China 61972028

National Natural Science Foundation of China 61902022

National Natural Science Foundation of China 62120106009

the Fundamental Research Funds for the Central Universities 2019JBM018

the Fundamental Research Funds for the Central Universities FRF-TP-19-015A1

More Information

Corresponding author: YAO Chao, E-mail: yaochao@ustb.edu.cn

摘要

摘要:
为了解决视频数据量日益增长与用户享受高质量视频体验需求之间的矛盾，HEVC在H.264/AVC标准的基础上通过引入新型的编码结构和算法进一步将编码效率提升了50%，但是也极大地提升了编码复杂度。基于此，提出对偶编码单元(CU)划分网络DualNet，来降低HEVC中帧内编码复杂度。该网络由预测网络和目标网络2个部分组成，其中，预测网络通过分析图像统计特征实现编码单元划分决策，从而跳过四叉树的遍历搜索，提高编码单元划分决策的时间效率；目标网络基于率失真代价评价和优化决策模型提升编码单元划分性能，实现模型互补和最优率失真估计。实验结果表明：与HEVC标准对比，所提算法在实现相近的压缩效果的前提下能够节省64.06%的编码时间。
- 视频编码 /
- H.265/HEVC /
- 编码单元(CU)划分 /
- 深度学习 /
- 对偶神经网络
Abstract:
To resolve the conflict between the increasing amount of video data and the demand for high-quality video experience, HEVC has boosted the compression performance by 50% based on dramatically increased complexity of H.264/AVC. In this paper, a fast coding unit (CU) partition algorithm is proposed to reduce the computational complexity of HEVC intra coding. To define the partition criteria, we design a convolutional neural network, named dual neural networks (DualNet). DualNet consists of two subnetworks, a prediction network and a target network. The prediction network is used to determine the partition actions by extracting images statistical features for skipping the traversal search of quadtree and improving the time efficiency of the CU partition. And the target network is to optimize the performance of the CU partition based on rate-distortion for achieving model complementarity. Experimental results show that the proposed algorithms can save 64.06% of the compression time with similar compression performance to HEVC.
- video coding /
- H.265/HEVC /
- coding unit (CU) partition /
- deep learning /
- dual neural networks

HTML全文

图 1 DualNet结构示意图

Figure 1. Structure of DualNet

下载: 全尺寸图片幻灯片

图 2 预测网络结构示意图

Figure 2. Structure of prediction network

下载: 全尺寸图片幻灯片

图 3 对偶优化网络结构

Figure 3. Structure of dual optimal networks

下载: 全尺寸图片幻灯片

表 1 JCT-VC测试序列参数

Table 1. JCT-VC test sequence parameters

类别	序列名称	分辨率	帧数	帧率/fps
A	Traffic	2 560×1 600	150	30
B	BasketballDrive	1 920×1 080	500	50
C	BasketballDrill	832×480	500	50
D	BasketballPass	416×240	500	50
E	Johnny	1 280×720	600	60
注：fps为帧/s。

下载: 导出CSV

表 2 消融实验结果(JCT-VC)

Table 2. Results of ablation study (JCT-VC)

类别	算法	BD-PSNR/dB	BD-BR/%	ΔT/%
A	CNN	-0.149	2.771	-63.19
	Thr-CNN	-0.133	2.480	-66.79
	DualNet-E₁	-0.148	2.757	-66.01
	DualNet-E₂	-0.131	2.429	-63.55
B	CNN	-0.119	4.981	-72.29
	Thr-CNN	-0.094	3.904	-77.10
	DualNet-E₁	-0.120	4.967	-75.31
	DualNet-E₂	-0.094	3.941	-74.29
C	CNN	-0.141	2.934	-43.77
	Thr-CNN	-0.134	2.796	-51.98
	DualNet-E₁	-0.142	2.969	-50.03
	DualNet-E₂	-0.130	2.738	-47.87
D	CNN	-0.138	2.412	-48.32
	Thr-CNN	-0.116	2.029	-50.53
	DualNet-E₁	-0.135	2.359	-44.75
	DualNet-E₂	-0.107	1.853	-57.06
E	CNN	-0.146	3.636	-75.04
	Thr-CNN	-0.136	3.355	-77.51
	DualNet-E₁	-0.141	3.501	-78.48
	DualNet-E₂	-0.138	3.421	-77.55
标准差	CNN	0.011	1.016	14.02
	Thr-CNN	0.018	0.735	13.08
	DualNet-E₁	0.011	1.013	15.01
	DualNet-E₂	0.019	0.821	12.23
平均值	CNN	-0.139	3.347	-60.52
	Thr-CNN	-0.123	2.913	-64.78
	DualNet-E₁	-0.137	3.311	-62.92
	DualNet-E₂	-0.120	2.876	-64.06

下载: 导出CSV

表 3 编码单元划分对比实验结果

Table 3. Experimental results of CU partition

类别	算法	BD-PSNR/dB	BD-BR/%	ΔT/%
	PPMAC	-0.240	4.945	-60.84
A	ETH-CNN	-0.125	2.550	-61.01
	DualNet-E₂	-0.131	2.429	-63.55
	PPMAC	-0.141	6.018	-69.51
B	ETH-CNN	-0.121	4.265	-76.32
	DualNet-E₂	-0.094	3.941	-74.29
	PPMAC	-0.538	12.205	-63.58
C	ETH-CNN	-0.133	2.863	-52.98
	DualNet-E₂	-0.130	2.738	-47.87
	PPMAC	-0.457	8.401	-63.53
D	ETH-CNN	-0.106	1.842	-56.42
	DualNet-E₂	-0.107	1.853	-57.06
	PPMAC	-0.307	7.956	-66.55
E	ETH-CNN	-0.153	3.822	-70.68
	DualNet-E₂	-0.138	3.421	-77.55
	PPMAC	0.160	2.787	3.32
标准差	ETH-CNN	0.017	0.977	9.78
	DualNet-E₂	0.019	0.821	12.23
	PPMAC	-0.337	7.905	-64.80
平均值	ETH-CNN	-0.128	3.068	-63.48
	DualNet-E₂	-0.120	2.876	-64.06

下载: 导出CSV

参考文献(31)

[1]	LIU D, LI Y, LIN J, et al. Deep learning-based video coding: A review and a case study[J]. ACM Computing Surveys, 2020, 53(1): 1-35. doi: 10.1145/3365199
[2]	TUDOR P. MPEG-2 video compression[J]. Electronics & Communication Engineering Journal, 1995, 7(6): 257-264.
[3]	WIEGAND T, SULLIVAN G J, BJONTEGAARD G, et al. Overview of the H. 264/AVC video coding standard[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2003, 13(7): 560-576. doi: 10.1109/TCSVT.2003.815165
[4]	SULLIVAN G J, OHM J R, HAN W J, et al. Overview of the high efficiency video coding (HEVC) standard[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2012, 22(12): 1649-1668. doi: 10.1109/TCSVT.2012.2221191
[5]	ZHANG Y, ZHAO Y, LIN C, et al. Block partitioning decision based on content complexity for future video coding[C]//International Conference on Image and Graphics. Berlin: Springer, 2019: 70-80.
[6]	GUO H, ZHU C, XU M, et al. Inter-block dependency-based CTU level rate control for HEVC[J]. IEEE Transactions on Broadcasting, 2019, 66(1): 113-126.
[7]	JAMALI M, COULOMBE S. Fast HEVC intra mode decision based on RDO cost prediction[J]. IEEE Transactions on Broadcasting, 2018, 65(1): 109-122.
[8]	ZHANG M, ZHAI X, LIU Z, et al. Fast algorithm for HEVC intra prediction based on adaptive mode decision and early termination of CU partition[C]//2018 Data Compression Conference. Piscataway: IEEE Press, 2018: 434-434.
[9]	LU J, LI Y. Fast algorithm for CU partitioning and mode selection in HEVC intra prediction[C]//2019 12th International Congress on Image and Signal Processing, BioMedical Engineering and Informatics. Piscataway: IEEE Press, 2019: 1-5.
[10]	WANG M, LI J, ZHANG L, et al. Extended quad-tree partitioning for future video coding[C]//2019 Data Compression Conference. Piscataway: IEEE Press, 2019: 300-309.
[11]	QING A, ZHOU W, WEI H, et al. A fast CU partitioning algorithm in HEVC inter prediction for HD/UHD video[C]//2016 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference. Piscataway: IEEE Press, 2016: 1-5.
[12]	KIBEYA H, BELGHITH F, AYED M A B, et al. A fast CU partitionning algorithm based on early detection of zero block quantified transform coefficients for HEVC standard[C]//International Image Processing, Applications and Systems Conference. Piscataway: IEEE Press, 2014: 1-5.
[13]	朱蕾琦, 张其善, 杨东凯, 等. 改进的帧内帧间模式选择快速算法[J]. 北京航空航天大学学报, 2008, 34(12): 1411-1414. https://bhxb.buaa.edu.cn/article/id/8905 ZHU L Q, ZHANG Q S, YANG D K, et al. Fast mode selection for intra and inter prediction[J]. Journal of Beijing University of Aeronautics and Astronautics, 2008, 34(12): 1411-1414(in Chinese). https://bhxb.buaa.edu.cn/article/id/8905
[14]	ZHANG D, DUAN X, ZANG D. Decision tree based fast CU partition for HEVC lossless compression of medical image sequences[C]//2017 9th International Conference on Wireless Communications and Signal Processing. Piscataway: IEEE Press, 2017: 1-6.
[15]	KIM H S, PARK R H. Fast CU partitioning algorithm for HEVC using an online-learning-based bayesian decision rule[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2016, 26(1): 130-138. doi: 10.1109/TCSVT.2015.2444672
[16]	FU B, ZHANG Q, HU J. Fast prediction mode selection and CU partition for HEVC intra coding[J]. IET Image Processing, 2020, 14(9): 1892-1900. doi: 10.1049/iet-ipr.2019.0259
[17]	LIU Z, YU X, GAO Y, et al. CU partition mode decision for HEVC hardwired intra encoder using convolution neural network[J]. IEEE Transactions on Image Processing, 2016, 25(11): 5088-5103. doi: 10.1109/TIP.2016.2601264
[18]	ZHANG Y, WANG G, TIAN R, et al. Texture-classification accelerated CNN scheme for fast intra CU partition in HEVC[C]// 2019 Data Compression Conference. Piscataway: IEEE Press, 2019: 241-249.
[19]	易清明, 林成思, 石敏. 利用深度学习的HEVC帧内编码单元快速划分算法[J]. 小型微型计算机系统, 2021, 42(2): 368-373. doi: 10.3969/j.issn.1000-1220.2021.02.025 YI Q M, LIN C S, SHI M. Fast HEVC coding units partitioning algorithm based on deep learning[J]. Journal of Computer Systems, 2021, 42(2): 368-373(in Chinese). doi: 10.3969/j.issn.1000-1220.2021.02.025
[20]	XU M, LI T, WANG Z, et al. Reducing complexity of HEVC: A deep learning approach[J]. IEEE Transactions on Image Processing, 2018, 27(10): 5044-5059. doi: 10.1109/TIP.2018.2847035
[21]	CHUNG C H, PENG W H, HU J H. HEVC/H. 265 coding unit split decision using deep reinforcement learning[C]//2017 International Symposium on Intelligent Signal Processing and Communication Systems. Piscataway: IEEE Press, 2017: 570-575.
[22]	BAE J H, YEO D, YIM J, et al. Densely distilled flow-based knowledge transfer in teacher-student framework for image classification[J]. IEEE Transactions on Image Processing, 2020, 29: 5698-5710. doi: 10.1109/TIP.2020.2984362
[23]	ABBASI S, HAJABDOLLAHI M, KARIMI N, et al. Modeling teacher-student techniques in deep neural networks for knowledge distillation[C]//2020 International Conference on Machine Vision and Image Processing. Piscataway: IEEE Press, 2020: 1-6.
[24]	XIAO R, LIU Z, WU B. Teacher-student competition for unsupervised domain adaptation[C]//2020 25th International Conference on Pattern Recognition. Piscataway: IEEE Press, 2021: 8291-8298.
[25]	LU Y, LI W, NING X, et al. Image quality assessment based on dual domains fusion[C]//2020 International Conference on High Performance Big Data and Intelligent Systems. Piscataway: IEEE Press, 2020: 1-6.
[26]	ZHOU B, ZHOU S K. DuDoRNet: Learning a dual-domain recurrent network for fast MRI reconstruction with deep T1 prior[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2020: 4273-4282.
[27]	WANG H, TIAN Q, LI L, et al. Image demoiréing with a dual-domain distilling network[C]//2021 IEEE International Conference on Multimedia and Exposition. Piscataway: IEEE Press, 2021: 1-6.
[28]	苏志雄, 李星梅, 乞建勋. 网络计划中构建对偶网络模型的理论和方法[J]. 北京航空航天大学学报, 2012, 38(2): 257-262. https://bhxb.buaa.edu.cn/article/id/12213 SU Z X, LI X M, QI J X. Theory and method of creating dual network model in network planning[J]. Journal of Beijing University of Aeronautics and Astronautics, 2012, 38(2): 257-262(in Chinese). https://bhxb.buaa.edu.cn/article/id/12213
[29]	HM software[CP/OL]. [2021-08-28]. https://hevc.hhi.fraunhofer.de/svn/svnHEVCSoftware/tags/HM-16.5/.
[30]	CPH-Intra[DS/OL]. [2021-08-28]. https://github.com/Projects/CPH.
[31]	GRELLERT M, BAMPI S, CORREA G, et al. Learning-based complexity reduction and scaling for HEVC encoders[C]//2018 IEEE International Conference on Acoustics, Speech and Signal Processing. Piscataway: IEEE Press, 2018: 1208-1212.