北京航空航天大学学报 ›› 2021, Vol. 47 ›› Issue (3): 558-571.doi: 10.13700/j.bh.1001-5965.2020.0463

• 论文 • 上一篇    下一篇

低延迟视频编码技术

宋利1,2, 刘孝勇1, 武国庆1, 朱辰1, 黄琰1, 解蓉1, 张文军1,2   

  1. 1. 上海交通大学 图像通信与网络工程研究所, 上海 200240;
    2. 上海交通大学 未来媒体网络协同创新中心, 上海 200240
  • 收稿日期:2020-08-26 发布日期:2021-04-08
  • 通讯作者: 宋利 E-mail:song_li@sjtu.edu.cn
  • 作者简介:宋利,男,博士,教授,博士生导师。主要研究方向:新型视频编码、大数据压缩、移动计算视觉;刘孝勇,男,博士研究生。主要研究方向:可伸缩视频编码、码率控制;武国庆,男,硕士研究生。主要研究方向:视频编码;朱辰,男,博士研究生。主要研究方向:视频编码;黄琰,男,博士研究生。主要研究方向:视频编码;解蓉,女,博士,副教授,硕士生导师。主要研究方向:视频编码与转码、图像/视频处理;张文军,男,博士,教授,博士生导师。主要研究方向:图像通信与数字电视、宽带无线传输、系统芯片设计。
  • 基金资助:
    国家重点研发计划(2019YFB1802701);国家自然科学基金(61671296)

Low-latency video coding techniques

SONG Li1,2, LIU Xiaoyong1, WU Guoqing1, ZHU Chen1, HUANG Yan1, XIE Rong1, ZHANG Wenjun1,2   

  1. 1. Institute of Image Communication and Network Engineering, Shanghai Jiao Tong University, Shanghai 200240, China;
    2. Cooperative Medianet Innovation Center, Shanghai Jiao Tong University, Shanghai 200240, China
  • Received:2020-08-26 Published:2021-04-08
  • Supported by:
    National Key R & D Program of China (2019YFB1802701); National Natural Science Foundation of China (61671296)

摘要: 随着视频编码和视频传输技术的广泛应用,视频需求量剧增,实时视频通信成为视频行业的一项重要研究内容,核心目标是提供更好的用户体验和更低的延迟。低延迟视频编码是实时视频通信应用的关键部分,通过降低编码延迟可以有效地降低系统的整体延迟。首先,分析了视频传输系统的延迟来源,从通用的视频编码框架出发着重介绍了编码延迟的产生机制;其次,概述了国内外主流的视频编码标准,介绍了率失真优化技术的原理和模型,为低延迟视频编码器的设计提供了理论基础;最后,从参考结构、流水线设计、编码模式搜索、码率控制和硬件加速多个维度描述了优化编码延迟的技术手段,并总结了业界具有代表性的低延迟视频编码方案,简要说明了现有低延迟视频编码技术的局限性,并对未来的发展方向做了展望。

关键词: 低延迟, 视频编码, 实时视频通信, 率失真优化, 编码用例

Abstract: With the widespread usage of video coding and transmission techniques, demands for video have increased dramatically. Real-time video communication has become the research focus of the video industry. Its core goal is to provide a better user experience and lower latency. Low-latency video coding is a key component for real-time video communication applications. The overall system latency can be effectively reduced by reducing the coding latency. First, this paper analyzes sources of latency in the video transmission system. Focusing on the general video coding framework, this paper introduces the generation mechanism of the coding latency. Then, mainstream video coding standards at home and abroad are outlined. The detailed description of the principle and models of rate-distortion optimization techniques provides a theoretical basis for the design of low-latency video encoders. Additionally, this paper summarizes how to optimize the coding latency in terms of the reference structures, pipeline design, encoding modes search, rate control, and hardware acceleration and generalizes industrial representative low-latency video coding schemes. Finally, this paper summarizes the limitations of the existing low-latency video coding techniques and presents future research directions.

Key words: low latency, video coding, real-time video communication, rate-distortion optimization, encoding use cases

中图分类号: 


版权所有 © 《北京航空航天大学学报》编辑部
通讯地址:北京市海淀区学院路37号 北京航空航天大学学报编辑部 邮编:100191 E-mail:jbuaa@buaa.edu.cn
本系统由北京玛格泰克科技发展有限公司设计开发