留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

基于置信度的TOF与双目系统深度数据融合

孙哲 张勇 常衢通

孙哲, 张勇, 常衢通等 . 基于置信度的TOF与双目系统深度数据融合[J]. 北京航空航天大学学报, 2018, 44(8): 1764-1771. doi: 10.13700/j.bh.1001-5965.2017.0653
引用本文: 孙哲, 张勇, 常衢通等 . 基于置信度的TOF与双目系统深度数据融合[J]. 北京航空航天大学学报, 2018, 44(8): 1764-1771. doi: 10.13700/j.bh.1001-5965.2017.0653
SUN Zhe, ZHANG Yong, CHANG Qutonget al. In-depth data fusion of TOF and stereo vision system based on confidence level[J]. Journal of Beijing University of Aeronautics and Astronautics, 2018, 44(8): 1764-1771. doi: 10.13700/j.bh.1001-5965.2017.0653(in Chinese)
Citation: SUN Zhe, ZHANG Yong, CHANG Qutonget al. In-depth data fusion of TOF and stereo vision system based on confidence level[J]. Journal of Beijing University of Aeronautics and Astronautics, 2018, 44(8): 1764-1771. doi: 10.13700/j.bh.1001-5965.2017.0653(in Chinese)

基于置信度的TOF与双目系统深度数据融合

doi: 10.13700/j.bh.1001-5965.2017.0653
详细信息
    作者简介:

    孙哲  女, 硕士研究生。主要研究方向:计算机视觉、三维重建

    张勇  男,博士,副教授。主要研究方向:模式识别、图像采集与处理

    常衢通  男, 硕士研究生。主要研究方向:计算机视觉、三维重建

    通讯作者:

    张勇, E-mail: 06952@buaa.edu.cn

  • 中图分类号: V219;TP391.4

In-depth data fusion of TOF and stereo vision system based on confidence level

More Information
  • 摘要:

    为了解决铁路火车等结构单一环境的三维重建问题,提出了置信度的概念,将TOF系统与双目系统的互补性特点有效结合。通过联合标定,建立起TOF系统与双目系统的坐标关系,将TOF中的点映射到左相机视角下,得出双目系统左相机视角下的TOF测量视差图,再利用图像分割以及曲面拟合对其上采样处理至双目图像的分辨率大小。根据各系统特点定义置信度,确定数据融合的不同系统权重。利用Middlebury的数据集处理结果,融合后的匹配精度较双目系统精度提高一倍以上,且视差图的分辨率提升至与双目系统相同大小。

     

  • 图 1  双目系统示意图

    Figure 1.  Schematic diagram of stereo vision system

    图 2  不同代价聚合算法的视差图对比

    Figure 2.  Comparison of disparity maps generated by different cost aggregation algorithms

    图 3  TOF与双目系统数据融合示意图

    Figure 3.  Schematic diagram of data fusion of TOF and stereo vision system

    图 4  TOF系统高分辨率视差图

    Figure 4.  Disparity map of TOF system with high resolution

    图 5  样图处理结果示例

    Figure 5.  Examples of processing results

    图 6  Teddy处理结果示例

    Figure 6.  Examples of Teddy processing results

    表  1  双目立体匹配和融合算法结果分析

    Table  1.   Analysis of stereo vision and fusion algorithm results

    %
    图像 双目立体匹配算法误匹配率 融合算法误匹配率
    Teddy 13.34 6.34
    Cones 8.56 4.63
    Tsukuba 6.52 2.39
    Venus 4.79 1.43
    下载: 导出CSV
  • [1] ZANUTTIGH P, MARIN G, DAL MUTTO C D, et al.Time-of-flight and structured light depth cameras:Technology and applications[M].Berlin:Springer, 2016:23-25.
    [2] HANSARD M, LEE S, CHOI O, et al.Time-of-flight cameras:Principles, methods and applications[M].Berlin:Springer, 2013:50-53.
    [3] TIPPETTS B, LEE D J, LILLYWHITE K, et al.Review of stereo vision algorithms and their suitability for resource-limited systems[J].Journal of Real-Time Image Processing, 2016, 11(1):5-25. doi: 10.1007/s11554-012-0313-2
    [4] YANG Q, AHUJA N, YANG R, et al.Fusion of median and bilateral filtering for range image up-sampling[J].IEEE Transactions on Image Processing, 2013, 22(12):4841-4852. doi: 10.1109/TIP.2013.2278917
    [5] GARRO V, MUTTO C D, ZANUTTIGH P, et al. A novel interpolation scheme for range data with side information[C]//Proceedings of the 2009 Conference on Visual Media Production. Piscataway, NJ: IEEE Press, 2009: 52-60.
    [6] ZHU J, WANG L, GAO J, et al.Spatial-temporal fusion for high accuracy depth maps using dynamic mrfs[J].IEEE Transactions on Pattern Analysis and Machine Intelligence, 2010, 32(5):899-909. doi: 10.1109/TPAMI.2009.68
    [7] VINEET G, JAN C, RADU H. High-resolution depth maps based on ToF-stereo fusion[C]//Proceedings of IEEE Conference on Robotics and Automation. Piscataway, NJ: IEEE Press, 2012: 4742-4749.
    [8] MUTTO C D, ZANUTTIGH P, MATTOCCIA S, et al. Locally consistent ToF and stereo data fusion[C]//Proceedings of the 12th International Conference on Computer Vision-Volume Part I. Berlin: Springer, 2012: 598-607.
    [9] FUSIELLO A, TRUCCO E, VERRI A.A compact algorithm for rectification of stereo pairs[J].Machine Vision and Applications, 2000, 12(1):16-22. doi: 10.1007/s001380050120
    [10] KAHLMANN T, INGENSAND H.Calibration and development for increased accuracy of 3D range imaging cameras[J].Journal of Applied Geodesy, 2008, 2(1):1-11. doi: 10.1515/JAG.2008.001
    [11] SCHARSTEIN D, SZELISKI R.A taxonomy and evaluation of dense twoframe stereo correspondence algorithms[J].International Journal of Computer Vision, 2002, 47(1-3):7-42. http://cn.bing.com/academic/profile?id=ff5d2a12d444cec636e7b4ee99d9fe62&encoded=0&v=paper_preview&mkt=zh-cn
    [12] PIATTI D, RINAUDO F.SR-4000 and Camcube 3.0 time of flight (ToF) cameras:Tests and comparison[J].Remote Sensing, 2012, 4(4):1069-1089. doi: 10.3390/rs4041069
    [13] MUTTO C D, ZANUTTIGH P, CORTELAZZO G.Probabilistic ToF and stereo data fusion based on mixed pixels measurement models[J].IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(11):2260-2272. doi: 10.1109/TPAMI.2015.2408361
    [14] DOLSON J, BAEK J, PLAGEMANN C, et al. Up-sampling range data in dynamic environments[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE Press, 2010: 1141-1148.
    [15] HU X, MORDOHAI P.A quantitative evaluation of confidence measures for stereo vision[J].IEEE Transactions on Pattern Analysis and Machine Intelligence, 2012, 34(11) 2121-2133. doi: 10.1109/TPAMI.2012.46
    [16] GUDMUNDSSON S, AANAES H, LARSEN R.Fusion of stereo vision and time of flight imaging for improved 3D estimation[J].International Journal Intelligent System Technologies and Applications, 2008, 5(3):425-433. http://cn.bing.com/academic/profile?id=2b63a06ea6c3faa0444039480bf93659&encoded=0&v=paper_preview&mkt=zh-cn
    [17] EVANGELIDIS G, HANSARD M, HORAUD R.Fusion of range and stereo data for high-resolution scene-modeling[J].IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(11):2178-2192. doi: 10.1109/TPAMI.2015.2400465
    [18] NAIR R, LENZEN F, MEISTER S, et al. High accuracy ToF and stereo sensor fusion at interactive rates[C]//Proceedings of European Conference on Computer Vision Workshops. Berlin: Springer, 2012: 7584.
    [19] 刘娇丽, 李素梅, 李永达, 等.基于TOF与立体匹配相融合的高分辨率深度获取[J].信息技术, 2016(12):190-193. http://d.old.wanfangdata.com.cn/Periodical/xxjs201612042

    LIU J L, LI S M, LI Y D, et al.High-resolution depth maps based on TOF-stereo fusion[J].Information Technology, 2016(12):190-193(in Chinese). http://d.old.wanfangdata.com.cn/Periodical/xxjs201612042
    [20] 张康. 基于图像深度感知中的立体匹配和深度增强算法研究[D]. 北京: 清华大学, 2015: 84-87.

    ZHANG K. Stereo matching and depth enhancement in image-based depth perception[D]. Beijing: Tsinghua University, 2015: 84-87(in Chinese).
  • 加载中
图(6) / 表(1)
计量
  • 文章访问数:  923
  • HTML全文浏览量:  137
  • PDF下载量:  605
  • 被引次数: 0
出版历程
  • 收稿日期:  2017-10-23
  • 录用日期:  2018-01-05
  • 网络出版日期:  2018-08-20

目录

    /

    返回文章
    返回
    常见问答