| Citation: | SUN Zhe, ZHANG Yong, CHANG Qutonget al. In-depth data fusion of TOF and stereo vision system based on confidence level[J]. Journal of Beijing University of Aeronautics and Astronautics, 2018, 44(8): 1764-1771. doi: 10.13700/j.bh.1001-5965.2017.0653(in Chinese)
|
To solve the problem of 3D reconstruction in textureless environment like railway, the confidence coefficient is proposed to combine TOF and stereo vision system effectively. Through the joint calibration, the coordinate relationship between TOF and stereo vision systems is established. Then by projecting the points in TOF to left camera in stereo vision system, the disparity map of TOF is obtained. After image segment and surface fitting the disparity map is up-sampled and its resolution is equal to that of stereo images. According to the confidence coefficients of different systems, the system weight values of data fusion are defined. Finally, the proposed method is evaluated with Middlebury dataset, and the results show that the accuracy has been raised twofold or more, and the resolution of disparity map is equal to that of stereo images as well.
| [1] |
ZANUTTIGH P, MARIN G, DAL MUTTO C D, et al.Time-of-flight and structured light depth cameras:Technology and applications[M].Berlin:Springer, 2016:23-25.
|
| [2] |
HANSARD M, LEE S, CHOI O, et al.Time-of-flight cameras:Principles, methods and applications[M].Berlin:Springer, 2013:50-53.
|
| [3] |
TIPPETTS B, LEE D J, LILLYWHITE K, et al.Review of stereo vision algorithms and their suitability for resource-limited systems[J].Journal of Real-Time Image Processing, 2016, 11(1):5-25. doi: 10.1007/s11554-012-0313-2
|
| [4] |
YANG Q, AHUJA N, YANG R, et al.Fusion of median and bilateral filtering for range image up-sampling[J].IEEE Transactions on Image Processing, 2013, 22(12):4841-4852. doi: 10.1109/TIP.2013.2278917
|
| [5] |
GARRO V, MUTTO C D, ZANUTTIGH P, et al. A novel interpolation scheme for range data with side information[C]//Proceedings of the 2009 Conference on Visual Media Production. Piscataway, NJ: IEEE Press, 2009: 52-60.
|
| [6] |
ZHU J, WANG L, GAO J, et al.Spatial-temporal fusion for high accuracy depth maps using dynamic mrfs[J].IEEE Transactions on Pattern Analysis and Machine Intelligence, 2010, 32(5):899-909. doi: 10.1109/TPAMI.2009.68
|
| [7] |
VINEET G, JAN C, RADU H. High-resolution depth maps based on ToF-stereo fusion[C]//Proceedings of IEEE Conference on Robotics and Automation. Piscataway, NJ: IEEE Press, 2012: 4742-4749.
|
| [8] |
MUTTO C D, ZANUTTIGH P, MATTOCCIA S, et al. Locally consistent ToF and stereo data fusion[C]//Proceedings of the 12th International Conference on Computer Vision-Volume Part I. Berlin: Springer, 2012: 598-607.
|
| [9] |
FUSIELLO A, TRUCCO E, VERRI A.A compact algorithm for rectification of stereo pairs[J].Machine Vision and Applications, 2000, 12(1):16-22. doi: 10.1007/s001380050120
|
| [10] |
KAHLMANN T, INGENSAND H.Calibration and development for increased accuracy of 3D range imaging cameras[J].Journal of Applied Geodesy, 2008, 2(1):1-11. doi: 10.1515/JAG.2008.001
|
| [11] |
SCHARSTEIN D, SZELISKI R.A taxonomy and evaluation of dense twoframe stereo correspondence algorithms[J].International Journal of Computer Vision, 2002, 47(1-3):7-42. http://cn.bing.com/academic/profile?id=ff5d2a12d444cec636e7b4ee99d9fe62&encoded=0&v=paper_preview&mkt=zh-cn
|
| [12] |
PIATTI D, RINAUDO F.SR-4000 and Camcube 3.0 time of flight (ToF) cameras:Tests and comparison[J].Remote Sensing, 2012, 4(4):1069-1089. doi: 10.3390/rs4041069
|
| [13] |
MUTTO C D, ZANUTTIGH P, CORTELAZZO G.Probabilistic ToF and stereo data fusion based on mixed pixels measurement models[J].IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(11):2260-2272. doi: 10.1109/TPAMI.2015.2408361
|
| [14] |
DOLSON J, BAEK J, PLAGEMANN C, et al. Up-sampling range data in dynamic environments[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE Press, 2010: 1141-1148.
|
| [15] |
HU X, MORDOHAI P.A quantitative evaluation of confidence measures for stereo vision[J].IEEE Transactions on Pattern Analysis and Machine Intelligence, 2012, 34(11) 2121-2133. doi: 10.1109/TPAMI.2012.46
|
| [16] |
GUDMUNDSSON S, AANAES H, LARSEN R.Fusion of stereo vision and time of flight imaging for improved 3D estimation[J].International Journal Intelligent System Technologies and Applications, 2008, 5(3):425-433. http://cn.bing.com/academic/profile?id=2b63a06ea6c3faa0444039480bf93659&encoded=0&v=paper_preview&mkt=zh-cn
|
| [17] |
EVANGELIDIS G, HANSARD M, HORAUD R.Fusion of range and stereo data for high-resolution scene-modeling[J].IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(11):2178-2192. doi: 10.1109/TPAMI.2015.2400465
|
| [18] |
NAIR R, LENZEN F, MEISTER S, et al. High accuracy ToF and stereo sensor fusion at interactive rates[C]//Proceedings of European Conference on Computer Vision Workshops. Berlin: Springer, 2012: 7584.
|
| [19] |
刘娇丽, 李素梅, 李永达, 等.基于TOF与立体匹配相融合的高分辨率深度获取[J].信息技术, 2016(12):190-193. http://d.old.wanfangdata.com.cn/Periodical/xxjs201612042
LIU J L, LI S M, LI Y D, et al.High-resolution depth maps based on TOF-stereo fusion[J].Information Technology, 2016(12):190-193(in Chinese). http://d.old.wanfangdata.com.cn/Periodical/xxjs201612042
|
| [20] |
张康. 基于图像深度感知中的立体匹配和深度增强算法研究[D]. 北京: 清华大学, 2015: 84-87.
ZHANG K. Stereo matching and depth enhancement in image-based depth perception[D]. Beijing: Tsinghua University, 2015: 84-87(in Chinese).
|
Figures(6) / Tables(1)
Copyright © Journal of Beijing University of Aeronautics and Astronautics
Address: Editorial Department of Journal of Beijing University of Aeronautics and Astronautics, 37 Xueyuan Road, Haidian District, Beijing Post Code: 100191 Email: jbuaa@buaa.edu.cn
Supported by:
Beijing Renhe Information Technology Co., Ltd.
DownLoad:
