基于编解码双路卷积神经网络的视觉自定位方法

doi:10.13700/j.bh.1001-5965.2019.0046

北京航空航天大学学报 ›› 2019, Vol. 45 ›› Issue (10): 1965-1972.doi: 10.13700/j.bh.1001-5965.2019.0046

基于编解码双路卷积神经网络的视觉自定位方法

贾瑞明¹, 刘圣杰¹, 李锦涛², 王赟豪², 潘海侠²

1. 北方工业大学信息学院, 北京 100144;
2. 北京航空航天大学软件学院, 北京 100083

收稿日期:2019-02-13 出版日期:2019-10-20 发布日期:2019-10-31
通讯作者: 贾瑞明 E-mail:jiaruiming@ncut.edu.cn
作者简介:贾瑞明男,博士,助理研究员。主要研究方向:计算机视觉、深度学习、模式识别;刘圣杰男,硕士研究生。主要研究方向:计算机视觉、深度学习。
基金资助:
国家重点研发计划（2017YFB0802300）；北京市教委面上项目（KM201510009005）；北方工业大学学生科技活动项目（110051360007）

A visual localization method based on encoder-decoder dual-stream CNN

JIA Ruiming¹, LIU Shengjie¹, LI Jintao², WANG Yunhao², PAN Haixia²

1. School of Information Science and Technology, North China University of Technology, Beijing 100144, China;
2. College of Software, Beihang University, Beijing 100083, China

Received:2019-02-13 Online:2019-10-20 Published:2019-10-31
Supported by:
National Key R & D Program of China (2017YFB0802300); The General Program of Beijing Municipal Education Commission (KM201510009005); Science and Technology Activities for Students of NCUT (110051360007)

摘要/Abstract

摘要： 为了从单张RGB图像估计出相机的位姿信息，提出了一种深度编解码双路卷积神经网络（CNN），提升了视觉自定位的精度。首先，使用编码器从输入图像中提取高维特征；然后，使用解码器提升特征的空间分辨率；最后，通过多尺度位姿预测器输出位姿参数。由于位置和姿态的特性不同，网络从解码器开始采用双路结构，对位置和姿态分别进行处理，并且在编解码之间增加跳跃连接以保持空间信息。实验结果表明：所提网络的精度与目前同类型算法相比有明显提升，其中相机姿态角度精度有较大提升。

关键词: 视觉自定位, 编解码结构, 卷积神经网络(CNN), 跳跃连接, 双路网络

Abstract: In order to calculate the camera pose from a single RGB image, a deep encoder-decoder dual-stream convolutional neural network (CNN) is proposed, which can improve the accuracy of visual localization. The network first uses an encoder to extract advanced features from input images. Second, the spacialresolution is enhancedby a pose decoder.Finally, a multi-scale estimator is used to output pose parameters. Becauseof the differentperformance of position and orientation, the network adopts a dual-stream structure from the decoder to process the position and orientationseparately. To restore the spatial information, several skip connections are added to encoder-decoder architecture. The experimental results show that the accuracy of the network is obviously improved compared with the congener state-of-the-art algorithms, and the orientation accuracy of camera pose is improved dramatically.

Key words: visual localization, encoder-decoder architecture, convolutional neural network (CNN), skip connection, dual-stream network

中图分类号:

贾瑞明, 刘圣杰, 李锦涛, 王赟豪, 潘海侠. 基于编解码双路卷积神经网络的视觉自定位方法[J]. 北京航空航天大学学报, 2019, 45(10): 1965-1972.

JIA Ruiming, LIU Shengjie, LI Jintao, WANG Yunhao, PAN Haixia. A visual localization method based on encoder-decoder dual-stream CNN[J]. JOURNAL OF BEIJING UNIVERSITY OF AERONAUTICS AND A, 2019, 45(10): 1965-1972.

参考文献

[1] CHEN D M,BAATZ G,KOSER K,et al.City-scale landmark identification on mobile devices[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Piscataway,NJ:IEEE Press,2011:12258110.
[2] TORII A,SIVIC J,PAJDLA T,et al.Visual place recognition with repetitive structures[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Piscataway,NJ:IEEE Press,2013:883-890.
[3] SCHINDLER G,BROWN M,SZELISKI R.City-scale location recognition[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Piscataway,NJ:IEEE Press,2007:1-7.
[4] ARTH C,PIRCHHEIM C,VENTURA J,et al.Instant outdoor localization and SLAM initialization from 2.5 D maps[J].IEEE Transactions on Visualization and Computer Graphics,2015,21(11):1309-1318.
[5] POGLITSCH C,ARTH C,SCHMALSTIEG D,et al.A particle filter approach to outdoor localization using image-based rendering[C]//IEEE International Symposium on Mixed and Augmented Reality(ISMAR).Piscataway,NJ:IEEE Press,2015:132-135.
[6] SATTLER T,LEIBE B,KOBBELT L.Improving image-based localization by active correspondence search[C]//Proceedings of European Conference on Computer Vision.Berlin:Springer,2012:752-765.
[7] LI Y,SNAVELY N,HUTTENLOCHER D,et al.Worldwide pose estimation using 3D point clouds[C]//Proceedings of European Conference on Computer Vision.Berlin:Springer,2012:15-29.
[8] CHOUDHARY S,NARAYANAN P J.Visibility probability structure from SFM datasets and applications[C]//Proceedings of European Conference on Computer Vision.Berlin:Springer,2012:130-143.
[9] SVARM L,ENQVIST O,OSKARSSON M,et al.Accurate localization and pose estimation for large 3D models[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Piscataway,NJ:IEEE Press,2014:532-539.
[10] SHOTTON J,GLOCKER B,ZACH C,et al.Scene coordinate regression forests for camera relocalization in RGB-D images[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Piscataway,NJ:IEEE Press,2013:2930-2937.
[11] KENDALL A,GRIMES M,CIPOLLA R.PoseNet:A convolutional network for real-time 6-DOF camera relocalization[C]//Proceedings of IEEE International Conference on Computer Vision.Piscataway,NJ:IEEE Press,2015:2938-2946.
[12] WALCH F,HAZIRBAS C,LEAL-TAIXÉ L,et al.Image-based localization with spatial LSTMs[EB/OL].(2016-11-23)[2018-12-25].https://arxiv.org/pdf/1611.07890v1.
[13] CLARK R,WANG S,MARKHAM A,et al.VidLoc:A deep spatio-temporal model for 6-DOF video-clip relocalization[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Piscataway,NJ:IEEE Press,2017:6856-6864.
[14] KENDALL A,CIPOLLA R.Modelling uncertainty in deep learning forcamera relocalization[C]//Proceedings of IEEE International Conference on Robotics and Automation (ICRA).Piscataway,NJ:IEEE Press,2016:4762-4769.
[15] MELEKHOV I,YLIOINAS J,KANNALA J,et al.Image-based localization using hourglass networks[EB/OL].(2017-08-24)[2018-12-25].https://arxiv.org/abs/1703.07971.
[16] LI R,LIU Q,GUI J,et al.Indoor relocalization in challenging environments with dual-stream convolutional neural networks[J].IEEE Transactions on Automation Science and Engineering,2018,15(2):651-662.
[17] RADWAN N,VALADA A,BURGARD W.Vlocnet++:Deep multitask learning for semantic visual localization and odometry[EB/OL].(2016-10-11)[2018-12-25].https://arxiv.org/abs/1804.08366.
[18] BRAHMBHATT S,GU J,KIM K,et al.Geometry-aware learning of maps for camera localization[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Piscataway,NJ:IEEE Press,2018:2616-2625.
[19] LI X,YLIOINAS J,KANNALA J.Full-frame scene coordinate regression for image-based localization[EB/OL].(2018-01-25)[2018-12-25].https://arxiv.org/abs/1802.03237.
[20] LASKAR Z,MELEKHOV I,KALIA S,et al.Camera relocalization by computing pairwise relative poses using convolutional neural network[C]//Proceedings of IEEE International Conference on Computer Vision.Piscataway,NJ:IEEE Press,2017:929-938.
[21] KENDALL A,GAL Y,CIPOLLA R.Multi-task learning using uncertainty to weigh losses for scene geometry and semantics[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Piscataway,NJ:IEEE Press,2018:7482-7491.
[22] SZEGEDY C,IOFFE S,VANHOUCKE V,et al.Inception-v4,inception-resnet and the impact of residual connections on learning[C]//Thirty-First AAAI Conference on Artificial Intelligence,2017:4-12.
[23] KENDALL A,GAL Y.What uncertainties do we need in Bayesian deep learning for computer vision?[EB/OL].(2017-10-05)[2018-12-26].https://arxiv.org/abs/1703.04977.
[24] IZADI S,KIM D,HILLIGES O,et al.KinectFusion:Real-time 3D reconstruction and interaction using a moving depth camera[C]//Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology.New York:ACM,2011:559-568.
[25] HE K M,ROSS G,PIOTR D.Rethinking imagenet pre-training[EB/OL].(2015-11-21)[2018-12-25].https://arxiv.org/abs/1811.08883.
[26] WU Y X,HE K M.Group normalization[C]//Proceedings of the European Conference on Computer Vision (ECCV),2018,3-19.

基于编解码双路卷积神经网络的视觉自定位方法

A visual localization method based on encoder-decoder dual-stream CNN

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 8

编辑推荐

Metrics

本文评价

[1]	孙宇航, 曾国奇, 刘春辉, 张多纳. 基于深度学习的无人机数据链信噪比估计算法[J]. 北京航空航天大学学报, 2019, 45(9): 1855-1863.
[2]	张新峰, 郭宇桐, 蔡轶珩, 孙萌. 基于DCNN和全连接CRF的舌图像分割算法[J]. 北京航空航天大学学报, 2019, 45(12): 2364-2374.
[3]	翟宇轩, 刘怡桑, 徐艺文, 陈忠辉, 房颖, 赵铁松. 基于HTTP自适应流媒体传输的3D视频质量评价[J]. 北京航空航天大学学报, 2019, 45(12): 2456-2462.
[4]	张梦琴, 孟权令, 张维刚. 基于深度视觉语义嵌入的视频缩略图推荐[J]. 北京航空航天大学学报, 2019, 45(12): 2479-2486.
[5]	佟骏超, 吴熙林, 丁丹丹. 时空域上下文学习的视频多帧质量增强方法[J]. 北京航空航天大学学报, 2019, 45(12): 2506-2513.
[6]	王俊, 郑彤, 雷鹏, 张原, 樵明朗. 基于卷积神经网络的手势动作雷达识别方法[J]. 北京航空航天大学学报, 2018, 44(6): 1117-1123.
[7]	韩京冶, 许福, 陈志泊, 刘辉. 一种基于深度学习的交互式电话号码识别方法[J]. 北京航空航天大学学报, 2018, 44(5): 1074-1080.
[8]	黄洁, 姜志国, 张浩鹏, 姚远. 基于卷积神经网络的遥感图像舰船目标检测[J]. 北京航空航天大学学报, 2017, 43(9): 1841-1848.