Feature-fusion and anti-occlusion based target tracking method for satellite videos
-
摘要:
卫星视频中的目标易受到遮挡和复杂环境干扰等影响，造成对目标的运动状态估计不够准确，导致目标跟踪失败。基于此，在核相关滤波(KCF)算法的基础上设计2种算法提高目标跟踪的成功率，实现鲁棒性的目标跟踪。通过提取目标的方向梯度直方图(HOG)特征、灰度特征和高斯曲率特征表述目标的外观模型；联合响应图的峰值和平均峰值相关能量(APCE)对目标的响应图进行自适应加权融合，并将融合后的响应图峰值作为置信度对目标的模型进行自适应更新；通过使用卡尔曼滤波的方法对遮挡的目标进行位置预测，当目标遮挡结束时，对目标进行重新跟踪，解决卫星视频中目标被遮挡的问题。大量实验结果表明：所改进的相关滤波算法对卫星视频中的目标跟踪，尤其是在复杂环境、目标被遮挡及场景光照发生变化的情况下，具有良好的效果，并且在目标跟踪的精度和成功率等方面都有很大的提高，为进一步对卫星视频中的目标跟踪奠定了基础。
-
关键词:
- 卫星视频 /
- 相关滤波 /
- 自适应特征融合和模型更新 /
- 卡尔曼滤波 /
- 目标跟踪
Abstract:Targets in satellite videos are susceptible to occlusion and interference from complex environments, resulting in inaccurate estimation of the target motion state, eventually leading to target tracking failure. Therefore, based on the kernelized correlation filter (KCF) algorithm, two algorithms are designed to improve the success rate of target tracking to achieve robust target tracking. Firstly, extracting the different features (HOG features, gray features and Gaussian curvature features) of the target, then adaptively weighted fusion is carried out on the correlation response of different features of the target, whose purpose is to improve the anti-interference ability of the target against complex environments; Secondly, calculating the weight according to the maximum and average peak correlation energy (APCE) of the target response patch and using it as the confidence level to adaptively update the target model; Finally, the issue of the target being occluded in satellite videos can be resolved by employing the Kalman filter method to anticipate the position of the occluded target after the occlusion of the target is over and reappears. Many experimental results show that the improved correlation filter algorithm has sound effects on target tracking, especially in complex environments, occluded targets, and illumination variation. The success rate and precision have dramatically improved, laying the foundation for further target tracking in satellite videos.
-
图 1 基于改进的相关滤波卫星视频目标跟踪算法流程
Figure 1. Flowchart of target tracking algorithm in satellite videos based on improved correlation filter
图 2 空中目标跟踪结果的精度和成功率曲线
Figure 2. Precision and success curves of target tracking results in Plane sequences
图 3 遮挡目标跟踪结果的精度和成功率曲线
Figure 3. Precision and success curves of occluded target tracking results in Car1 sequences
图 4 运动目标跟踪结果的精度和成功率曲线
Figure 4. Precision and success curves of target tracking results in Car2 sequences
图 5 海面目标跟踪结果的精度和成功率曲线
Figure 5. Precision and success curves of target tracking results in Ship sequences
图 8 APCE值和视频帧数的关系
Figure 8. Relationship between APCE values and number of video frames
表 1 Plane视频序列中目标跟踪结果
Table 1. Target tracking results in Plane video sequences
算法 成功率/% 精度/% AUC/% 改进KCF算法 99.47 99.20 86.00 原始KCF算法 69.87 78.93 76.83 CSK算法 72.46 55.08 75.51 MOSSE算法 1.34 1.87 7.01 MEDIANFLOW算法 1.07 0.80 1.91 MIL算法 36.00 32.00 50.77 TLD算法 0.27 0.27 19.79 
下载: 导出CSV
表 2 Car1视频序列中遮挡目标跟踪结果
Table 2. Occluded target tracking results in Car1 video sequences
算法 成功率/% 精度/% AUC/% 改进KCF算法 86.59 97.73 78.41 原始KCF算法 5.23 27.50 25.80 CSK算法 3.42 8.88 17.65 MOSSE算法 0 3.19 7.59 MEDIANFLOW算法 0.45 0.68 1.48 MIL算法 8.86 29.32 29.73 TLD算法 0.23 0.23 0.23
下载: 导出CSV
表 3 Car2视频序列中目标跟踪结果
Table 3. Target tracking results in Car2 video sequences
算法 成功率/% 精度/% AUC/% 改进KCF算法 88.0 94.60 80.22 原始KCF算法 14.83 39.68 47.70 CSK算法 2.60 21.20 30.54 MOSSE算法 0 0.60 3.49 MEDIANFLOW算法 1.4 6.40 7.74 MIL算法 10.2 52.20 50.34 TLD算法 0 0.20 0.14
下载: 导出CSV
表 4 Ship视频序列中目标跟踪结果
Table 4. Target tracking results in Ship video sequences
算法 成功率/% 精度/% AUC/% 改进KCF算法 85.86 100 80.30 原始KCF算法 65.00 100 72.60 CSK算法 47.47 89.90 65.56 MOSSE算法 0 4.04 12.63 MEDIANFLOW算法 4.00 30.00 18.00 MIL算法 7.00 18.00 45.00 TLD算法 1.00 1.00 1.00
下载: 导出CSV
-
[1] 孟琭, 杨旭. 目标跟踪算法综述[J]. 自动化学报, 2019, 45(7): 1244-1260. https://www.cnki.com.cn/Article/CJFDTOTAL-MOTO201907003.htmMENG L, YANG X. A survey of target tracking algorithms[J]. Acta Automatica Sinica, 2019, 45(7): 1244-1260(in Chinese). https://www.cnki.com.cn/Article/CJFDTOTAL-MOTO201907003.htm [2] KOPSIAFTIS G, KARANTZALOS K. Vehicle detection and traffic density monitoring from very high resolution satellite video data[C]//2015 IEEE International Geoscience and Remote Sensing Symposium (IGARSS). Piscataway: IEEE Press, 2015: 1881-1884. [3] YANG T, WANG X W, YAO B W, et al. Small moving vehicle detection in a satellite video of an urban area[J]. Sensors, 2016, 16(9): 1528. doi: 10.3390/s16091528 [4] 赵学敏, 马文坡, 刘兆军. 低轨卫星面阵凝视成像技术研究[J]. 航天返回与遥感, 2007, 28(2): 10-14. doi: 10.3969/j.issn.1009-8518.2007.02.003ZHAO X M, MA W P, LIU Z J. Study on area array staring imaging technology for low-orbit satellite[J]. Spacecraft Recovery and Remote Sensing, 2007, 28(2): 10-14(in Chinese). doi: 10.3969/j.issn.1009-8518.2007.02.003 [5] BOLME D S, BEVERIDEG J R, DRAPER B A, et al. Visual target tracking using adaptive correlation filters[C]//2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2010: 2544-2550. [6] HENRIQUES J F, CASEIRO R, MARTINS P, et al. Exploiting the circulant structure of tracking-by-detection with kernels[C]//Proceedings of the European Conference on Computer Vision. Berlin: Springer, 2012: 702-715. [7] HENRIQUES J F, CASEIRO R, MARTINS P, et al. High-speed tracking with kernelized correlation filters[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(3): 583-596. doi: 10.1109/TPAMI.2014.2345390 [8] DU B, SUN Y J, CAI S, et al. Target tracking in satellite videos by fusing the kernel correlation filter and the three-frame-difference algorithm[J]. IEEE Geoscience and Remote Sensing Letters, 2018, 15(2): 168-172. doi: 10.1109/LGRS.2017.2776899 [9] XUAN S Y, LI S Y, HAN M F, et al. Tracking in satellite videos by improved correlation filters with motion estimations[J]. IEEE Transactions on Geoscience and Remote Sensing, 2020, 58(2): 1074-1086. doi: 10.1109/TGRS.2019.2943366 [10] HU Z P, YANG D Q, ZHANG K, et al. Target tracking in satellite videos based on convolutional regression network with appearance and motion features[J]. IEEE Journal of Selected Topics in Applied Earth Observation and Remote Sensing, 2020, 13: 783-793. doi: 10.1109/JSTARS.2020.2971657 [11] SHAO J, DU B, WU C, et al. PASiam: Predicting attention inspired Siamese network, for space-borne satellite video tracking[C]//Proceedings of the IEEE International Conference on Multimedia and Expo. Piscataway: IEEE Press, 2019: 1504-1509. [12] MA C Y, YU G. An improved kernel correlation filter for occlusion target tracking[C]//IEEE 4th International Conference on Image, Vision and Computing(ICIVC). Piscataway: IEEE Press, 2019: 674-678. [13] HEIMBACH M, EBADI K, WOOD S. Improving target tracking accuracy in video sequences subject to noise and occlusion impediments by combining feature tracking with Kalman filtering[C]//IEEE 52nd Asilomar Conference on Signals, Systems, and Computers. Piscataway: IEEE Press, 2018: 1499-1502. [14] 杨剑锋, 张建鹏. 基于核相关滤波的长时间目标跟踪[J]. 激光与光电子学进展, 2019, 56(2): 149-155. https://www.cnki.com.cn/Article/CJFDTOTAL-JGDJ201902018.htmYANG J F, ZHANG J P. Long-term target tracking based on nuclear correlation filtering[J]. Progress in Laser and Optoelectronics, 2019, 56(2): 149-155(in Chinese). https://www.cnki.com.cn/Article/CJFDTOTAL-JGDJ201902018.htm [15] FELZENSZWALB P F, GIRSHICK R B, MCALLESTER D, et al. Target detection with discriminatively trained part-based models[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2010, 32(9): 1627-1645. doi: 10.1109/TPAMI.2009.167 [16] HANBAY K, ALPASLAN N. Continuous rotation invariant features for gradient-based texture classification[J]. IEEE Computer Vision and Image Understanding, 2015, 132: 87-101. doi: 10.1016/j.cviu.2014.10.004 [17] 王暐, 王春平, 李军, 等. 特征融合和模型自适应更新相结合的相关滤波目标跟踪[J]. 光学精密工程, 2016, 24(8): 2059-2066. https://www.cnki.com.cn/Article/CJFDTOTAL-GXJM201608029.htmWANG W, WANG C P, LI J, et al. Correlation filter tracking based on feature fusing and model adaptive updating[J]. Optics and Precision Engineering, 2016, 24(8): 2059-2066(in Chinese). https://www.cnki.com.cn/Article/CJFDTOTAL-GXJM201608029.htm [18] 石龙伟, 邓欣, 王进, 等. 基于光流法和卡尔曼滤波的多目标跟踪[J]. 计算机应用, 2017, 37(S1): 131-136. https://www.cnki.com.cn/Article/CJFDTOTAL-JSJY2017S1029.htmSHI L W, DENG X, WANG J, et al. Multiple target tracking based on optical flow and Kalman filtering[J]. Journal of Computer Applications, 2017, 37(S1): 131-136(in Chinese). https://www.cnki.com.cn/Article/CJFDTOTAL-JSJY2017S1029.htm [19] KALAL Z, MIKOLAZYK K, MATAS J. Forward-backward error: Automatic detection of tracking failures[C]//Proceedings of the IEEE International Conference on Pattern Recognition. Piscataway: IEEE Press, 2010: 2756-2759. [20] BABENKO B, YANG M H, BELONGIE S. Robust target tracking with online multiple instance learning[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011, 33(8): 1619-1632. doi: 10.1109/TPAMI.2010.226 [21] KALAL Z, MIKOLAZYK K, MATAS J. Tracking-learning-detection[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011, 34(7): 1409-1422. [22] WU Y, LIM J, YANG M H. Online target tracking: A benchmark[C]//Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2013, 2411-2418. -
下载:
点击查看大图
计量
- 文章访问数: 407
- HTML全文浏览量: 121
- PDF下载量: 33
- 被引次数: 0
