-
摘要:
行人重识别是刑侦案件中重要的侦查手段,而跨域是行人重识别的主要挑战之一,也是制约其实际应用的瓶颈问题。在带标签的源域和无标签的目标域学习跨域行人局部语义不变性特征模型。首先,在源域上通过只含有行人标识无部件标签的监督学习方式学习行人的各部件特征,并在源域和目标域上采用无监督学习方式对齐行人部件。然后,基于对齐后的行人全局与局部特征,引入特征模板池存储对齐后的目标域全局和局部特征,并设计了跨域不变性损失函数进行特征不变性约束,提高行人重识别的跨域适应能力。最后,在Market-1501、DukeMTMC-reID和MSMT17数据集之间开展了跨域行人重识别验证实验,实验结果表明,所提方法在跨域行人重识别上取得了显著的性能提升。
Abstract:Cross-domain is one of the main challenges of person re-identification which is an important investigative method in criminal investigation cases, and that restricts the practical application of re-identification. In this paper, cross-domain invariance feature model based on pedestrian partial semantics is learned from the labeled source domain and the unlabeled target domain. First, features of person parts are learned by supervised learning with only pedestrian signs without labels of parts, and pedestrian parts are aligned by unsupervised learning in the source and target domains. Second, based on the aligned global and local features, feature template pooling is introduced to store the aligned global and local partial features of the target domain, and cross-domain invariance loss function is designed for feature invariance constraints to improve the cross-domain adaptability of person re-identification. Finally, verification experiments of cross-domain person re-identification are conducted on the Market-1501, DukeMTMC-reID and MSMT17 datasets. The experiment results show that the proposed method achieves significant performance improvements in cross-domain person re-identification.
-
表 1 在Market-1501和DukeMTMC-reID数据集上的跨域行人重识别消融实验
Table 1. Ablation experiment of cross-domain person re-identification on Market-1501 and DukeMTMC-reID datasets
行人重识别跨域方法 DukeMTMC-reID→Market-1501 Market-1501→DukeMTMC-reID R-1/% R-5/% R-10/% R-20/% mAP/% R-1/% R-5/% R-10/% R-20/% mAP/% GAP 45.1 62.4 68.9 75.2 18.8 29.6 43.9 50.2 57.0 15.1 PAF 45.9 63.5 69.9 75.7 19.7 30.0 45.4 51.8 58.1 15.5 GAP+PAF 46.6 63.9 70.6 76.3 20.4 30.3 46.7 52.4 58.8 16.0 Inv+GAP 65.7 81.5 86.0 89.9 31.9 52.7 64.7 69.7 73.4 27.0 Inv+PAF 75.7 89.4 92.0 94.3 43.5 64.1 76.5 80.3 83.8 41.0 Inv+GAP+PAF 77.6 88.7 92.0 95.2 45.0 65.5 77.6 81.1 84.5 42.8 行人重识别单域方法 DukeMTMC-reID→Market-1501 Market-1501→DukeMTMC-reID R-1/% R-5/% R-10/% R-20/% mAP/% R-1/% R-5/% R-10/% R-20/% mAP/% Inv+GAP+PAF 74.7 85.5 89.1 91.8 54.1 88.2 95.6 97.4 98.3 67.0 表 2 特征模板池与Mini-batch的比较
Table 2. Comparison of feature template pooling memory and Mini-batch
方法 DukeMTMC-reID→Market-1501 R-1/% R-5/% R-10/% R-20/% mAP/% Mini-batch 66.4 83.5 88.3 90.6 36.2 Mini-batch+Memory 77.6 88.7 92.0 95.2 45.0 表 3 在Market-1501和DukeMTMC-reID到MSMT17跨域数据集上与当前先进方法的实验比较
Table 3. Experimental comparison with other advanced methods from Market-1501 and DukeMTMC-reID to MSMT17 cross-domain datasets
表 4 在Market-1501和DukeMTMC-reID跨域数据集上与当前先进方法的实验比较
Table 4. Experimental comparison with other advanced methods on Market-1501 and DukeMTMC-reID cross-domain datasets
行人重识别跨域方法 DukeMTMC-reID →Market-1501 Market-1501→DukeMTMC-reID R-1/% R-5/% R-10/% mAP/% R-1/% R-5/% R-10/% mAP/% MMFA[33] 56.7 75.0 81.8 27.4 45.3 59.8 66.3 24.7 SPGAN+LM[29] 57.7 75.8 82.4 26.7 46.4 62.3 68.0 26.2 TJ-AIDL[32] 58.2 74.8 81.1 26.5 44.3 59.6 65.0 23.0 CamStyle[45] 58.8 78.2 84.3 27.4 48.4 62.5 68.9 25.1 HHL[46] 62.2 78.8 84.0 31.4 46.9 61.0 66.7 27.2 ECN[44] 75.1 87.6 91.6 43.0 63.3 75.8 80.4 40.4 本文 77.6 88.7 92.0 45.0 65.5 77.6 81.1 42.8 -
[1] ZHENG L, YANG Y, HAUPTMANN A G.Person re-identification: Past, present and future[EB/OL].(2016-10-10)[2020-03-01].https://arxiv.org/abs/1610.02984. [2] 田萱, 王亮, 丁琪.基于深度学习的图像语义分割方法综述[J].软件学报, 2019, 30(2):440-468. http://www.cnki.com.cn/Article/CJFDTotal-RJXB201902014.htmTIAN X, WANG L, DING Q.Review of image semantic segmentation based on deep learning[J].Journal of Software, 2019, 30(2):440-468(in Chinese). http://www.cnki.com.cn/Article/CJFDTotal-RJXB201902014.htm [3] HAN C C, YE J C, ZHONG Y S, et al.Re-ID driven localization refinement for person search[C]//IEEE International Conference on Computer Vision.Piscataway: IEEE Press, 2019: 9814-9823. https://blog.csdn.net/qq_36269513/article/details/102215503 [4] VARIOR R R, HALOI M, WANG G.Gated Siamese convolutional neural network architecture for human re-identiflcation[C]//IEEE Conference on Computer Vision and Pattern Recognition.Piscataway: IEEE Press, 2016: 791-808. https://www.zhangqiaokeyan.com/open-access_resources_thesis/0100091157726.html [5] CHENG D, GONG Y H, ZHOU S P, et al.Person re-identiflcation by multi-channel parts-based CNN with improved triplet loss function[C]//IEEE Conference on Computer Vision and Pattern Recognition.Piscataway: IEEE Press, 2016: 1335-1344. https://blog.csdn.net/yuanchheneducn/article/details/53116657 [6] ZHENG Z D, ZHENG L, YANG Y.Unlabeled samples generated by GAN improve the person re-identification baseline in vitro[C]//IEEE International Conference on Computer Vision.Piscataway: IEEE Press, 2017: 3754-3762. https://blog.csdn.net/VacantField/article/details/80238962 [7] WEN Y, ZHANG K, LI Z F, et al.A discriminative feature learning approach for deep face recognition[C]//IEEE Conference on Computer Vision and Pattern Recognition.Piscataway: IEEE Press, 2016: 499-515. doi: 10.1007%2F978-3-319-46478-7_31 [8] 蔡轶珩, 王雪艳, 胡绍斌, 等.基于多源图像弱监督学习的3D人体姿态估计[J].北京航空航天大学学报, 2019, 45(12):2375-2384. http://html.rhhz.net/BJHKHTDXXBZRB/20191204.htmCAI Y H, WANG X Y, HU S B, et al.Three-dimensional human pose estimation based on multi-source image weakly-supervised learning[J].Journal of Beijing University of Aeronautics and Astronautics, 2019, 45(12):2375-2384(in Chinese). http://html.rhhz.net/BJHKHTDXXBZRB/20191204.htm [9] CAO Z, SIMON T, WEI S E, et al.Realtime multi-person 2D pose estimation using part affinity fields[C]//IEEE Conference on Computer Vision and Pattern Recognition.Piscataway: IEEE Press, 2017: 7291-7299. https://blog.csdn.net/tangbin2009/article/details/81122788 [10] WEI S E, RAMAKRISHNA V, KANADE T, et al.Convolutional pose machines[C]//IEEE Conference on Computer Vision and Pattern Recognition.Piscataway: IEEE Press, 2016: 4724-4732. https://blog.csdn.net/zziahgf/article/details/79643752 [11] 郭琳, 秦世引.遥感图像飞机目标高效搜检深度学习优化算法[J].北京航空航天大学学报, 2019, 45(1):159-173. http://html.rhhz.net/BJHKHTDXXBZRB/20190119.htmGUO L, QIN S Y.Deep learning and optimization algorithm for high efficient searching and detection of aircraft targets in remote sensing images[J].Journal of Beijing University of Aeronautics and Astronautics, 2019, 45(1):159-173(in Chinese). http://html.rhhz.net/BJHKHTDXXBZRB/20190119.htm [12] ZHENG L, HUANG Y, LU H, et al.Pose invariant embedding for deep person re-identification[J].IEEE Transactions on Image Processing, 2019, 28(9):4500-4509. doi: 10.1109/TIP.2019.2910414 [13] SU C, LI J, ZHANG S, et al.Posedriven deep convolutional model for person re-identification[C]//IEEE International Conference on Computer Vision.Piscataway: IEEE Press, 2017: 3960-3969. https://www.cnblogs.com/laokanblog/p/10324137.html [14] WEI L, ZHANG S, YAO H, et al.GLAD: Global-local-alignment descriptor for pedestrian retrieval[C]//ACM International Conference on Multimedia.New York: ACM, 2017: 420-428. [15] YAO H, ZHANG S, HONG R, et al.Deep representation learning with part loss for person re-identification[J].IEEE Transactions on Image Processing, 2019, 28(6):2860-2871. doi: 10.1109/TIP.2019.2891888 [16] SUN Y F, XU Q, LI Y L, et al.Perceive where to focus: Learning visibility-aware part-level features for partial person re-identification[C]//IEEE Conference on Computer Vision and Pattern Recognition.Piscataway: IEEE Press, 2019: 393-402. http://www.doc88.com/p-81261135181349.html [17] SUN Y F, ZHENG L, YANG Y, et al.Beyond part models: Person retrieval with refined part pooling[C]//European Conference on Computer Vision.Berlin: Springer, 2019: 480-496. doi: 10.1007/978-3-030-01225-0_30 [18] LIU X, ZHAO H, TIAN M, et al.HydraPlus-Net: Attentive deep features for pedestrian analysis[C]//IEEE International Conference on Computer Vision.Piscataway: IEEE Press, 2017: 350-359. https://blog.csdn.net/weixin_44138807/article/details/104022543 [19] ZHAO L, LI X, WANG J, et al.Deeply-learned part-aligned representations for person re-identification[C]//IEEE International Conference on Computer Vision.Piscataway: IEEE Press, 2017: 3219-3228. https://xueshu.baidu.com/usercenter/paper/show?paperid=77fdc3eed9d528f33d2dbaf25e2ae5fd&site=xueshu_se [20] CHEN T L, DING S J, XIE J Y, et al.ABD-Net: Attentive but diverse person re-identification[C]//IEEE International Conference on Computer Vision.Piscataway: IEEE Press, 2019: 8351-8361. https://blog.csdn.net/qq_38204302/article/details/103744942 [21] LI W, ZHU X T, GONG S G, et al.Harmonious attention network for person re-identification[C]//IEEE Conference on Computer Vision and Pattern Recognition.Piscataway: IEEE Press, 2018: 2285-2294. https://www.zhangqiaokeyan.com/academic-conference-foreign_ieeecvf-cference-computer-visi-and-patt_thesis/020512457702.html [22] SUN X X, ZHENG L.Dissecting person re-identification from the viewpoint of viewpoint[C]//IEEE Conference on Computer Vision and Pattern Recognition.Piscataway: IEEE Press, 2019: 608-617. https://www.jianshu.com/p/1dd9676e314a [23] ZHENG Z D, YANG X D, YU Z D, et al.Joint discriminative and generative learning for person re-identification[C]//IEEE Conference on Computer Vision and Pattern Recognition.Piscataway: IEEE Press, 2019: 2138-2147. https://www.cnblogs.com/king-lps/p/12583712.html [24] LUO Y W, ZHENG L, GUAN T, et al.Taking a closer look at domain shift: Category-level adversaries for semantics consistent domain adaptation[C]//IEEE Conference on Computer Vision and Pattern Recognition.Piscataway: IEEE Press, 2019: 2507-2516. [25] HE K M, ZHANG X Y, REN S Q, et al.Deep residual learning for image recognition[C]//IEEE Conference on Computer Vision and Pattern Recognition.Piscataway: IEEE Press, 2016: 770-778. [26] DONG X Y, YANG Y.Searching for a robust neural architecture in four GPU hours[C]//IEEE Conference on Computer Vision and Pattern Recognition.Piscataway: IEEE Press, 2019: 1761-1770. https://www.jianshu.com/p/a9e7016fc4b3 [27] FAN H H, ZHENG L, YAN C G, et al.Unsupervised person re-identification:Clustering and finetuning[J].ACM Transactions on Multimedia Computing, Communications, and Applications, 2018, 14(4):1-18. https://blog.csdn.net/sinat_42239797/article/details/104903077 [28] YU H X, WU A C, ZHENG W S.Crossview asymmetric metric learning for unsupervised person re-identification[C]//IEEE International Conference on Computer Vision.Piscataway: IEEE Press, 2017: 994-1002. [29] DENG W J, ZHENG L, YE Q X, et al.Image-image domain adaptation with preserved self-similarity and domain-dissimilarity for person re-identification[C]//IEEE Conference on Computer Vision and Pattern Recognition.Piscataway: IEEE Press, 2018: 994-1003. [30] WEI L H, ZHANG S J, GAO W, et al.Person transfer GAN to bridge domain gap for person re-identification[C]//IEEE Conference on Computer Vision and Pattern Recognition.Piscataway: IEEE Press, 2018: 79-88. [31] BAK S, CARR P, LALONDE J F.Domain adaptation through synthesis for unsupervised person re-identification[C]//European Conference on Computer Vision.Berlin: Springer, 2018: 189-205. https://blog.csdn.net/hyk_1996/article/details/87873695 [32] WANG J Y, ZHU X T, GONG S G, et al.Transferable joint attribute-identity deep learning for unsupervised person re-identification[C]//IEEE Conference on Computer Vision and Pattern Recognition.Piscataway: IEEE Press, 2018: 2275-2284. https://blog.csdn.net/weixin_39417323/article/details/103493695 [33] LIN S, LI H L, LI C T, et al.Multi-task mid-level feature alignment network for unsupervised cross-dataset person re-identification[C]//British Machine Vision Conference.Berlin: Springer, 2018: 1-13. [34] ZHONG Z, ZHENG L, ZHENG Z D, et al.Camera style adaptation for person re-identification[C]//IEEE Conference on Computer Vision and Pattern Recognition.Piscataway: IEEE Press, 2018: 5157-5166. https://www.zhangqiaokeyan.com/academic-conference-foreign_ieeecvf-cference-computer-visi-and-patt_thesis/020512457400.html [35] YU H X, ZHENG W S, WU A C, et al.Unsupervised person re-identification by soft multilabel learning[C]//IEEE Conference on Computer Vision and Pattern Recognition.Piscataway: IEEE Press, 2019: 2148-2157. https://www.cnblogs.com/Thinker-pcw/p/10807681.html [36] CHEN Y B, ZHU X T, GONG X T, et al.Instance-guided context rendering for cross-domain person re-identification[C]//IEEE International Conference on Computer Vision.Piscataway: IEEE Press, 2019: 232-242. https://blog.csdn.net/weixin_41792787/article/details/103973474 [37] HUANG H J, YANG W J, CHEN X T, et al.EANet: Enhancing alignment for cross domain person re-identification[EB/OL].(2018-12-29)[2020-03-01].https://arxiv.org/abs/1812.11369. [38] DENG J, DONG W, SOCHER R, et al.ImageNet: A large-scale hierarchical image database[C]//IEEE Conference on Computer Vision and Pattern Recognition.Piscataway: IEEE Press, 2009: 248-255. https://www.zhangqiaokeyan.com/open-access_resources_thesis/0100058910627.html [39] ZHANG X, LUO H, FAN X, et al.AlignedReID: Surpassing human-level performance in person re-identification[EB/OL].(2017-11-22)[2020-03-01].https://arxiv.org/abs/1711.08184. [40] CHOI Y, CHOI M, KIM M, et al.StarGAN: Unified generative adversarial networks for multi-domain image-to-image translation[C]//IEEE Conference on Computer Vision and Pattern Recognition.Piscataway: IEEE Press, 2018: 8789-8797. https://www.zhangqiaokeyan.com/academic-conference-foreign_ieeecvf-cference-computer-visi-and-patt_thesis/020512457876.html [41] ZHENG L, SHEN L Y, TIAN L, et al.Scalable person re-identification: A benchmark[C]//IEEE International Conference on Computer Vision.Piscataway: IEEE Press, 2015: 1116-1124. https://ieeexplore.ieee.org/document/7410490?arnumber=7410490 [42] RISTANI E, SOLERA F, ZOU R, et al.Performance measures and a data set for multi-target, multi-camera tracking[C]//European Conference on Computer Vision.Berlin: Springer, 2016: 17-35. doi: 10.1007%2F978-3-319-48881-3_2 [43] RUSSAKOVSKY O, DENG J, SU H, et al.ImageNet large scale visual recognition challenge[J].International Journal of Computer Vision, 2015, 115:211-252. doi: 10.1007/s11263-015-0816-y [44] ZHONG Z, ZHENG L, LUO Z M, et al.Invariance matters: Exemplar memory for domain adaptive person re-identification[C]//IEEE Conference on Computer Vision and Pattern Recognition.Piscataway: IEEE Press, 2019: 598-607. https://blog.csdn.net/weixin_39417323/article/details/89152925 [45] ZHONG Z, ZHENG L, ZHENG Z D, et al.CamStyle:A novel data augmentation method for person re-identification[J].IEEE Transactions on Image Processing, 2019, 28(3):1176-1190. http://cn.bing.com/academic/profile?id=c40d0f9b73d8fcb53c5c2ad654b829bd&encoded=0&v=paper_preview&mkt=zh-cn [46] ZHONG Z, ZHENG L, LI S Z, et al.Generalizing a person retrieval model hetero-and homogeneously[C]//European Conference on Computer Vision.Berlin: Springer, 2018: 172-188. doi: 10.1007/978-3-030-01261-8_11