| Citation: | ZHAO S Y，A Q，GAO Y. Cross-modality nearest neighbor loss for visible-infrared person re-identification[J]. Journal of Beijing University of Aeronautics and Astronautics，2024，50（2）：433-441 （in Chinese） doi: 10.13700/j.bh.1001-5965.2022.0422
|
The goal of the visual-infrared person re-identification task is to search the image of a specific person in a given modality in the image set taken by other cameras in different modality to find out the corresponding image of the same person. Due to the different imaging methods, there are obvious modal differences between images of different modalities. Therefore, from the perspective of metric learning, the loss function is improved to obtain more discriminative information. The cohesiveness of image features is analyzed theoretically, and a re-recognition method based on cohesiveness analysis and cross-modal nearest neighbor loss function is proposed to strengthen the cohesiveness of different modal samples. The similarity measurement problem of cross-modal hard samples is transformed into the similarity measurement of cross-modal nearest neighbor sample pairs and the same modality sample pairs, which makes the optimization of modal cohesion of the network more efficient and stable. The proposed method is experimentally verified on the baseline networks of global feature representation and partial feature representation. Compared with the baseline method, the proposed method can improve the average accuracy of the visual and infrared person re-identification by up to 8.44%. The universality of the proposed method in different network architectures is proved. Moreover, at the cost of less model complexity and less computation, the reliable visual-infrared person re-identification results are achieved.
| [1] |
GONG S G, XIANG T. Person re-identification[M]. GONG S G, XIANG T. Visual analysis of behaviour. Berlin: Springer, 2011: 301-313.
|
| [2] |
ZHENG L, YANG Y, HAUPTMANN A G. Person re-identification: Past, present and future[EB/OL]. (2016-10-10)[2020-03-15]. https://arxiv.org/abs/1610.02984.pdf.
|
| [3] |
GHEISSARI N, SEBASTIAN T B, HARTLEY R. Person reidentification using spatiotemporal appearance[C]//Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2006: 1528-1535.
|
| [4] |
KARANAM S, LI Y, RADKE R J. Person re-identification with discriminatively trained viewpoint invariant dictionaries[C]//Proceedings of the IEEE International Conference on Computer Vision. Piscataway: IEEE Press, 2016: 4516-4524.
|
| [5] |
BAK S, ZAIDENBERG S, BOULAY B, et al. Improving person re-identification by viewpoint cues[C]//Proceedings of the IEEE International Conference on Advanced Video and Signal Based Surveillance. Piscataway: IEEE Press, 2014: 175-180.
|
| [6] |
HUANG Y K, ZHA Z J, FU X Y, et al. Illumination-invariant person re-identification[C]//Proceedings of the 27th ACM International Conference on Multimedia. New York: ACM, 2019: 365-373.
|
| [7] |
CHO Y J, YOON K J. Improving person re-identification via pose-aware multi-shot matching[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2016: 1354-1362.
|
| [8] |
ZHAO H Y, TIAN M Q, SUN S Y, et al. Spindle Net: Person re-identification with human body region guided feature decomposition and fusion[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2017: 907-915.
|
| [9] |
WANG Z X, WANG Z, ZHENG Y Q, et al. Learning to reduce dual-level discrepancy for infrared-visible person re-identification[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2020: 618-626.
|
| [10] |
HAO Y, WANG N N, LI J E, et al. HSME: Hypersphere manifold embedding for visible thermal person re-identification[C]//Proceedings of the AAAI Conference on Artificial Intelligence. Palo Alto: AAAI Press, 2019, 33(1): 8385-8392. doi: 10.1609/aaai.v33i01.33018385
|
| [11] |
YE M, LAN X Y, LENG Q M. Modality-aware collaborative learning for visible thermal person re-identification[C]//Proceedings of the 27th ACM International Conference on Multimedia. New York: ACM, 2019: 347-355.
|
| [12] |
HAO Y, WANG N N, GAO X B, et al. Dual-alignment feature embedding for cross-modality person re-identification[C]//Proceedings of the 27th ACM International Conference on Multimedia. New York: ACM, 2019: 57-65.
|
| [13] |
WANG G A, ZHANG T Z, CHENG J, et al. RGB-infrared cross-modality person re-identification via joint pixel and feature alignment[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. Piscataway: IEEE Press, 2020: 3622-3631.
|
| [14] |
JIA M X, ZHAI Y P, LU S J, et al. A similarity inference metric for RGB-infrared cross-modality person re-identification[EB/OL]. (2020-07-03)[2022-03-18].https://arxiv.org/abs/2007.01504.pdf.
|
| [15] |
WEI X, LI D G, HONG X P, et al. Co-attentive lifting for infrared-visible person re-identification[C]//Proceedings of the 28th ACM International Conference on Multimedia. New York: ACM, 2020: 1028-1037.
|
| [16] |
WANG G A, ZHANG T Z, YANG Y, et al. Cross-modality paired-images generation for RGB-infrared person re-identification[C]//Proceedings of the AAAI Conference on Artificial Intelligence. Palo Alto: AAAI Press, 2020, 34(7): 12144-12151. doi: 10.1609/aaai.v34i07.6894
|
| [17] |
LU Y, WU Y, LIU B, et al. Cross-modality person re-identification with shared-specific feature transfer[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2020: 13376-13386.
|
| [18] |
YE M, SHEN J B, CRANDALL D, et al. Dynamic dual-attentive aggregation learning for visible-infrared person re-identification[C]//Proceedings of the European Conference on Computer Vision. Berlin: Springer, 2020: 229-247.
|
| [19] |
PU N, CHEN W, LIU Y, et al. Dual Gaussian-based variational subspace disentanglement for visible-infrared person re-identification[C]//Proceedings of the 28th ACM International Conference on Multimedia. New York: ACM, 2020: 2149-2158.
|
| [20] |
LI D G, WEI X, HONG X P, et al. Infrared-visible cross-modal person re-identification with an X modality[C]//Proceedings of the AAAI Conference on Artificial Intelligence. Palo Alto: AAAI Press, 2020, 34(4): 4610-4617. doi: 10.1609/aaai.v34i04.5891
|
| [21] |
PARK H, LEE S, LEE J, et al. Learning by aligning: Visible-infrared person re-identification using cross-modal correspondences[EB/OL]. (2021-08-17)[2022-03-20]. https://arxiv.org/abs/2108.07422.
|
| [22] |
MIAO Z, LIU H, SHI W, et al. Modality-aware style adaptation for RGB-infrared person re-identification[C]//Proceedings of the International Joint Conference on Artificial Intelligence. New York: ACM, 2021: 916-922.
|
| [23] |
LING Y G, LUO Z M, LIN Y J, et al. A multi-constraint similarity learning with adaptive weighting for visible-thermal person re-identification[C]//Proceedings of the International Joint Conference on Artificial Intelligence. New York: ACM, 2021: 845-851.
|
| [24] |
ZHAO Z W, LIU B, CHU Q, et al. Joint color-irrelevant consistency learning and identity-aware modality adaptation for visible-infrared cross modality person re-identification[C]//Proceedings of the AAAI Conference on Artificial Intelligence. Palo Alto: AAAI Press, 2021, 35(4): 3520-3528. doi: 10.1609/aaai.v35i4.16466
|
| [25] |
CHEN Y, WAN L, LI Z H, et al. Neural feature search for RGB-infrared person re-identification[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2021: 587-597.
|
| [26] |
WU Q, DAI P Y, CHEN J, et al. Discover cross-modality nuances for visible-infrared person re-identification[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2021: 4328-4337.
|
| [27] |
TIAN X D, ZHANG Z Z, LIN S H, et al. Farewell to mutual information: Variational distillation for cross-modal person re-identification[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2021: 1522-1531.
|
| [28] |
YE M, WANG Z, LAN X, et al. Visible thermal person re-identification via dual-constrained top-ranking[C]//Proceedings of the International Joint Conference on Artificial Intelligence. New York: ACM, 2018: 1092-1099.
|
| [29] |
LIU H J, TAN X H, ZHOU X C. Parameter sharing exploration and hetero-center based triplet loss for visible-thermal person re-identification[EB/OL]. (2020-14-04)[2022-03-20]. https://arxiv.org/abs/2008.06223.pdf.
|
| [30] |
NGUYEN D T, HONG H G, KIM K W, et al. Person recognition system based on a combination of body images from visible light and thermal cameras[J]. Sensors, 2017, 17(3): 605. doi: 10.3390/s17030605
|
| [31] |
WANG P Y, ZHAO Z C, SU F, et al. Deep multi-patch matching network for visible thermal person re-identification[J]. IEEE Transactions on Multimedia, 2021, 23: 1474-1488. doi: 10.1109/TMM.2020.2999180
|
| [32] |
YE M, SHEN J B, LIN G J, et al. Deep learning for person re-identification: A survey and outlook[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, 44(6): 2872-2893. doi: 10.1109/TPAMI.2021.3054775
|
Figures(3) / Tables(7)
Copyright © Journal of Beijing University of Aeronautics and Astronautics
Address: Editorial Department of Journal of Beijing University of Aeronautics and Astronautics, 37 Xueyuan Road, Haidian District, Beijing Post Code: 100191 Email: jbuaa@buaa.edu.cn
Supported by:
Beijing Renhe Information Technology Co., Ltd.
DownLoad:
