| Citation: | XU G Z，LIU G F，KUANG W，et al. Accurate license plate location based on synchronous vertex and body region detection[J]. Journal of Beijing University of Aeronautics and Astronautics，2024，50（2）：376-387 （in Chinese） doi: 10.13700/j.bh.1001-5965.2022.0396
|
A novel unconstrained license plate accurate location algorithm is designed by simultaneously detecting the four local vertex regions and the body of a license plate and fusing the results to address the issue that the widely used rectangular bounding boxes in mainstream target detection methods cannot meet the license plate location accuracy requirement in many unconstrained environments where the license plate images are not commonly rectangle. At first, the four local rectangular sub-regions with centers on four vertices of a license plate were annotated as vertex-region objects according to the size of the plate’s contour-rectangle and the vertex coordinates. Then, a multi-class image dataset is built up in which the contour-rectangle region covering the whole license plate body is a class and the four kinds of vertex-region construct other four classes. In order to locate these five object classes efficiently, the output structure of the YOLOv5 network is modified by taking accuracy and efficiency into consideration and trained with the newly constructed multi-class dataset.Finally, vertex region grouping and single missing vertex forecasting are carried out as the post-processing to address the issue that there are multiple candidate license plates in an image and a few vertices region false or missing detection errors will happen in some unique instances.By exploiting the relationship among the vertexes, the post-processing can effectively recognize missing and false detection errors in some special complex scenarios and improve the whole system’s performance greatly. The proposed algorithm is evaluated on the Chinese city parking dataset (CCPD), and reaches an average positioning accuracy of 99.25% and an average recall rate of 98.70%. The performance certificates our method not only can accurately predict the coordinates of the four vertices but also can run at 121 frame/s on a moderate GPU hardware platform, which has great application potential.
| [1] |
JAMTSHO Y, RIYAMONGKOL P, WARANUSAST R. Real-time license plate detection for non-helmeted motorcyclist using YOLO[J]. ICT Express, 2021, 7(1): 104-109. doi: 10.1016/j.icte.2020.07.008
|
| [2] |
HUANG Q, CAI Z, LAN T. A single neural network for mixed style license plate detection and recognition[J]. IEEE Access, 2021, 9: 21777-21785. doi: 10.1109/ACCESS.2021.3055243
|
| [3] |
曾丽华, 李超, 熊璋. 基于边缘与颜色信息的车牌精确定位算法[J]. 北京航空航天大学学报, 2007, 33(9): 1112-1116. doi: 10.3969/j.issn.1001-5965.2007.09.026
ZENG L H, LI C, XIONG Z. Precisely license plate locating algorithm based on edge and color information[J]. Journal of Beijing University of Aeronautics and Astronautics, 2007, 33(9): 1112-1116(in Chinese). doi: 10.3969/j.issn.1001-5965.2007.09.026
|
| [4] |
CHEN J I Z, ZONG J I. Automatic vehicle license plate detection using K-means clustering algorithm and CNN[J]. Journal of Electrical Engineering and Automation, 2021, 3(1): 15-23. doi: 10.36548/jeea.2021.1.002
|
| [5] |
XU Z B, YANG W, MENG A J, et al. Towards end-to-end license plate detection and recognition: A large dataset and baseline[C]//Proceedings of the European Conference on Computer Vision. Berlin: Springer, 2018: 255-271.
|
| [6] |
SILVA S M, JUNG C R. License plate detection and recognition in unconstrained scenarios[C]//Proceedings of the European Conference on Computer Vision. Berlin: Springer, 2018: 580-596.
|
| [7] |
SELMI Z, HALIMA M B, PAL U, et al. DELP-DAR system for license plate detection and recognition[J]. Pattern Recognition Letters, 2019, 129: 213-223.
|
| [8] |
HE K M, GKIOXARI G, DOLLÁR P, et al. Mask R-CNN[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 42(2): 386-397. doi: 10.1109/TPAMI.2018.2844175
|
| [9] |
SZEGEDY C, LIU W, JIA Y Q, et al. Going deeper with convolutions[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2015: 1-9.
|
| [10] |
CHEN L S, TIAN S, MA W J, et al. End-to-end trainable network for degraded license plate detection via vehicle-plate relation mining[J]. Neurocomputing, 2021, 446: 1-10. doi: 10.1016/j.neucom.2021.03.040
|
| [11] |
WANG W, YANG J, CHEN M, et al. A light CNN for end-to-end car license plates detection and recognition[J]. IEEE Access, 2019, 7: 173875-173883. doi: 10.1109/ACCESS.2019.2956357
|
| [12] |
YIN X, LIU X. Multi-task convolutional neural network for pose-invariant face recognition[J]. IEEE Transactions on Image Processing, 2017, 27(2): 964-975.
|
| [13] |
WANG Y, BIAN Z P, ZHOU Y, et al. Rethinking and designing a high-performing automatic license Plate recognition approach[EB/OL]. (2021-06-17) [2022-05-20]. https://arxiv.org/abs/2011.14936.
|
| [14] |
LIN T Y, DOLLÁR P, GIRSHICK R, et al. Feature pyramid networks for object detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2017: 2117-2125.
|
| [15] |
Ultralytics. YOLOv5[EB/OL]. (2022-05-12) [2022-05-20]. https://github.com/ultralytics/yolov5.
|
| [16] |
REDMON J, FARHADI A. YOLOv3: An incremental improvement[EB/OL]. (2018-04-08) [2022-05-20]. https://arxiv.org/abs/1804.02767.
|
| [17] |
ALEXEY B, WANG C Y, LIAO H. YOLOv4: Optimal speed and accuracy of object detection[EB/OL]. (2020-04-23) [2022-05-20]. https://arxiv.org/abs/2004.10934v1.
|
| [18] |
WANG C Y, LIAO H, YEH I H, et al. CSPNet: A new backbone that can enhance learning capability of CNN[EB/OL]. (2019-11-27) [2022-05-20].https://arxiv.org/abs/1911.11929.
|
| [19] |
HE K M, ZHANG X, REN S, et al. Spatial pyramid pooling in deep convolutional networks for visual recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(9): 1904-1916.
|
| [20] |
LIU S, QI L, QIN H, et al. Path aggregation network for instance segmentation[EB/OL]. (2018-03-05) [2022-05-20]. https://arxiv.org/abs/1803.01534.
|
| [21] |
LOSHCHILOV I, HUTTER F. SGDR: Stochastic gradient descent with warm restarts[EB/OL]. (2016-08-13) [2022-05-20]. https://arxiv.org/abs/1608.03983.
|
| [22] |
徐光柱, 匡婉, 李兴维, 等. YOLOv3与顶点偏移估计相结合的车牌定位[J]. 计算机辅助设计与图形学学报, 2021, 33(4): 569-579.
XU G Z, KUANG W, LI X W, et al. License plate location based on YOLOv3 and vertex offset estimation[J]. Journal of Computer-Aided Design & Computer Graphics, 2021, 33(4): 569-579(in Chinese).
|
| [23] |
唐倩, 贺伟, 张林江. 基于空间转换网络的端到端车牌检测与识别[J]. 光电子·激光, 2021, 32(5): 524-531. doi: 10.16136/j.joel.2021.05.0423
TANG Q, HE W, ZHANG L J. End to end license plate detection and recognition based on spatial transformer network[J]. Journal of Optoelectronics·Laser, 2021, 32(5): 524-531(in Chinese). doi: 10.16136/j.joel.2021.05.0423
|
| [24] |
QIN S S, LIU S J. Towards end-to-end car license plate location and recognition in unconstrained scenarios[EB/OL]. (2020-08-25) [2022-05-20]. https://arxiv.org/abs/2008.10916.
|
| [25] |
SILVA S M, JUNG C R. A flexible approach for automatic license plate recognition in unconstrained scenarios[J]. IEEE Transactions on Intelligent Transportation Systems, 2021, 23(6): 5693-5703.
|
| [26] |
REDMON J, FARHADI A. YOLO9000: Better, faster, stronger[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2017: 7263-7271.
|
| [27] |
ZOU Y, ZHANG Y, YAN J, et al. License plate detection and recognition based on YOLOv3 and ILPRNET[J]. Signal Image and Video Processing, 2021, 9: 312-320.
|
| [28] |
LI B, TIAN B, LI Y, et al. Component-based license plate detection using conditional random field model[J]. IEEE Transactions on Intelligent Transportation Systems, 2013, 14(4): 1690-1699.
|
| [29] |
YUAN Y, ZOU W, ZHAO Y, et al. A robust and efficient approach to license plate detection[J]. IEEE Transactions on Image Processing, 2016, 26(3): 1102-1114.
|
Figures(21) / Tables(3)
Copyright © Journal of Beijing University of Aeronautics and Astronautics
Address: Editorial Department of Journal of Beijing University of Aeronautics and Astronautics, 37 Xueyuan Road, Haidian District, Beijing Post Code: 100191 Email: jbuaa@buaa.edu.cn
Supported by:
Beijing Renhe Information Technology Co., Ltd.
DownLoad:
