Currently, the number of cars in the world is well over 1 billion. It is no wonder that one of the most common computer vision tasks is the effective control of these vehicles through automatic number plate recognition (ANPR) systems. The applications of automatic vehicle number plate detection and recognition vary depending on the area of use and include, among others, border control, stolen car detection, automatic ticketing of vehicles and toll collection, traffic monitoring and safety control, smart parking, tracking of transportation, etc.
A typical automatic number plate recognition algorithm includes several steps (Fig. 1): car license plate detection (1), character segmentation (2) and recognition (3).
Within this vehicle number plate recognition project, we concentrated on the first step of the algorithm: locating the area on the image that corresponds to the license plate number. It is indeed a crucial step of the whole recognition pipeline since it strongly affects the overall system performance.
As the first step of the project, we extracted the plate candidates areas based on the following two approaches:
- stroke width transform (SWT), which was used to locate the image regions containing the text);
- blob detection algorithm, which increased the final detection rate.
The SWT is a well-known text detection algorithm. The idea lies behind the careful analysis of the image edge map and local gradient directions. The basic preparation steps include the calculation of the Sobel gradients and Canny edge map construction.
The results of SWT application to the image and its 3D view are illustrated in Fig. 2.
Fig. 2. SWT image example
Read also:
Road DetectorTo locate the text region on the image, which corresponds to the vehicle number plate, we proceed with the following steps:
To increase the rate of the vehicle number plate detection, we have additionally integrated the blobs analysis procedure, which is based on morphological operators and contours calculation and applied to the preprocessed images. Fig. 3 illustrates an example of found number plate candidates using the developed method.
Fig. 3. Detected blobs and bounding boxes
The combination of SWT and blob detection approaches returns the list of the number plate candidates. The final step of the project was to extract the true car license plate area.
For this, we used a 3-layer perceptron neural network (NN) which classifies the obtained sequence of number plate candidates. The NN was trained using different sets of features including:
- Haar-based features (based on integral images);
- statistical features (mean, std, skewness);
- principal component analysis (PCA) features.
Fig. 4 demonstrates an example of the obtained receiver operating characteristics (ROC) curves for three different learning scenarios. The ROC curve represents the dependence of true positive rate (TP) on the false positive rate (FP).
The best classification results are obtained when using a combination of all three groups of features for NN training.
Our paper which describes the above-mentioned results in more detail was presented at the Signal Processing Symposium 2015 where it received the 1st prize for the best publication.
The ANPR pipeline discussed in this post is an open-ended research task and all steps can be further improved, however, it proves that CNN is not the only solution for solving such kind of tasks.