See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/377560436

Monocular Vision-Based Vehicle Distance Prediction utilizing Number Plate

Preprint · January 2024

CITATIONS READS
0 150

6 authors, including:

Fahim Hasan Khan

University of California, Santa Cruz
18 PUBLICATIONS 41 CITATIONS

SEE PROFILE

All content following this page was uploaded by Fahim Hasan Khan on 21 January 2024.

The user has requested enhancement of the downloaded file.

 Not the final version. Preprint accepted to the 6th International Conference on Electrical Information and Communication Technology (EICT),
07-09 December 2023, Khulna, Bangladesh

Monocular Vision-Based Vehicle Distance

Prediction utilizing Number Plate
Mahedi Kamal1 , Md. Abu Syed1 , Tasnim Fariha1 ,
Afrina Kabir Zinia 1 , Fahim Hasan Khan2 , Md. Mahbubur Rahman1 ,
1
Department of Computer Science and Engineering, Military Institute of Science and Technology,
Mirpur Cantonment, Dhaka-1216, Bangladesh
2
University of California, Santa Cruz, CA 95064, USA
Email: [email protected], [email protected], [email protected],
[email protected], [email protected], [email protected]

Abstract—To reduce road traffic accidents, maintaining a safe The two main categories of distance estimation approaches
distance between vehicles can play an important role. Advanced are sensor-based [4] and vision-based [5] systems. Most
driver assistance systems (ADAS) primarily focus on collision autonomous cars use expensive sensors like LIDAR, RADAR,
avoidance technologies. This article introduces two vehicle dis-
tance detection methods based on monocular vision. Parameters and high-precision global positioning systems as the founda-
extracted from the number plate are utilized to predict the tion of their vision systems (GPS).
distance. Two different parameters are considered for the two
methods, one is the weight-height, and another one is the digit Vision-based distance estimation nowadays attracts the re-
size of the number plate in pixels. To calculate these parameters, searchers because it can provide richer sensing at a far
the number plate must first be detected, for which YOLOv5 lower cost. Vision-based systems tend to be divided into the
is employed. Finally, the distance is predicted using either a following groups: monocular vision [6] and stereo vision [7].
regression model or an ANN model that takes the extracted
parameters from the number plate as input. As the number plate Nevertheless, because two cameras must be calibrated and
is focused rather than the whole car size, the posture of the vehicle matched, stereo vision systems need a longer execution time,
on the road doesn’t affect the prediction much. Furthermore, demonstrate poor efficiency, and include a significant amount
different car models do not require separate trained models. of computational complexity. Monocular distance estimation
Despite the fact that number plates also vary, which is mostly techniques currently available still have shortcomings, such as
dependent on region, they may generally be more standardized
than automobile types. low accuracy and constrained application space. Here in the
Index Terms—ADAS, monocular vision. YOLOV5, regression paper, a new approach for monocular distance measurement
model, ANN is proposed.
For monocular vision, distance estimate methods fall into
I. I NTRODUCTION two broad categories [8]: relative depth estimations [9], [10]
and absolute distance [11] estimations. Relative depth estima-
The study of ADAS (advanced driver assistance systems) is tion is achieved by analyzing the size and position of objects in
growing significantly [1], [2]. Using technology to aid drivers the image and using this information to estimate their relative
in avoiding collisions and lowering the likelihood of damage distances from the camera. Absolute distance estimation, on
or death, ADAS is crucial for enhancing driving efficiency the other hand, requires additional information, such as the
and safety. Among other things, ADAS capabilities include size of the objects in the scene and the camera’s intrinsic
automated emergency braking, adaptive cruise control, lane parameters. This information is used to calculate the absolute
departure warning, and collision avoidance. ADAS can save distance from the camera to the objects in the scene. Thus, we
lives, prevent injuries, and lower collision costs by lowering propose a technique for relative depth estimation.
the likelihood of accidents.
Distance measurement is vital [3] for ADAS because it In this paper, we present a system for calculating the
offers important details about the surroundings of the vehicle, distance between cars based on the characteristics of the num-
which are required for assuring safety and preventing acci- ber plate using monocular vision from a single camera. For
dents. The ADAS may alert the driver about impending crashes number plate detection, a YOLOv5 model is employed [12],
by precisely calculating the distance to nearby objects, and in [13]. After detecting the number plate, the distance is predicted
extreme circumstances, it may even take control of the car to using a regression model or an ANN (artificial neural network)
prevent collisions altogether. Furthermore, accurate distance model. The two main advantages of the work are posture
readings are necessary for functions like adaptive cruise con- affects the result less, and there is no effect of different car
trol and automated emergency braking, which depend on them models on the result. As monocular vision is used, it is also
to function safely and effectively. cost-efficient and easy to place.
 Not the final version. Preprint accepted to the 6th International Conference on Electrical Information and Communication Technology (EICT),
07-09 December 2023, Khulna, Bangladesh

II. R ELATED W ORK

There already exists much research work for calculating dis-

tance with a monocular distance. Some prominent distinctive
types of works are an estimation of the distance between the
camera and frontal vehicles based on camera calibration by
combining three main steps: vanishing point extraction, lanes
detection, and vehicles detection in the field of a 3-D real
scene [14], another work was done on a CARLA vehicle
and its distance detection system in a virtual environment
by adjusting the network layer structure of Yolo v5s [15],
authors of [16] proposed a calibrated feature-point-based
(CFPB) method to estimate the car-following distance adaptive Fig. 1. ANN model structure for method 1
to rough roads using a single camera. Instead of using the pre-
calibrated parameters to improve the distance estimation of
vehicles in a Forward Collision Warning System, a method
was proposed by a probabilistic fusion of the geometric
constraint and the width constraint [17], to improve the
accuracy and robustness of ranging results, the Authors of [18]
proposed a monocular vision end-to-end inter-vehicle distance
estimation method based on 3D detection. Here in this paper,
we presented a distance prediction method that depends on the
pixel or size calculation of a car in an image which is different
than the works discussed above.
There are some works available that were found mostly
closer to the idea presented here. In one work, authors [19] Fig. 2. ANN model structure for method 2
proposed a monocular vision ranging method based on a
convolutional neural network, and the idea is for a vehicle,
measurement between vehicles to vehicles by plate car. The
distance and the pixel size of the object in the image are
picture process by Fast Fourier Transform (FFT). There is a
inversely proportional. In another work, authors [20] proposed
database that has already stored the size of the plates and the
a method where detection outputs, along with the height of
distances between the cars. The output is obtained by data
the camera lens, were used to train the distance estimation
comparison in the database. In our method, rather than using
models. Detection outputs include bounding box height (H),
a database to compare, we used the ML model.
width (W), bounding box center vector (lx and ly), originating
from the upper left of the video frame, and type of vehicle. III. M ETHODOLOGY
Three main methods for distance estimation were considered Our objective is to calculate the distance of a car using the
and compared: linear regression model, pinhole model, and vision only of a single camera. To reach the objective, there
artificial neural network (ANN). The model ANN was deter- are two main steps. The first step is object detection, and the
mined to be the best estimator for distance. However, these second step is distance prediction.
methods need to deal with the posture error of a car.
To deal with the errors caused by the posture of the car, there A. Object Detection
are some research works. For monocular vision, a measure In this case, the object to be detected is the vehicle’s
issue that affects the accuracy is the attitude angle. Authors of number plate, and the YOLOv5 ML model was utilized for this
[8] proposed a method to address the vehicle attitude angle purpose. The dataset was annotated using an online platform
problem. Authors of [21] proposed the distance estimation called makesense.ai. Subsequently, a model was trained on
method based on the vehicle pose information, which can be the annotated data for 100 iterations, with a batch size of 4.
used to eliminate the error of distance estimation effectively The accuracy of this object detection will impact the distance
caused by the change of the pitch angle and roll angle of estimation. The trained model generates different outputs for
the unmanned vehicle. In addition, the pose information could each frame of the video, and two of these outputs were
also help us estimate whether the vehicle is on a slope. But considered here.
methods to deal with the error makes these model complex. Output 1: The YOLOv5 model provides 4 parameters of the
So a simpler solution is investigated in this paper. detected number plate: xmin, ymin, xmax, ymax. Then, using
The simple idea is we can use the number plate rather equations (1) and (2) below, which represent the width and
than the full car. A number plate is a 2D object which height of the detected number plate, dy and dx were calculated.
has less posture effect. There is work done considering the
number plate. That is, the authors of [22] proposed distance dy = ymax − ymin (1)
 Not the final version. Preprint accepted to the 6th International Conference on Electrical Information and Communication Technology (EICT),
07-09 December 2023, Khulna, Bangladesh

Fig. 3. flowchart of method 1

dx = xmax − xmin (2)

Output 2: Following the number plate detection performed

by the YOLOv5 model, the number plate was cropped from
the image. Subsequently, image processing was applied to the Fig. 4. process in method 1
cropped image, and the digit size of the number plate was
calculated.

B. Distance Prediction
During the object detection process, two different types of
outputs were extracted. In this step, there are two distinct
regression models of degree 4 for each output, as well as two
separate ANN models. These ANN models were compiled
with mean squared error loss and Adam optimizer. Both
the regression model and ANN model predict the distance Fig. 5. detailed 3rd step in method 1
independently.
Therefore, there are essentially two methods for predicting
the distance, which are briefly discussed below. used on the plate to traverse over the black pixels to calculate
Method 1: This method was applied to stationary cars. the digit size. The detailed 3rd step is shown in Figure 5.
Initially, an image of a stationary car was captured, and Finally, the entire process of method 1 is shown in Figure
the number plates within the image were detected using 4. The advantage of this method is that highly accurate results
YOLOv5. Subsequently, the digit size of each number plate can be obtained if the camera captures high-megapixel images.
was calculated. Based on the calculated size, the distance of This method can also be applied to moving cars if it can be
each car present in the image was predicted using either a ensured that the pixels are not significantly affected by the
polynomial regression model or an ANN. The structure of car’s speed.
ANN is shown in the figure 1. ANN takes the digit height as Method 2: This method was applied to moving cars. First, a
input. The process is illustrated in Figure 3 with a flow chart. video was recorded and then converted into frames. For each
In step 3 of method 1, for calculating the height of the digit frame, cars and their distances need to be predicted. To achieve
in the number plate; image processing is used. this, the number plate is first detected in each frame, and then
Image processing approach: First, a part of the text of the the width and height of the number plates are calculated. Based
number plate is taken and the other part is excluded, here on the weight and height, a polynomial regression model or
excluded means pixels on that part are converted into 256 an ANN predicts the distance. The structure of ANN is shown
intensity. Then the selected part is converted into black(pixel in Figure 2. ANN takes the width and height of the number
value 255) and white(pixel value 0) using a threshold value. plate as input. The process is illustrated in Figure 6 with a
Pixels of text areas are made black. Then a BFS algorithm is flow chart.
 Not the final version. Preprint accepted to the 6th International Conference on Electrical Information and Communication Technology (EICT),
07-09 December 2023, Khulna, Bangladesh

Fig. 6. flowchart of method 2

Fig. 8. System process flowchart of the entire system

IV. DATA C OLLECTION A PPROACH

There is a dataset set for training a model that can detect
expected number plates. We collected video frames and images
of cars and annotated them using makesense.ai.
Two types of data were collected for training the distance
predictor model: one for still cars and another for moving cars.
For collecting still car data, we used the Samsung Galaxy A12,
which features a 48-megapixel camera. For capturing video
data of moving cars, we utilized the iPhone 13 Pro, which has
a frame rate of 60 fps and a resolution of 12 MP.
Data collection for still images: A measuring tape and a
mobile camera were used. After capturing each image, the
distance of the car was measured using the measuring tape
and recorded in meters. The ground truth for the still car data
is highly accurate.
Data collection for moving cars: Collecting ground truth
for moving cars posed a challenge. A complete setup was
arranged on the road, with marked points placed alongside
the road. The distance between each marked point was set
to one meter. Two cars were involved in the data collection
Fig. 7. process in method 2 process. In one car, a person was seated in the front seat with
a mobile device to capture video footage. Then, the two cars
would start moving on the marked road. The data collection
process was repeated several times. In certain setups, both cars
So, the main difference between method 1 and method 2
moved in the same direction, while in other setups, they moved
is that method 1 calculates the digit size, whereas method 2
in opposite directions.
calculates the width and height of the number plate.
Datasets needed to recreate proposed model can be found
The complete process of method 2 is illustrated in Figure here: https://github.com/MahediKamal/Monocular-Vision-
7. The accuracy of the number plate detection model plays Based-Vehicle-Distance-Prediction-utilizing-Number-Plate
a crucial role in the results obtained from method 2. The
more accurately the model detects the number plate, the more V. D EVELOPING ML M ODEL
precise the predicted distance will be. It is important to note We have three ML models for both methods: one model
that this method can also be applied to stationary cars. The for number plate detection, for which we used YOLOv5,
overall process of the system and its functioning are depicted and two models for predicting distance, namely a polynomial
in Figure 8. regression model and an ANN model.
 Not the final version. Preprint accepted to the 6th International Conference on Electrical Information and Communication Technology (EICT),
07-09 December 2023, Khulna, Bangladesh

TABLE II
ACTUAL VS PREDICTED DISTANCE FOR METHOD 2

Actual Predicted Predicted Absolute Absolute

Distance Distance Distance Error for Error for
(meter) by by ANN regression ANN
regression (meter)
(meter)
4.50 4.52 4.44 0.02 0.06
4.70 4.66 4.59 0.04 0.11
5.00 4.99 4.93 0.01 0.07
5.50 5.60 5.48 0.10 0.02
5.80 5.83 5.75 0.03 0.05
5.90 5.85 5.76 0.05 0.14
6.00 6.02 5.96 0.02 0.04
6.50 6.44 6.44 0.06 0.06
7.00 6.94 6.90 0.06 0.10
Fig. 9. Fitted regression curve of method 1 7.50 7.43 7.43 0.07 0.07
8.30 8.51 8.26 0.21 0.04
8.50 8.67 8.42 0.17 0.08
TABLE I 9.00 9.05 9.01 0.05 0.01
ACTUAL VS PREDICTED DISTANCE FOR METHOD 1 9.20 9.11 9.08 0.09 0.12
9.50 9.26 9.38 0.24 0.12
Actual Predicted Predicted Absolute Absolute
Distance Distance Distance Error for Error for
(meter) by by ANN regression ANN TABLE III
regression (meter) C OMPARISON OF RESULT WITH OTHER PAPERS
(meter)
4.00 4.22 4.24 0.22 0.24 Method MSE MAE RMSE
4.50 4.49 4.52 0.01 0.01 [23] 0.26 0.49 0.51
5.00 4.58 4.78 0.15 0.22 [15] 1.77 1.33 1.33
6.00 5.91 5.94 0.09 0.06 [8] — 0.38(distance:0-10m) —
6.50 6.23 6.39 0.27 0.11 0.85(distance:10-20m)
8.00 8.03 8.18 0.03 0.18 our method:1 0.09 0.22 0.29
9.00 8.98 8.85 0.02 0.15 (regression)
10.00 10.12 9.89 0.12 0.11 our method:1 0.03 0.14 0.18
11.00 11.45 11.15 0.45 0.15 (ANN)
12.00 12.47 11.98 0.47 0.02 our method:2 0.01 0.08 0.11
13.00 13.02 12.68 0.02 0.32 (regression)
13.50 13.60 13.51 0.10 0.01 our method:2 0.006 0.07 0.08
14.50 14.21 14.33 0.29 0.17 (ANN)
15.50 14.58 15.15 0.65 0.35
16.00 15.52 15.97 0.48 0.03
The final predicted distance and the actual distance of some
data for method 2 are shown in Table 2.
The YOLOv5 model was constructed through the following Using Table 2, the calculation of the Mean Squared Error,
steps: data collection, data annotation, and training YOLOv5 Root Mean Squared Error, and Mean Absolute Error for
using the annotated data. regression are 0.01, 0.11, and 0.08 respectively and the Mean
The regression model and ANN model were built using the Squared Error, Root Mean Squared Error, and Mean Absolute
following steps: data collection, preparing the ground truth Error for ANN are 0.006, 0.08, and 0.07 respectively.
data for the models (calculating pixel and distance for method A comparison between our two methods and some other
1, and width, height, and distance for method 2), and training proposed methods based on a monocular camera is presented
the regression model. The fitted regression curve for method in Table 3. From the deviation of the previous methods from
1 is displayed in Figure 9. the actual value, it can be observed that our method shows
significant improvement in distance calculation, especially in
VI. R ESULT A NALYSIS
the close range. Additionally, among the regression and ANN
Our final desired result is obtained from a regression or models, the ANN model performs better in terms of accuracy.
ANN model. The table shown in Table 1 displays the final How the proposed method is displayed to the user in
predicted distance and the actual distance for some data in practice is shown in Figure 10. The figure shows that the car
method 1. Various error values are calculated using the data is 7.2 meters away from the mobile camera. Some example of
from Table 1. distance prediction is shown in Figure 11.
The Mean Squared Error, Root Mean Squared Error, and
Mean Absolute Error for regression in Table 1 are 0.09, 0.29, VII. L IMITATIONS
and 0.22 respectively and the Mean Squared Error, Root Mean This study on the depth measurement of cars has a few
Squared Error, and Mean Absolute Error for ANN are 0.03, shortcomings. The accuracy of our depth measurement is
0.18, and 0.14 respectively. influenced by factors like weather and lighting conditions,
 Not the final version. Preprint accepted to the 6th International Conference on Electrical Information and Communication Technology (EICT),
07-09 December 2023, Khulna, Bangladesh

[3] Mingchang Zhao, Abdelhamid Mammeri, and Azzedine Boukerche. Dis-

tance measurement system for smart vehicles. In 2015 7th International
Conference on New Technologies, Mobility and Security (NTMS), pages
1–5. IEEE, 2015.
[4] Diarmaid O Cualain, Martin Glavin, Edward Jones, and P Denny.
Distance detection systems for the automotive environment: A review.
In Irish Signals and Systems Conf, pages 13–14. Citeseer, 2007.
[5] Apoorva Joglekar, Devika Joshi, Richa Khemani, Smita Nair, and
Shashikant Sahare. Depth estimation using monocular camera. In-
ternational journal of computer science and information technologies,
2(4):1758–1763, 2011.
[6] Eric Raphael, Raymond Kiefer, Pini Reisman, and Gaby Hayon. De-
velopment of a camera-based forward collision alert system. SAE
International Journal of Passenger Cars-Mechanical Systems, 4(2011-
Fig. 10. Distance prediction on a frame 01-0579):467–478, 2011.
[7] Vinh Dinh Nguyen, Thuy Tuong Nguyen, Dung Duc Nguyen, and
Jae Wook Jeon. Toward real-time vehicle detection using stereo vision
and an evolutionary algorithm. In 2012 IEEE 75th Vehicular Technology
Conference (VTC Spring), pages 1–5. IEEE, 2012.
[8] Liqin Huang, Ting Zhe, Junyi Wu, Qiang Wu, Chenhao Pei, and
Dan Chen. Robust inter-vehicle distance estimation method based on
monocular vision. IEEE Access, 7:46059–46070, 2019.
[9] Jae-Han Lee and Chang-Su Kim. Monocular depth estimation using
relative depth maps. In Proceedings of the IEEE/CVF Conference on
Computer Vision and Pattern Recognition, pages 9729–9738, 2019.
[10] Kyuhong Shim, Jiyoung Kim, Gusang Lee, and Byonghyo Shim. Depth-
relative self attention for monocular depth estimation, 2023.
[11] Armin Masoumian, David G.F. Marei, Saddam Abdulwahab, Juliá n
Cristiano, Domenec Puig, and Hatem A. Rashwan. Absolute distance
prediction based on deep learning object detection and monocular
depth estimation models. In Frontiers in Artificial Intelligence and
Applications. IOS Press, oct 2021.
Fig. 11. Distance prediction examples [12] Shreya Raj, Yash Gupta, and Ruchika Malhotra. License plate recogni-
tion system using yolov5 and cnn. In 2022 8th International Conference
on Advanced Computing and Communication Systems (ICACCS), vol-
ume 1, pages 372–377. IEEE, 2022.
vehicle speed, distance traveled, and the angle at which the [13] Qianqian Zhu, Yingnan Liu, Zihao Zhao, and Xin Ma. Research on
number plate is inserted. Additionally, the system was only license plate location algorithm based on yolov5. Journal of Physics:
tested on one type of regular-sized plates. To make it workable Conference Series, 2278(1):012040, may 2022.
properly different models for different types of number plates [14] Sana Bougharriou, Fayçal Hamdaoui, and Abdellatif Mtibaa. Vehicles
distance estimation using detection of vanishing point. Engineering
should be trained. Computations, 2019.
[15] Tian-Hao Wu, Tong-Wen Wang, and Ya-Qi Liu. Real-time vehicle and
VIII. C ONCLUSION A ND F UTURE W ORK distance detection based on improved yolo v5 network. In 2021 3rd
World Symposium on Artificial Intelligence (WSAI), pages 24–28. IEEE,
In this study, a mobile camera was used, and the number 2021.
[16] Zhong Cao, Diange Yang, Kun Jiang, Shaobing Xu, Sijia Wang,
plate was selected as the detecting object to estimate distance. Minghan Zhu, and Zhongyang Xiao. A geometry-driven car-following
Since everyone has a smartphone these days, the model can be distance estimation algorithm robust to road slopes. Transportation
used to create an affordable ADAS mobile app. The number research part C: emerging technologies, 102:274–288, 2019.
[17] Stephanie Leßmann, Mirko Meuter, Dennis Müller, and Josef Pauli.
plate has the advantages of being less affected by vehicle Probabilistic distance estimation for vehicle tracking application in
posture, not requiring consideration of different automobile monocular vision. In 2016 IEEE Intelligent Vehicles Symposium (IV),
models, and increasing accuracy at close range. Solely daylight pages 1199–1204. IEEE, 2016.
was used to test the model. Therefore, the model can be [18] Ting Zhe, Liqin Huang, Qiang Wu, Jianjia Zhang, Chenhao Pei, and
Liangyu Li. Inter-vehicle distance estimation method based on monocu-
enhanced to account for various meteorological situations. To lar vision using 3d detection. IEEE transactions on vehicular technology,
more correctly anticipate the distance, further features can be 69(5):4907–4919, 2020.
incorporated. In this model, the impact of camera resolution [19] Dongsheng Bao and Peikang Wang. Vehicle distance detection based
on monocular vision. In 2016 International Conference on Progress in
can be investigated. Informatics and Computing (PIC), pages 187–191. IEEE, 2016.
[20] Saeed Arabi, Anuj Sharma, Michelle Reyes, Cara Hamann, and Corinne
R EFERENCES Peek-Asa. Farm vehicle following distance estimation using deep
learning and monocular camera images. Sensors, 22(7):2736, 2022.
[1] Abdallah Moujahid, Mounir ElAraki Tantaoui, Manolo Dulva Hina, As- [21] S H Qi, J Li, Z P Sun, J T Zhang, and Y Sun. Distance estimation
sia Soukane, Andrea Ortalda, Ahmed ElKhadimi, and Amar Ramdane- of monocular based on vehicle pose information. Journal of Physics:
Cherif. Machine learning techniques in adas: a review. In 2018 In- Conference Series, 1168(3):032040, feb 2019.
ternational Conference on Advances in Computing and Communication [22] J Phelawan, P Kittisut, and N Pornsuwancharoen. A new technique for
Engineering (ICACCE), pages 235–242. IEEE, 2018. distance measurement of between vehicles to vehicles by plate car using
[2] Lutz Eckstein and Adrian Zlocki. Safety potential of adas–combined image processing. Procedia Engineering, 32:348–353, 2012.
methods for an effective evaluation. In 23rd International Technical [23] K Karthika, S Adarsh, and KI Ramachandran. Distance estimation of
Conference on the Enhanced Safety of Vehicles (ESV) Seoul, South preceding vehicle based on mono vision camera and artificial neural
Korea, pages 15–25, 2013. networks. In 2020 11th International Conference on Computing,
 Not the final version. Preprint accepted to the 6th International Conference on Electrical Information and Communication Technology (EICT),
07-09 December 2023, Khulna, Bangladesh

Communication and Networking Technologies (ICCCNT), pages 1–5.

IEEE, 2020.

View publication stats