Dynamic Pricing for Parking System

Using Reinforcement Learning

Li Zhe Poh, Connie Tee, Thian Song Ong, and Michael Goh

Abstract The number of vehicles in urban cities has increased and raised atten-
tion towards the need for effective parking lot management in public areas such as
hospital, shopping mall and office building. In this study, dynamic pricing is deployed
with real time parking information to maximize the parking usage rate and alleviate
traffic congestion. Dynamic pricing is a practice of varying the price of product of
service reflected by the market conditions. This technique can be used to deal with
vehicle flow around the parking area including peak and non-peak hour. During
peak hours, the dynamic pricing mechanism will regulate the price of parking fee to
a relatively high rate, and vice versa for non-peak hours. Reinforcement Learning
(RL) is used in this paper to develop a dynamic pricing model for parking manage-
ment. Dynamic pricing over time is divided into episodes and shuffled back and forth
through an hourly increment. The parking usage rate and traffic congestion rate are
regarded as the rewards for price regulation.

Keywords Parking management · Reinforcement learning · Dynamic pricing

1 Introduction

Huge increase in private vehicles is one of the pain points in many urban areas
especially big cities like London, Hong Kong and Kuala Lumpur. The increase of
vehicles brings negative impacts to the environment and also people around the cities.
Increased number of vehicles leads to raising demand for parking lots. According
to a survey published by New Strait Time (2017), the average time spent every
day looking for parking areas in some major cities is about 25 min. The drivers
keep circling around the destination and time is wasted to look for available parking
lot. This leads to the increase of fuel combustion and carbon dioxide emission that
destroys the atmosphere and causes greenhouse effect. The higher the time driver

L. Z. Poh (B) · C. Tee · T. S. Ong · M. Goh

Faculty of Information Science and Technology, Multimedia University, Melaka, Malaysia
e-mail: [email protected]

© The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2021 157
H. Kim et al. (eds.), Information Science and Applications, Lecture Notes
in Electrical Engineering 739, https://doi.org/10.1007/978-981-33-6385-4_15
158 L. Z. Poh et al.

spent on road, the higher the traffic congestion in that area. This creates a chain effect
that further delays and causes aggravation for other drivers.
Recognizing the challenges associated with the increased number of vehicles,
many researchers have tried to solve the traffic congestion problem and satisfy the
huge parking lot demand. With technology such as sensor (loop or ultrasonic sensor)
and ticket or e-payment systems, parking information such as availability of free
parking space can be obtained real-time. This provides an opportunity to implement
a dynamic pricing based smart parking solution. By using dynamic pricing, the
parking operator can offer flexible price regulation based on different time periods
to optimize their revenue and at the same time increase parking space usage rate.
In this paper, we present a dynamic pricing model that regulates the parking price
based on vehicle volume and traffic congestion rate. We monitor dynamic pricing
with different pricing schemes and observe how they are utilized in different settings
to bring a higher reward to traffic congestion and revenue for the parking operators.
The dynamic pricing model can forecast traffic congestion and vehicle volume and
disperse vehicle flows that leads to traffic congestion. During peak hours at certain
time of the day, the vehicle flows are dispersed to non-peak hours to increase the
parking usage rate and reduce traffic congestion by suppressing driver’s visit to the
specific areas. This is achieved by price regulation through price reduction in terms
of parking fee rebate.
The dynamic pricing model which provides traffic congestion forecast and parking
lot availability enables time-saving in parking search. Drivers can go straight to the
parking area with lower parking fee (lower parking usage). Within the pre-defined
parking area, traffic congestion can be alleviated because less vehicles will cruise
for parking lot around that area. This in turn reduces carbon dioxide emission and
increases mobility efficiency.
The rest of this paper is organized as follows. Section 2 reviews related work
such as dynamic pricing and smart parking system. Section 3 defined the proposed
method. Section 4 concludes and discuss future improvements.

2 Literature Review

2.1 Dynamic Pricing

Dynamic pricing is a pricing strategy that has a huge influence in our society because
e-commerce has become typical carrier of business model. Every trade-off can be
made by the Internet and it has spared many physical expenses and gave rise to an
easy entry to the market. Nowadays, many researchers focus on dynamic pricing in
e-commerce due to the availability of big data that makes user behaviour visible.
Karpowicz and Szajowski [1] presented four pricing strategies for e-commerce
and they are (1) time-based pricing strategy; (2) market segmentation and limited
rations strategy; (3) dynamic marketing strategy; (4) the comprehensive use of above
Dynamic Pricing for Parking System Using Reinforcement Learning 159

three types. On the other hand, Chen and Wang [2] introduced a dynamic pricing
model for e-commerce based on data mining. The model was composed of three
bottom–up layers: data layer, analytic layer and decision layer.
In a multi-agent environment, the optimal pricing policy of agent depends on
the pricing policies of the other agents [3]. Han et al. [3] proposed a multi-agent
reinforcement learning which integrates the observed objective actions as well as
the subjective inferential intention of the opponents. Pan et al. [4] proposed a new
continuous time model with price and time sensitive demand to consider dynamic
pricing, order cancellation ration and different quality of service (QoS) levels in web
network.
Chinthalapati et al. [5] proposed reinforcement learning (RL) as a tool to study
price dynamics in an electronic retail market. The proposed method consisted of two
competing sellers, price sensitive and lead time sensitive customer. They considered
two representative cases: (1) no information case and (2) partial information case.
Ujjwal et al. [6] proposed a bargaining agent which used genetic algorithm for imple-
menting dynamic pricing on internet. Online bargaining is a “win–win” situation for
both the seller and buyer because mutually agreed deal price is higher than the seller’s
reserved price but lower than the buyer’s reserved price.
The authors in [7] proposed Pareto-efficient and subgame-perfect equilibrium
and offered a bounded regret over an infinite horizon, where regret was defined
as the expected cumulative profit loss as compared to the ideal scenario with a
known demand model. They considered an oligopoly dynamic pricing under demand
uncertainty but assumed that all sellers had the same marginal cost.
Another study in [8] investigated pricing strategy of firms in the context of uncer-
tain demand. According to the study, reference prices and the price of competition
were the two factors that affected demand dynamics. Simulations results demon-
strated that firms could lower the volatility of their price path if they gathered and
processed the information about consumers and competitors because this gave a
greater control over uncertainty than ever before.
Wang [9] proposed a dynamic pricing model of the retailer in the case of
supply exceeding demand vice versa. The study listed the equilibrium conditions
for the optimal dynamic pricing strategies and derived the optimal dynamic pricing
strategies.

2.2 Smart Parking Solutions

Parking solutions integrate interdisciplinary knowledge for successful large-scale

deployment. Recent research like the 5G network also makes the Machine-to-
Machine (M2M) communication in real time practicable. There are several review
papers that provide a good viewpoint to the recent smart solutions using different
technology. In [10], smart parking solutions are classified into three macro-
theme according to the goals of the research fields: information collection, system
deployment, and service dissemination.
160 L. Z. Poh et al.

The study in [11] proposed a smart parking system that was based on intelligent
resource allocation, reservation and pricing. The system offered guaranteed parking
reservation with the lowest possible cost and searching time for drivers and the highest
revenue and resource utilization for parking manager. Mitsopoulou and Kalogeraki
[12] proposed a crowdsourcing system known as ParkForU that aimed to find the
available and most suitable parking options for users in a smart city. It was a parking-
matching and price-regulator algorithm that allow user to specify their destination
along with a set of preferences regarding price and distance (from the parking lot to
their destination), as well as a maximum number of results they wish to receive.
In [13], Nugraha and Tanamas proposed a dynamic allocation method to reserve
parking lot using Internet application. The method reduced the need for the driver to
explore the entire parking spot by finding the vacant parking lot and performed the
reservation for the driver. They used an event-driven schema allocation at the event
of a vehicle arrived at the parking lots gate to maintain parking lot utilization level.
The authors in [14] proposed a ridesharing system with two matching algorithm
that followed single and dual side search paradigm to offer more options on pick-
up time and price, so that the travelers could choose the vehicle that matches their
preferences best. The baseline algorithm of ridesharing matching algorithms inserted
ridesharing request to a kinetic tree and it returned all possible pairs of pick-up time
and price that did not dominate each other.
Jioudi et al. [15] proposed dynamic pricing policy that changes prices propor-
tionally to the arrival rates on each parking and therefore reduces congestion and
eliminates the driver’s preference for some parking. They analyzed the parking time
under near-real-life conditions using the Discrete Batch Arrival Process (D-BMAP).
The drivers arriving according to D-BMAP were selected for service in random order
(ROS) according to their parking time.
In [16], the authors aim to improve the rotation of attractive spot and set a usage-
based parking assignment via appropriate incentive which was equivalent to using a
strategic pricing to control parking dwell time and improve traffic conditions in the
target area. Improving the utilization of the limited parking capacity in high demand
areas by reducing the number of long-term parking spots combined with suitable
measures allowing short-term parking are elemental.
The authors in [17] proposed two prediction models for qualitative and quantitative
improvement of parking availability information. Quality issue refers to fixed interval
to updated information and network delay between parking sensors and data server.
Quantity issue refers to non-smart parking lots which are unmonitored. A future
availability prediction model was introduced by learning the tendency of parking
and changes of occupancy rate from historical data for quality improvement. They
also proposed an availability prediction model for non-smart parking which were
unequipped with sensors based on the assumption that prediction target’s occupancy
rate could be estimated with occupancy rate data of neighbor smart parking located
in nearby for quantity improvement.
Recent research above focuses on reservation or price regulation under real time
algorithm and feedback the result to user of smart parking system. For example,
Dynamic Pricing for Parking System Using Reinforcement Learning 161

iParker [11] uses real time reservation with share time reservation to find the avail-
able and most suitable parking options for user while ParkForU [12] uses parking-
matching and price-regulator algorithm, parking operator was notified after driver’s
selection and adjust their price dynamically to attract driver. Reservation-based
parking system [13] reserves vacant parking slot by calculating driver’s ETA and
dynamically reallocate parking slot when certain car arrived without reservation.
Jioudi et al. [15] uses dynamic pricing policy that changes prices proportionally to
the arrival rates. Jioudi et al. [16] uses area zonificaion to encourage short and mid-
term parking dwells by allowing them to park in the most attractive zones permit
to increase short-term user. Kim and Koshizuka [17] use two prediction models
for qualitative and quantitative improvement of parking availability information. In
this paper, we planned to use dynamic pricing and reinforcement learning to forecast
future arrival rate and traffic volume to define a satisfactory discount scheme to maxi-
mize parking occupancy usage and alleviate traffic congestion. With the proposed
system, the simulation can be designed to different scenario with number of parking
operator, its pricing policy and the traffic volume in the parking area. This had make
the proposed method become reliable.

3 Proposed Method

This study proposes a Reinforcement Learning (RL) model for training the dynamic
pricing mechanism. The proposed model performs as an environment in the parking
industry market. In this environment, a parking operator acts as the player with the
other opponents (the other parking operator). The reward is the number of vehicles
that park in the player’s parking area and the revenue obtained from parking.
Unlike recent smart parking research that focus on reservation for driver and price
regulation by parking provider, we focus on smoothening the regression distribution
model of the historical parking occupancy rate by increasing the occupancy rate at
non-peak hours. According to [16], the driver behaviour can be assumed and captured
using multinomial log-linear regression, also known as multinomial Logit model.
The sales process can be modelled as a Stackelberg game between the retailer
and consumers. It is a two-stage sequential game where the retailer moves first by
setting the prices and then customers follow by making their purchase decisions
[9]. In this paper, the parking operator moves first by setting the prices and this is
followed by the driver who makes his/her purchase decision whether to park the
vehicle. The concept of sub-game perfect equilibrium is also practiced because the
strategy of parking operator and driver constitute the best response to optimize their
expectation. There is a Pareto efficiency between the parking operators because
no individual or preference criterion can be better off without making at least one
individual or preference criterion worse off in this parking environment with parking
demand (Fig. 1).
162 L. Z. Poh et al.

Fig. 1 Expected output of regression model after dynamic pricing

3.1 Architecture

Consider a district parking area containing multiple parking operators with pre-
defined dynamic pricing policies, the RL model is constructed with entity and compo-
nent to perform each single task to fulfil the notation and assumption in this paper.
Below we describe the entities and modules in the proposed smart parking solution:
1. Parking Operator
An entity who provides parking service. Parking operator is defined with its
capacity, building type, price rate, and location. It performs as a player in this
RL environment.
2. Pricing Engine
Fetch the parking utility information including pricing policy used and current
price rate and discount range. The engine run and design new price rate for next
state according the pricing policy predefined such as arrival-time-dependent
pricing (ATP), usage-aware pricing (UAP) [16] and flat rate (a fixed price in a
day).
3. Driver
An entity who seeks a parking space. The drivers are smartphone users who use
access the smart parking application to get discount information of that parking
area. The driver is defined with his/her current location, destination and pref-
erences between price and distance (described as price-focused, balanced and
distance-focused) [12]. The driver preferences are denoted as decimal number
such as 0.7 to price and 0.3 to distance means the driver make decision tend to
lower price parking area.
4. Reward Module
The module to calculate the reward for state transition considering historical
data, predicted reward with and without dynamic pricing, and make comparison
to provide the final reward for each state of episode. In this module, the reward
is normalized with player capacity and discounted price to avoid bias.
5. Arrival Process
Batch arrival prediction for interval of 1 h using Auto Regressive Integrated
Moving Average (ARIMA). It learns the tendency of parking and changes of
Dynamic Pricing for Parking System Using Reinforcement Learning 163

occupancy rate from historical data then forecast future parking occupancy.
This module will input the weekday and period section of episode and return
forecasted occupancy.

3.2 Notation and Assumption

In this study, we assume a pricing scheme for both the player and opponents. The
pricing scheme includes the price range of parking fee for the driver. Both player and
opponents should determine the total volume of parking lot, and also the maximum
and minimum parking fees at the start of the simulation. To model the relationship
between vehicle flow and the traffic congestion, the following assumptions are made:
1. Episode is defined as weekly incremental, each episode consists of 7 days.
2. Problem formulated for parking operators: maximize parking usage, maximize
driver satisfaction and maximize global revenue.
3. Parking operators with different pricing policies: Arrival-time-dependent
pricing (ATP), Usage-aware pricing (UAP), progressive pricing, discount on
flat rate, season pass, incentive pricing.
4. Pricing engine takes action three times per day, make three section such as
morning, afternoon and night.
5. Estimate parking market needs by calculating history vehicle flow as supply
and demand rate.

3.3 Reinforcement Learning Mechanism

The proposed model follows the general RL mechanism as an outer layer. The envi-
ronment receives the environment variables such as vehicle flow and traffic conges-
tion rate. The player learns to increase its reward by taking an action from the Q-table.
An epsilon value is used to increase exploration to avoid sticking in local optimum.
In the reward function, the environment entity calculates the in and out flow of
the player and opponents. With the calculated in and out flows, the number of drivers
learn to recognize their parking area with their distance and price preference. At the
same time, the player learns to alleviate traffic congestion by increasing their parking
revenue. Meanwhile, the drivers also learn to choose the parking area. Algorithm 1
shows the procedure of the reward function.
164 L. Z. Poh et al.

Algorithm 1:
(1) Input: batch arrival rate of in and out flow of vehicle
(2) Input: player’s current state (occupancy and discounted price)
(3) Input: opponents’ current state (occupancy and discounted price)
(4) Initialize reward, in and out flow of vehicle to 0
(5) Foreach hour in period section
(6) Calculate in and out flow of this hour
(7) Calculate reward by referring vehicle in / out flow and discounted price
(8) Update occupancy of player and opponent

3.4 Experiments and Results

We assume there are one player and one opponent in the environment. The action
space is set to 4 which means the parking fee will be discounted between 0 and 3.
Each episode goes through seven days with three section. The morning section will
start from 5 am to 12 pm, afternoon section from 12 to 5 pm and evening section
from 5 pm to 4 am. In every section, the player will make an action to select its price
regarding the action space and loop through the period’s hours to get the arrival and
exit rate from the Arrival Module. The in and out vehicle volume of player and its
opponents will be counted based on driver’s decision by Driver Module. The reward
is calculated by vehicle in volume, player’s revenue and traffic condition.
In this experiment, the driver’s preference ratio are set to (price, distance). For
price-focused driver, the preference is (0.7, 0.3), while it is (0.5, 0.5) for balanced
driver and (0.3, 0.7) for distance-focused driver. The driver’s parking fee is counted
based on his/her arrival time by Pricing Engine. Each action, rebate will be applied to
player parking fee which attracts driver to park at the player’s parking area (Fig. 2).
From the simulation result, the reward reaches its optimal value at 20 thousand
episodes. The episode reward is normalized using the discounted parking fee, which
means less discount, the higher the reward return and profit of parking operator. With
regulating the price of parking lot, decreasing the vehicle flow on the peak hour by
increasing the rebate on non-peak hour which gently the daily parking occupancy
and decrease traffic congestion on the peak hours. From the result, we also can see the
proposed method decreases the traffic congestion and increases parking occupancy.
However, it still needs more effort to relate the simulation to real environment such
as classifying the drivers as long-term, medium-term or short-term visitors.

4 Conclusion

In this paper, a dynamic pricing model for parking industry is presented for traffic
congestion forecast and parking price regulation. Firstly, the features of traffic
Dynamic Pricing for Parking System Using Reinforcement Learning 165

Fig. 2 Simulation result with 50 thousand episodes

congestion and parking revenue are represented. Then a dynamic pricing model for
parking industry is proposed and trained by reinforcement learning. It is shown that
the dynamic pricing is influential to the optimal solution. The new concepts intro-
duced in this paper is the smart parking system with dynamic pricing to increase the
revenue of parking operator by providing a discount scheme to maximize the parking
occupancy usage and alleviate the traffic congestion by considering traffic volume as
an objective. This proposed method is not handling real-time parking but a forecasting
method on vehicle arrival rate and parking occupancy. With the outputted discount
scheme to maximize revenue of parking operator and alleviate traffic congestion,
the discount scheme can be promoted to frequent visit driver or scheduled visitor.
However, there are still problems to be solved for the model, because currently the
research focuses on predefined parking operator with pre-defined details of parking
operator and its opponents such as location and occupancy. We planned to increase
the reliability of the dynamic pricing algorithm to fit with multiple variety type of
parking operator such as the building type, occupancy and pricing scheme.

Acknowledgements This work was supported by the TM and TM R&D [Project SAP ID:
MMUE/180024]

References

1. “(PDF) Double optimal stopping times and dynamic pricing problem (2020) Description
of the mathematical model ResearchGate. https://www.researchgate.net/publication/225715
166 L. Z. Poh et al.

898_Double_optimal_stopping_times_and_dynamic_pricing_problem_Description_of_the_
mathematical_model Accessed 20 Feb 2020
2. Chen Y, Wang F (2009) A dynamic pricing model for e-commerce based on data mining. In:
2009 Second international symposium on computational intelligence and design. https://doi.
org/10.1109/ISCID.2009.99
3. Han W, Liu L, Zheng H (2008) Dynamic pricing by multiagent reinforcement learning. In:
2008 International symposium on electronic commerce and security, August 2008, pp 226–229.
https://doi.org/10.1109/ISECS.2008.179
4. Pan W, Yue W, Wang S (2009) A dynamic pricing model of service provider with different
QoS levels in web networks. In: 2009 International symposium on information engineering
and electronic commerce, May 2009, pp 735–739. https://doi.org/10.1109/IEEC.2009.160
5. Chinthalapati VLR, Yadati N, Karumanchi R (2006) “Learning dynamic prices in MultiSeller
electronic retail markets with price sensitive customers, stochastic demands, and inventory
replenishments. IEEE Trans Syst Man Cybernet Part C (Appl Rev) 36(1):92–106. https://doi.
org/10.1109/TSMCC.2005.860578
6. Ujjwal K, Aronson J (2007) Genetic algorithm based bargaining agent for implementing
dynamic pricing on internet. In: 2007 IEEE symposium on foundations of computational
intelligence, April 2007, pp 339–343. https://doi.org/10.1109/FOCI.2007.372189
7. Zhai Y, Zhao Q (2016) Oligopoly dynamic pricing: a repeated game with incomplete infor-
mation. In: 2016 IEEE International conference on acoustics, speech and signal processing
(ICASSP), Shanghai, March 2016, pp 4772–4775. https://doi.org/10.1109/ICASSP.2016.747
2583
8. Wu L-LB, Wu D (2016) Dynamic pricing and risk analytics under competition and stochastic
reference price effects. IEEE Trans Ind Inf 12(3):1282–1293. https://doi.org/10.1109/TII.2015.
2507141
9. Wang Y (2016) Dynamic pricing considering strategic customers. In: 2016 International confer-
ence on logistics, informatics and service sciences (LISS), Sydney, Australia, July 2016, pp
1–5. https://doi.org/10.1109/LISS.2016.7854471
10. Lin T, Rivano H, Le Mouel F (2017) A survey of smart parking solutions. IEEE Trans Intell
Transport Syst 18(12):3229–3253. https://doi.org/10.1109/TITS.2017.2685143
11. Kotb AO, Shen Y-C, Zhu X, Huang Y (2016) iParker—a new smart car-parking system based on
dynamic resource allocation and pricing. IEEE Trans Intell Transport Syst 17(9):2637–2647.
https://doi.org/10.1109/TITS.2016.2531636
12. Mitsopoulou E, Kalogeraki V (2018) ParkForU: a dynamic parking-matching and price-
regulator crowdsourcing algorithm for mobile applications. In: 2018 IEEE International confer-
ence on pervasive computing and communications workshops (PerCom Workshops), Athens,
March 2018, pp 603–608. https://doi.org/10.1109/PERCOMW.2018.8480321
13. Nugraha IGBB, Tanamas FR (2017) Off-street parking space allocation and reservation system
using event-driven algorithm. In: 2017 6th International conference on electrical engineering
and informatics (ICEEI), Langkawi, November 2017, pp 1–5. https://doi.org/10.1109/ICEEI.
2017.8312456.
14. Chen L, Zhong Q, Xiao X, Gao Y, Jin P, Jensen CS (2018) Price-and-time-aware dynamic
ridesharing. In: 2018 IEEE 34th International conference on data engineering (ICDE), Paris,
April 2018, pp 1061–1072. https://doi.org/10.1109/ICDE.2018.00099
15. Jioudi B, Sabir E, Moutaouakkil F, Medromi H (2019) Estimating parking time under batch
arrival and dynamic pricing policy. In: 2019 IEEE 5th World forum on internet of things (WF-
IoT), Limerick, Ireland, April 2019, pp 819–824. https://doi.org/10.1109/WF-IoT.2019.876
7179
16. Jioudi B, Sabir E, Moutaouakkil F, Medromi H (2019) Congestion awareness meets zone-based
pricing policies for efficient urban parking. IEEE Access 7:161510–161523. https://doi.org/
10.1109/ACCESS.2019.2951674
17. Kim K, Koshizuka N (2019) Data-driven parking decisions: proposal of parking availability
prediction model. In: 2019 IEEE 16th international conference on smart cities: improving
quality of life using ICT & IoT and AI (HONET-ICT), Charlotte, NC, USA, October 2019, pp
161–165. https://doi.org/10.1109/HONET.2019.8908028