Smart Terminal Management: A Real-Time Congestion Detection

and Resource Allocation System Using Apache Flink, DBSCAN,

and Kalman Filters for Public Transport

Submitted by:

Leader: Cabral, Justin Hendrick

Member 1: Tomaquin, Julius Jr.

Member 2: Cruz, Julian Riley

Member 3: Razon, Gerardo III

1.0 Proposed Thesis Title:

“Smart Terminal Management: A Real-Time Congestion Detection and Resource

Allocation System Using Apache Flink, DBSCAN, and Kalman Filters for Public Transport”

2.0 Thesis Concept:

The thesis focuses on the development of an application intended to assist both

commuters and operators concurrently. This application aims to deliver real-time alerts
concerning congestion levels at public vehicle terminals. The fundamental premise is that
crowded terminals result in extended waiting times, consequently delaying commuters
from returning home in a timely manner. To mitigate this issue, the system will notify
terminal management when crowding is detected, thereby prompting the deployment of
additional public utility vehicles. Conversely, if a terminal is not congested, fewer drivers
will be required. The application will take advantage of real-time stream processing
algorithms and methodologies, including Apache Flink and Apache Storm, Streaming
SQL, Kalman Filter, Map Matching Algorithms, and Density-Based Spatial Clustering
(DBSCAN), Geofencing Algorithms, Windowing Techniques, Complex Event Processing
(CEP). By employing these technologies, the application will provide users and operators
with timely updates, potentially reducing congestion during peak hours. Data for this
application will be obtained through GPS live tracking. Potential data sources may
include government agencies involved in transportation as well as companies that possess
historical data regarding the aggregate number of individuals passing through various
terminals and routes.

3.0 General Objectives:

The primary goal of the thesis is to create an application that uses real-time alerts
to assist operators and commuters in controlling terminal congestion. In addition to algorithms
like the Kalman Filter, DBSCAN clustering, and geofencing, the application will make use of
real-time stream processing techniques like Apache Flink, Apache Storm, and Complex Event
Processing (CEP). The students' aim is to develop an application that offers precise prompts and
alerts regarding the level of terminal congestion. This will allow terminal operators to cut
resources as needed or deploy more public utility vehicles. The program seeks to improve
commuter experiences and streamline terminal operations by incorporating GPS live tracking
and predictive analytics, promising an effective transportation system.
3.1 Key Specific Objectives:

1. Development of Backend and Frontend: Develop a cross-platform mobile application

using React Native for the frontend and implement backend functionalities using Node.js
and MongoDB to store and process GPS data, commuter density information, and
congestion reports. Use Expo for testing, deployment, and iterative development.
2. Data Collection and Modeling: Gather data on terminal occupancy levels and commuter
movements through GPS live tracking, historical records, and government transportation
data. Model terminal congestion patterns using real-time stream processing algorithms,
including DBSCAN clustering and Kalman Filters.
3. Algorithm Integration: Implement core algorithms such as Apache Flink and Apache
Storm for stream processing, DBSCAN for clustering commuter density, geofencing for
terminal monitoring, and Complex Event Processing (CEP) for event detection and
congestion alerts.
4. System Testing and Validation: Conduct unit testing, integration testing, and performance
validation of real-time congestion detection. Compare system outputs against actual
terminal congestion conditions and expert evaluations to ensure accuracy and reliability.
5. Deployment and User Feedback: Deploy the application in a pilot phase with select
terminals to collect real-world user feedback from commuters and operators. Use this
feedback to refine the app’s functionalities and optimize the congestion alert and resource
allocation mechanisms.

4.0 Target Audience:

The target audience for this study would include:

1. Commuters: Individuals who frequently rely on public utility vehicles to travel,

especially those who experience long waiting times and inconvenience due to
overcrowded terminals. This app aims to improve their daily commuting experience by
reducing delays.
2. Public Utility Vehicle Operators: Drivers and operators who manage public utility
vehicles and rely on demand-based deployment for efficient operations. The app will help
optimize resource allocation and reduce idle times.
 3. Terminal Management: Administrators or personnel responsible for managing public
vehicle terminals. The system will aid them in real-time monitoring of congestion levels
and provide actionable insights to manage terminal operations effectively.
4. Local Government Units (LGUs) and Transportation Agencies: These organizations can
use the application to improve the management of public transportation networks and
enhance urban mobility, particularly during peak hours or special events.
5. Transportation Companies: Firms that own and operate public utility vehicles, seeking to
optimize operations, reduce costs, and improve commuter satisfaction through real-time
congestion management.
6. Government Agencies: Agencies such as the Department of Transportation (DOTr) or
local traffic management bureaus, which can benefit from insights into commuter flow
and terminal congestion trends for policymaking and infrastructure planning.
7. Public Utility Vehicle Organizations: Associations or unions representing drivers and
operators who can use the app to improve coordination and vehicle allocation during
peak and off-peak hours.

5.0 Algorithm Specification

The application will focus on real time processing algorithms because it allows for continuous,
real-time data which is crucial for managing the congestion of public vehicle terminals, and this
is why.

Real-time Stream Processing frameworks include Apache Flink and Apache Storm. These two
will serve as the backbone of handling and processing of the data streams. These frameworks
will ensure low-latency data, enabling it to provide timely updates and alerts to both operators
and commuters.

Streaming SQL will complement the two frameworks by ensuring efficient querying of data and
accumulation of data. It also analyzes the commuter density trends at a specific terminal.

Data preprocessing algorithms will play the role of ensuring the accuracy and the reliability of
the data collected, meanwhile the Kalman Filter, a powerful tool for smooth GPS data. This will
be employed to reduce inaccuracies caused by other factors such as signal interference.
Additionally, the map matching algorithm will align GPS data with the students created digital
maps, ensuring accurate tracking of commuter positions.

Density-Based Spatial Clustering (DBSCAN) is used for analyzing congestion levels, then
clustering algorithms will be applied. This will be supported by Geofencing Algorithms, it will
establish the boundaries around terminals, and track commuter entry and exit. It will dynamically
calculate the terminal occupancy levels providing actions to take for the operators.

Windowing techniques will sort the data into manageable intervals, this will allow the system to
compute for the trends such as entry and exit over periods of time. Complex Event Processing
(CEP) will also identify patterns and trends, such as increase in commuters or unexpected public
utility vehicle delays.

6.0 Scope and Limitations

Scope:

The scope of the study includes:

1. The design and implementation of a mobile application to reduce terminal congestion for
both operators and passengers is the main focus of this study. The application uses
geofencing, real-time stream processing technologies (such Apache Flink and DBSCAN),
and GPS live tracking to deliver real-time notifications. It focuses on terminals in cities
and includes functions like dynamic resource allocation, predictive analytics, and
congestion monitoring. Live GPS streams, historical commuting data, and government
transportation agencies are some of the data sources. Using React Native for
cross-platform development and Node.js for backend features, the application is made for
both the iOS and Android operating systems.
2. The application's design prioritizes efficient performance and user-friendly functionality
to meet the needs of terminal operators and commuters alike. In order to assist
commuters in making well-informed travel selections, the app offers real-time updates on
terminal congestion, anticipated wait times, and available public utility vehicles. It
provides dynamic tools for terminal operators to manage vehicle deployment, find
underutilized resources, and deal with peak-hour congestion. By decreasing delays and
improving rider happiness overall, this dual-purpose design seeks to establish a more
coordinated and effective public transportation system.
3. The application will automate real-time updates to both commuters and operators
ensuring an efficient solution to terminal congestions. It collects GPS data from the
devices that the commuters and public utility vehicles and processes instantly using
frameworks such as Apache Flink. Filtering algorithms like Kalman Filters will refine the
data that is being retrieved. The students can provide real time updates by using
DBSCAN to identify the density of such terminals and geofencing to localize alerts.
When congestion is detected, automated notifications inform commuters about terminal
 conditions and vehicle availability, while operators receive alerts to deploy additional
resources.
4. The application will feature a robust feedback system to gather input from both
commuters and terminal operators, ensuring continuous improvement and adaptation to
user needs. Feedback will be collected through various channels, including in-app
surveys, issue reporting tools, and a suggestion box, allowing users to express
satisfaction, report issues, and propose enhancements.Advanced analytics, particularly
Natural Language Processing (NLP) algorithms, will analyze user feedback to identify
common themes and sentiments. This analysis will help prioritize improvements based on
user expectations. The system will also implement adaptive features to efficiently
integrate user-driven changes into periodic updates, enhancing app functionalities and
user interface design.Engaging users through this feedback mechanism fosters a sense of
community and ownership over the application. By regularly communicating how
feedback has influenced updates, the system builds trust and encourages ongoing
participation. This iterative process ensures the application remains relevant and effective
in addressing terminal congestion challenges in a user-centric manner.

Limitations:

The limitations of the study includes:

1. The application is dependent on comprehensive and reliable data sources which are
insufficient or incomplete. These can pose significant challenges because the accuracy of
real-time notifications hinges the ability of robust datasets such as GPS signals and
historical commuting patterns. When such data is limited or unavailable, the system may
generate delayed or inaccurate alerts, affecting its effectiveness in managing terminal
congestion.
2. The reliance on stable internet connection presents another problem. Real-time data
collection, processing, and notification depends on the consistency of the user's internet
connection which may not be available. When the internet connection is unstable, the
system's ability to collect GPS data, analyze terminal conditions, and notify the users
timely alerts can be compromised. This can lead to delays, inaccurate reporting, or to
miss the opportunity to optimize public utility vehicle deployment. Additionally, abrupt
network disruptions during peak periods may make terminal inefficiencies worse by
depriving operators and commuters of vital updates.
3. Privacy and security concerns with the collection and processing of sensitive user data,
particularly the location information. The application relies on real-time GPS tracking
 which is essential for providing accurate alerts. Users may be uncomfortable with their
location data being continuously monitored, tracked, and stored, particularly when such
information can reveal detailed patterns of behavior, including their travel routes,
frequented terminals, and movement habits. Real-time GPS data can expose not only a
user’s exact location but also their daily routines, preferred transportation routes, and
potentially other personal details that could be exploited if mishandled.
4. Reliance on predictable congestion patterns, which can be disrupted by a number of
external and environmental causes. Under typical operational circumstances, the system
is made to track and examine terminal activities. Unexpected occurrences, however, such
severe weather, natural catastrophes, road closures, major traffic accidents, or even public
protests, can cause sudden and notable departures from normal congestion patterns.
Because they introduce anomalies that do not match prior data or the predictive models
used, these issues make it difficult for the system to anticipate or manage efficiently. In
areas like the Philippines, where they occur frequently, natural calamities like typhoons,
floods, and earthquakes present a special difficulty. Transportation systems might be
severely interrupted during these periods, with some terminals shutting down entirely and
others being overcrowded. The chaotic and unpredictable nature of these situations may
be too much for the application's algorithms, which rely on previous data and real-time
GPS tracking, to handle, which could result in notifications that are erroneous or
unnecessary. For example, a terminal may seem less crowded after a flood because
commuters are unable to get to it because of impassable roads, not because demand has
dropped.

7.0 Testing:

● Unit testing - The task involves verifying the accuracy and efficiency of GPS data
collection, geofencing functionality, and congestion detection algorithms, including
DBSCAN clustering and Kalman Filters.
● Integration testing - The test involves ensuring the correct transmission, processing, and
display of GPS data on the mobile application using React Native and backend services
like Node.js and APIs.
● Functional testing - Ensure all features, including congestion alerts, vehicle deployment
recommendations, and operator notifications, function properly and that real-time data
processing triggers accurate and timely responses.
● Usability testing - The project involves gathering feedback from terminal operators and
commuters to evaluate the app's interface design and ease of use, aiming to improve
clarity and simplicity in providing actionable information.
 ● Performance testing - Assess system responsiveness during peak hours and real-time
processing capabilities of algorithms like Apache Flink under high data influx.
● Security testing - The task involves identifying potential security vulnerabilities in GPS
data collection, storage, and transmission, as well as ensuring encryption protocols and
data privacy regulations are upheld.
● Acceptance testing - The application must meet user needs and study objectives,
enhancing terminal efficiency and reducing wait times, and undergo stakeholder testing
for final approval.
● Error handling - The application is tested for its ability to handle incomplete, incorrect, or
conflicting data inputs, and its stability and error reporting are evaluated through
simulation scenarios.
● Longitudinal testing - The application's performance and reliability are evaluated over
time to assess its accuracy and consistency in predicting congestion and assessing its
adaptability to changes in commuting patterns and terminal operations.
● Anomaly detection testing - The study aims to test the robustness of anomaly detection
algorithms by simulating unusual scenarios like sudden terminal congestion spikes or
drops.

Review of Related Literature

The use of machine learning technologies in real-time congestion detection has been our
primary solution. This literature review aims to highlight the findings from relevant studies that
are aligned in our proposed thesis titled Smart Terminal Management: A Real-Time Congestion
Detection and Resource Allocation System Using Apache Flink, DBSCAN, and Kalman Filters
for Public Transport.

Real-Time Traffic Congestion Detection Technology in Intelligent Transportation Systems

Author: Shanwen Li

The study focuses on intelligent transportation systems that use real-time congestion
detecting technology. It gathers traffic data in real time, including vehicle speed, traffic flow, and
density, using a range of sensors, such as GPS, ground induction coils, and video surveillance
cameras. The study then incorporates the gathered data to increase the precision of congestion
forecasts. Optimizing traffic flow, lowering congestion, and raising the general effectiveness of
urban transportation systems are the primary goals of the study. The study emphasizes how
real-time traffic congestion monitoring can greatly alleviate the problems associated with urban
transportation, resulting in fewer traffic accidents, less pollution, and more sustainable
transportation systems. The goal of the project is to improve traffic management
decision-making and congestion prediction accuracy by utilizing machine learning models, such
as clustering and prediction algorithms. This technique correlates well with the goals of the
students thesis, as both studies focus on real-time data analysis and resource allocation to
increase operational efficiency in transportation systems.

Deep Learning-Based Congestion Detection at Urban Intersections

Authors: Xinghai Yang, Fengjiao Wang, Zhiquan Bai, Feifei Xun, Yulin Zhang, and Xiuyang
Zhao

The study combines object identification, optical flow analysis, and traffic condition
discrimination algorithms to present a deep learning-based approach for identifying traffic
congestion at urban crossroads. The Lucas-Kanade (LK) optical flow method is used to
determine vehicle speeds, and the YOLOv3 object detection model is integrated for vehicle
identification in the ROI. The technology is able to accurately assess the congestion state of the
intersection by analyzing these speeds using a traffic state discrimination algorithm. The study
demonstrates the method's strength and efficacy, showcasing a comprehensive anti-interference
capability appropriate for practical uses. The algorithm's capacity to accurately identify vehicle
speeds and traffic circumstances is confirmed by the testing findings, which helps to improve
traffic management. An important development in congestion detection technology is the
system's innovative application of optical flow algorithms for motion analysis and deep learning
for object detection. The use of real-time data and sophisticated algorithms for congestion
management is in line with the goals of the student's thesis. The application of real-time stream
processing and clustering techniques to monitor and control terminal congestion is mirrored in
the usage of vehicle speed detection and traffic state discrimination in the student's thesis.

Optimizing Public Transport Services using AI to Reduce Congestion in Metropolitan

Areas
Author: Ivan Petrovich Kozlov

In order to address major urban mobility issues, the project investigates the use of
artificial intelligence (AI) to improve public transportation networks and reduce traffic in urban
areas. The study finds a number of cutting-edge AI-driven solutions that improve commuter
experiences and public transportation efficiency through a thorough analysis of the literature,
case studies, and expert opinions. The study emphasizes how AI may be used for real-time traffic
monitoring, allowing timetables and routes to be dynamically adjusted depending on real-time
traffic data, thereby reducing delays and avoiding clogged areas. This supports the focus on
adaptive resource allocation and real-time congestion monitoring in the students thesis. One
significant innovation that sticks out is the incorporation of demand forecasting driven by AI.
Transportation providers can forecast commuter demand with accuracy by examining contextual
and historical data, including events, time patterns, and weather. This makes it possible to scale
the deployment of vehicles dynamically, which improves the commuter experience overall by
cutting down on waiting times and increasing route efficiency. AI-based route optimization,
which uses algorithms to identify the most efficient travel routes by examining environmental
parameters, passenger demand, and congestion levels, is another crucial component. These
techniques mirror the use of stream-processing and clustering algorithms to alleviate terminal
congestion in the students thesis.

Design an Intelligent Real-Time Public Transportation Monitoring System Based on IoT

Authors: Thair A. Salih and Noor K. Younis

The design and implementation of an intelligent real-time public transit monitoring

system using Internet of Things technology are presented in this study. The study's main goal is
to improve the commuter experience by cutting down on wait times and making information on
public transit more easily accessible. A microcontroller with an integrated Wi-Fi module
(ESP32), a mobile application developed on the Blynk IoT platform, and GPS for tracking
real-time bus locations are all used in the development of this system. Passengers may get
real-time information on bus whereabouts, arrival timings, and speeds thanks to the technology.
It determines the distance between the user's location and the closest bus using the Haversine
formula. With its user-friendly UI, the smartphone app shows users the bus's current location on
Google Maps along with the estimated distance and arrival time. The information given to users
is more reliable because the arrival times are established by taking into account the average bus
speed and distance. The deployment of this technology shows how the Internet of Things may be
used to update the infrastructure of public transportation. The study tackles important urban
mobility issues by providing real-time monitoring and precise forecasts, which is in line with
more general objectives of raising commuter satisfaction and enhancing the effectiveness of
urban transportation. Scalability and applicability in various urban contexts can be further
improved by integrating IoT with smart city projects.

Smart Cities: How AI is Revolutionizing Urban Traffic Management

Author: AI Tech Daily
 Insights from the article provide a strong basis for supporting the goals stated in the
thesis. They highlight successful real-world implementations of AI in traffic management,
emphasizing methodologies such as real-time data processing, predictive modeling, and dynamic
optimization—key elements that align seamlessly with the proposed system for terminal
congestion management. These illustrations show how AI has the revolutionary potential to
solve urgent urban mobility problems by offering flexible and effective solutions. The article's
thorough case studies—such as Singapore's predictive congestion management and Los Angeles'
AI-driven traffic light optimization—showcase how AI can evaluate enormous datasets from
sensors, GPS, and cameras to produce quick and significant outcomes. By decreasing traffic,
cutting down on delays, and raising commuter satisfaction, these deployments demonstrate not
only the technological viability of AI-driven systems but also their potential to enhance urban
transportation in general. By using these established methods, the thesis's suggested solution
gains more legitimacy and relevance. Moreover, the parallels between AI’s ability to streamline
traffic flow and the thesis’s aim to optimize public transport routes and schedules highlight a
shared commitment to reducing environmental impacts and improving sustainability. The
suggested use can drastically cut fuel consumption and greenhouse gas emissions by tackling
congestion with real-time modifications and intelligent resource allocation, resulting in a more
sustainable and greener urban ecosystem. The potential influence of the research is highlighted
by this convergence of theoretical frameworks and real-world applications. In addition to
improving terminal congestion management, the project proposes a comprehensive solution that
turns public transportation systems into more dependable, efficient, and user-centered networks
by using successful AI implementations. By paving the way for smarter and more livable cities,
this integration represents a larger effort to address current issues with urban mobility.

Conclusion:
An innovative solution to the persistent problems of managing congestion in public
transportation terminals is offered by this proposed thesis. The program seeks to provide
insightful information and timely notifications that are advantageous to commuters and terminal
operators by leveraging cutting-edge technologies including Apache Flink, DBSCAN clustering,
Kalman Filters, and real-time stream processing. In order to improve commuter experiences,
minimize delays, and expedite terminal operations, the study highlights the significance of
combining real-time GPS tracking, predictive analytics, and dynamic resource allocation. The
thesis properly integrates feedback systems, system testing, and algorithms to guarantee that the
program stays user-friendly and adjusts to users' needs. The study shows its dedication to
developing a dependable and safe system by addressing potential drawbacks, such as the reliance
on dependable data sources, persistent internet access, and privacy concerns. Recognizing
outside influences like natural disasters demonstrates a thorough comprehension of real-world
difficulties and guarantees that the system will continue to function well in erratic circumstances.
The testing phase ensures a thorough examination of the system's resilience by addressing
anomaly detection and performance assessments during peak hours. In addition to adding to the
body of knowledge, this study offers a workable method that might greatly enhance the
administration of urban public transportation. This thesis lays the groundwork for a more
effective, environmentally responsible, and commuter-friendly transportation system by
matching technical advancements with the requirements of both commuters and operators.

References:

Daily, A. T. (2024, November 20). Smart cities: How AI is revolutionizing urban traffic
management. Medium.
https://medium.com/@aitechdaily/smart-cities-how-ai-is-revolutionizing-urban-traffic-managem
ent-abefbdb020aa#:~:text=Detailed%20Analysis%3A%20Smart%20traffic%20lights,adjusting%
20timings%20to%20minimize%20congestion

Kozlov, I. P. (2022, November 12). Optimizing public transport services using AI to reduce
congestion in metropolitan area. International Journal of Intelligent Automation and Control,
2022, 1-12. https://research.tensorgate.org/index.php/IJIAC/article/view/34

Manila LGU to implement “no apprehension during rush hour.” (n.d.). Philippine News Agency.
https://www.pna.gov.ph/articles/1230816#:~:text=The%20rush%20hours%20covered%20are,4%
20p.m.%20to%207%20p.m.

Rivera, K. (2018, May 4). The Filipino commuters’ crisis. INQUIRER.net.

https://opinion.inquirer.net/112930/filipino-commuters-crisis

Salih, T. A., & Younis, N. K. (2021). Designing an intelligent real-time public transportation
monitoring system based on IoT. OALib, 8(10), 1–14. https://doi.org/10.4236/oalib.1107985

Sayed, S. A., Abdel-Hamid, Y., & Hefny, H. A. (2023). Artificial intelligence-based traffic flow
prediction: A comprehensive review. Journal of Electrical Systems and Information Technology,
10(1). https://doi.org/10.1186/s43067-023-00081-6

Statista. (2024, January 29). Time lost during rush hour in Manila, Philippines 2019-2023.
https://www.statista.com/statistics/1262018/philippines-time-lost-during-rush-hour-manila/

Yang, X., Wang, F., Bai, Z., Xun, F., Zhang, Y., & Zhao, X. (2021). Deep learning-based
congestion detection at urban intersections. Sensors, 21(6), 2052.
https://doi.org/10.3390/s21062052
Yuan, Y. (2020). Application of intelligent technology in urban traffic congestion. 2020
International Conference on Computer Engineering and Application (ICCEA).
https://doi.org/10.1109/iccea50009.2020.00157