Human-Centered Machine Learning Implementation in Banking:

Case Study in BRILink (BRI Branchless Banking) Agent Acquisition,

Upgrade, and Activation
2022 IEEE International Conference on Industrial Engineering and Engineering Management (IEEM) | 978-1-6654-8687-3/22/$31.00 ©2022 IEEE | DOI: 10.1109/IEEM55944.2022.9989784

F. I. Mahlidah1, A. K. Sukarno 1, Y. Yustiawan 1, M. D. R. Bakry2

1
BRIBRAIN Department, Digital Banking Development and Operation Division, Bank Rakyat Indonesia (BRI)
2
BRILink Network Division, Bank Rakyat Indonesia (BRI)
([email protected])

Abstract - BRILink is a branchless banking service that agents. Agent acquisition is the process of acquiring
provides quintessential banking transactions to the highly potential customers to become BRILink agents.
unbanked population in Indonesia through human agents. Agent upgrade is the action of rewarding BRILink agents
Due to the important role of agents, the quantity and quality who exceed their target by leveling up their tier. There are
of agents become critical in BRILink operations to achieve three tiers of BRILink agents: pemula agent (first tier),
the expected business performance. Thus, new agent
acquisition as well as agent upgrade and activation are
jawara agent (second tier), and juragan agent (third tier).
essential as endeavors to maintain and improve agent Agent activation is the process of increasing existing
quantity and quality. However, the existing acquisition, agent’s transactions in order to achieve their monthly
upgrade, and activation method is troublesome due its high transaction targets based on the Break Even Point (BEP)
reliance on subjective judgment and non-data-driven formula, where the net income must be at least equal to
approach. Therefore, human-centered machine learning the total expenditure. BRILink Agent Officers (PAB)
solutions are implemented to determine highly qualified and plays a crucial role in acquiring new agents and
potential customers to be acquired, and agents to be developing the existing agents (i.e., activation and
upgraded and activated. Based on evaluation, the upgrade function). Currently, PAB relies predominantly
implemented machine learning solutions significantly can
achieve thousands of agent acquisition and up to 18%
on subjective judgment, intuition, and random selection to
increases of agent upgrade and activation, then increase determine potential new agents to be acquired and
above 13% fee-based income on average. existing agents to be activated and upgraded.
Consequently, many BRILink agents do not perform as
Keywords - branchless banking, machine learning, expected and have relatively low productivity. Currently,
rule-based, software development, change management only 49% of all existing agents reached their BEP target.
To tackle the aforementioned challenges, we develop
an end-to-end Machine Learning (ML) solution to help
I. INTRODUCTION PAB in selecting highly qualified customers to be
acquired and potential existing agents to be upgraded and
BRILink is a branchless banking service, provided by activated. The acquisition function is an ML-based
Bank Rakyat Indonesia (BRI), that serves quintessential solution that utilizes existing BRI customers and BRILink
banking transactions (e.g., transfers, deposits, agents data (e.g., demography, historical transactions and
withdrawals, etc.) to more than 100 million of Indonesia's portfolio). The upgrade and activation function uses a
unbanked population. Customers are served by BRILink rule-based approach based on the business domain
agents, who carry out transactions for customers using knowledge. A web mobile application is developed to
Electronic Data Capture (EDC) or BRILink mobile. On provide a convenient user-interface for PAB to interact
average, BRILink serves more than 100 trillion with the outputs of these functions. In addition, a change
transactions with up to IDR 100 billion revenue per management initiative, in the form of workshops, is held
month. The Financial Services Authority (OJK) has also to provide understanding to PAB regarding the
initiated a branchless banking program that aims to implementation of this new approach, considering that
provide basic savings accounts for all Indonesian citizens this new approach changes their day-to-day workflow.
[1]. As a result, the status of financial inclusion in This solution is expected to improve the existing approach
Indonesia continues to increase, reaching 83.6% in 2021 by utilizing a data-driven approach while reducing the use
from 81.4% in 2020 [2]. of subjective judgment and intuition. Another goal of this
Due to the important role of agents in BRILink daily solution is to increase business profits. In the 6 months
operations and in supporting the growth of financial since the implementation of this solution, the average
inclusion in Indonesia, the quantity and quality of agents fee-based income has increased by up to 13%. The growth
become critical. Thus, new agent acquisition as well as of agent quality to achieve BEP activation to Juragan also
agent upgrade and activation are essential as endeavors to increased by up to 18%.
maintain and improve quantity and quality of BRILink

1348
Authorized licensed use limited to: UNIVERSITAS GADJAH MADA. Downloaded on October 02,2023 at 04:56:27 UTC from IEEE Xplore. Restrictions apply.
 Proceedings of The 2022 IEEE International Conference on Industrial Engineering and Engineering Management (IEEM)

II. METHODOLOGY
Each of the CRISP-DM stages is described as
In the banking industry, machine learning (ML) follows.
techniques have been used extensively in a variety of 1. Business understanding. The main business goal is
situations, including product recommendation, credit risk to provide PABs list of potential customers who are
analysis, and customer churn prediction. The financial suitable to be acquired as new BRILink agents.
industry has been the primary impetus for the Data-driven method is proposed as an improvement
development of this ML technology, perhaps as a result of of the existing intuition-based method.
the risk exposure and the significant amount of cash 2. Data understanding. Customers’ demographic,
generated. K-nearest Neighbor (KNN), Support Vector portfolio, transactional, and geospatial data are
Machine (SVM), Tree-Based Models, Ensembles, utilized to analyze highly potential customers.
Boosting Techniques, and Artificial Neural Networks Exploratory data analysis (EDA) is performed to
(ANN) are some of the most pertinent ML techniques. discover and visualize trends and patterns.
However, the Boosting Technique was chosen as the 3. Data preparation. Data preparation is performed to
pre-eminent solution, both for this classification issue and improve the data data quality and to ensure the data is
when used with structured data [3]. ready to be used for training [6]. Some data
There are three main phases in this project. The first preparation that is performed are missing values
one is the development of an ML-based and rule-based imputation, outliers removal, and feature selection.
engine to perform agent acquisition, upgrade, and 4. Modeling. The ML model is developed by using
activation function. The second one is the development of LightGBM. LightGBM is a Gradient Boosting
a web mobile application to provide a convenient Decision Tree (GBDT) method that utilizes Gradient
user-interface for users (i.e., PABs) to interact with the Based One-Side Sampling (GOSS) and Exclusive
outputs of the aforementioned engines. The third one is a Feature Bundling (EFB) [7]. GOSS is a sampling
change management initiative, in the form of workshops, method that keeps data instances with larger
to provide understanding to users regarding the gradients and omits instances with small gradients.
implementation of this new approach. Each phase will be Meanwhile, EFB is an efficient algorithm that
discussed in the following sections. optimizes a bundling problem to a graph coloring
problem and resolves it by greedy algorithm with
A. Models Development constant approximation ratio. In many machine
learning tasks, LightGBM is an efficient GBDT
ML-based solution is used to develop the agent algorithm that delivers greater accuracy in many
acquisition function while rule-based approach is utilized applications, including peer-to-peer lending [8].
to build the agent upgrade and activation function [4]. Additionally, LightGBM significantly enhances
CRISP-DM (Cross Industry Standard Process for Data parallel learning to process large volumes of data in
Mining) methodology [5] is adopted as the process model accordance with the numerous sector data banks [9].
to develop the ML-based solution. CRISP-DM comprises 5. Evaluation. The developed ML models are evaluated
six sequential stages: business understanding, data using precision, recall, and AUC (Area Under the
understanding, data preparation, modeling, evaluation, ROC Curve) metric. Multiple models are assessed
and deployment. The schema of the CRISP-DM before the final model is selected. Recall measures
methodology is shown in Fig. 1. the fraction of all positive instances that are
successfully retrieved while precision describes the
fraction of all positive instances among the retrieved
positive instances. The formula to calculate recall and
precision is shown in (1) and (2).

𝑇𝑃
𝑅𝑒𝑐𝑎𝑙𝑙 = 𝑇𝑃 + 𝐹𝑁
(1)

𝑇𝑃
𝑃𝑟𝑒𝑐𝑖𝑠𝑖𝑜𝑛 = 𝑇𝑃 + 𝐹𝑃
(2)

Where TP is the number of true positive instances,

FN is the number of false negative instances, and FP
is the number of false positive instances.
6. Deployment. The developed ML model is deployed
in a web mobile application. The development of this
web mobile application is described in the subsequent
section.
Fig. 1. CRISP-DM diagram

1349
Authorized licensed use limited to: UNIVERSITAS GADJAH MADA. Downloaded on October 02,2023 at 04:56:27 UTC from IEEE Xplore. Restrictions apply.
 Proceedings of The 2022 IEEE International Conference on Industrial Engineering and Engineering Management (IEEM)

To develop the agent activation and upgrade function, ensure the developed UI is in accordance with what is
rule-based solution is preferred over ML-based solution needed and required, usability testing is carried out. If the
because of the existence of a specific business domain UI is deemed to be satisfactory, the project can proceed to
knowledge and rules. Additionally, the rule-based frontend and backend development.
approach offers better explainability than the ML-based The application development needs to consider the
approach and the possibility of making ad-hoc business deployment and integration mechanism of the ML model.
rules if there is a change in the business process [10]. Additionally, the application also provides a feature where
The upgrade rule is simply governed by the users can find the recommended agents/customers easily
transaction frequency and volume. If the agent’s by using turn-by-turn directions. To enable this feature,
transaction frequency and volume surpass the defined we use the O-D (origins to destinations) travel time
threshold, then he/she can be promoted to the next class. matrix estimation using Google Maps API [13].
There are three tiers of BRILink agents: pemula agent
(first tier), jawara agent (second tier), and juragan agent
(third tier).
The Break Even Point (BEP) for any financial
product or financial institution is a condition where the
net income must be at least equal to the total expenditure
[11]. The development of BEP activation
recommendations for existing agents is made based on the
rules that have been made by the business by adopting
BEP calculation analysis.

B. Web Application Development

Extreme Programming (XP) [12] is used as the Fig. 3. Travel time estimation with the Google Maps API
primary software development methodology to develop
the web application development in this project. This Quality assurance (QA) process is embedded in the
methodology is slightly modified to adapt with BRI’s process to ensure the application is working as expected
development environment. The XP methodology being and no defects are found that could deter the application
used is visualized in Fig. 2. to work as per the requirement.

C. Change Management

ML-based implementation in the operational

workplace affects PABs performance, both at individual
and team level [14]. Therefore, there is a need to initiate a
change management initiative to provide understanding
regarding the implementation of this ML-based approach,
considering that this new approach changes PABs
day-to-day workflow.
The Matrix of Change (MoC) is used as the change
management tool for business process re-engineering
(BPR) [15]. The MoC consists of a four-step plan:
Fig. 2. Modified Extreme Programming Method
1. Identify existing and target practices and
processes. The existing practice uses subjective
The initial step in the XP methodology is defining judgment, intuition, and random selection to
user requirements. A discovery session, involving all determine potential new agents to be acquired and
project stakeholders (e.g., product owners, product existing agents to be activated and upgraded.
managers, business analysts, business owners, UI/UX Meanwhile, the target practice utilizes ML and a web
research team, and end-users), is held to explore user application to adopt a more data-driven acquisition,
needs and to gather user requirements. The requirements upgrade, and activation approach.
that have been obtained from the users are then analyzed. 2. Identify system interactions. Several features are
Prioritization is made to put focus and allocate resources presented to facilitate user interaction with the
to the most important requirements. The requirements are system: location suggestions, AI explainability
also transformed into application features for further feature, and scores to determine priorities.
development. 3. Identify transition interactions. The existence of a
Subsequently, a user interface (UI) prototype is web application creates a change in PABs’ existing
developed based on the agreed user requirements. To workflow. Therefore, it is necessary to educate them

1350
Authorized licensed use limited to: UNIVERSITAS GADJAH MADA. Downloaded on October 02,2023 at 04:56:27 UTC from IEEE Xplore. Restrictions apply.
 Proceedings of The 2022 IEEE International Conference on Industrial Engineering and Engineering Management (IEEM)

regarding the implementation of the new workflow selected as the final model to achieve both robustness and
and the technical know-how to use the new system. efficiency that is further deployed to the application.
4. Stakeholder survey. After the implementation of the
new system, checkpoints are carried out periodically B. Web Application Result
to get feedback from the users. This feedback is then
used as the basis for improving the application. There are several features that have been
Step three and four are carried out by holding implemented in the web application:
presentations/workshops explaining the use of the newly 1. Evaluation feature. The evaluation feature shows
developed application, including introduction of all the target and progress of achieving acquisition,
features embedded in the system. activation, and upgrade function. Thus, PABs can
monitor their own performance conveniently. A
snapshot of the evaluation feature interface is shown
III. RESULTS in Fig. 4.
2. Recommendation feature. The recommendation
A. Models Result feature displays the result of the developed ML
model and rule-based model, which generate highly
Some ML algorithms have been tried, including qualified customers to be acquired and potential
Logistic Regression, Random Forest, Decision Tree, and existing agents to be upgraded and activated. Map
LightGBM. Based on equation (2), precision calculates representation is used to represent the list of
the number of correct positive predictions made. recommendations, so that PABs are aware about their
Furthermore, AUC is an aggregate measure of location relative to the recommendees’ positions.
performance through all possible classification thresholds. Color coding is used to differentiate between the
Below are Logistic Regression, Random Forest, Decision acquisition, activation, and upgrade recommendation
Tree, and LightGBM models that are evaluated through (blue for acquisition, green for upgrade, and yellow
accuracy, precision, recall, and AUC. for activation). A snapshot of the recommendation
feature interface is shown in Fig. 5.
TABLE I 3. Detailed recommendation feature. We can select
PERFORMANCE METRICS OF COMPARATIVE MODELS one of the recommendations and view the detailed
information (e.g. recommendation score, the reason
Model Type Accuracy Precision Recall AUC why the recommendees are selected, and basic
information) of the recommendees. A snapshot of the
Logistic detailed recommendation feature interface is shown
0.636 0.506 0.048 0.545
Regression
in Fig. 6.
Random Forest 0.655 0.555 0.269 0.659 4. Routing feature. The routing feature is provided to
make it easier for PABs to find the most efficient
Decision Tree 0.578 0.423 0.435 0.548 route to reach their target. A snapshot of the routing
feature interface is shown in Fig. 7.
LightGBM 0.625 0.489 0.704 0.694

The model development includes the use of location

data to improve model performance, transaction data is
also generated on a monthly basis to improve
explainability, and use Mutual Information (MI) to select
the 100 features with the highest MI score. Accuracy and
AUC are two crucial and frequently used performance
indicators for binary classification. On the ROC curve of
the model, accuracy is a measure at one operating point or
decision threshold, whereas AUC measures all operating
points [16]. For imbalanced data, AUC is a more
informative metric than accuracy [17]. With 22.3% of the
data we utilize in this model being classified as a good
agent and 77.7% as a bad agent, indicate the data is
imbalanced. Based on Table I, Random Forest gains the
best in accuracy and precision but for recall and AUC,
LightGBM gains better. Although Random Forest gains
better accuracy and precision, it takes longer inference
time (4.44 minutes) by around two minutes than
LightGBM (1.47 minutes). Therefore, LightGBM was

1351
Authorized licensed use limited to: UNIVERSITAS GADJAH MADA. Downloaded on October 02,2023 at 04:56:27 UTC from IEEE Xplore. Restrictions apply.
 Proceedings of The 2022 IEEE International Conference on Industrial Engineering and Engineering Management (IEEM)

Fig. 4. Evaluation feature

Fig. 6. Detailed recommendation feature

Fig. 5. Recommendation feature

Fig. 7. Routing feature

1352
Authorized licensed use limited to: UNIVERSITAS GADJAH MADA. Downloaded on October 02,2023 at 04:56:27 UTC from IEEE Xplore. Restrictions apply.
 Proceedings of The 2022 IEEE International Conference on Industrial Engineering and Engineering Management (IEEM)

C. Change Management Result by a convenient user interface to interact with the

recommendation pipeline, such that they understand the
As previously mentioned. change management is reasons why the recommendations are made, the location
performed to provide understanding regarding the of the recommendees, the most efficient route to visit the
implementation of this ML-based approach, considering recommendees, and the target they need to achieve in a
that this new approach changes PABs day-to-day given day. A convenient user interface also helps PABs in
workflow. Therefore, piloting activities, as part of the making decisions and prioritizing activities to be carried
change management initiative, are held to give out in their day-to-day operations.
comprehension to PABs regarding how the new system The successful implementation of a ML-based
works and to provide an opportunity for PABs to give
solution is not solely based on the quality of technologies.
feedback based on their experience using the system.
It is proven that a carefully planned change management
It is worth mentioning that BRI operations span
initiative plays an important role to develop confidence of
throughout Indonesia, and each province/region in
Indonesia has their own characteristics and culture, which PABs in using the application and adopt it in their daily
influence how PABs work. Thus, it is important to operations.
consider this factor during any change management
initiative such that the approach is suitable to the
characteristics and culture of each province/region. V. CONCLUSION
After a month of doing the change management
initiative in two specific regions, we have successfully The ML-based recommendation engine managed to
acquired 200 new agents, activated 71% of the activation provide a robust result for acquiring highly potential
pipeline, and upgraded 55% of the upgrade pipeline. customers to become banking agents. Based on direct user
Moreover, there is a 13% increase in fee-based income validation, the recommendations are highly matched to
within 6 months (Q4 2021-Q1 2022). profiles of prospective BRILink agents defined by
Meanwhile, for all regions in Indonesia, 1.5% business targets.
potential agents have been successfully acquired to The recommendations for acquisition of new agents,
become new agents. The existing agents also experienced upgrades and BEP activation of existing agents presented
18.3% growth in achieving BEP and increasing 10.6% the in the BRIBrain application have had a significant
quality of the agent class. positive effect on user performance in achieving daily
activity targets. Users who previously only relied on
intuition and made independent decisions based on
IV. DISCUSSION experience in the field, are now able to rely on their
mobile applications to make decisions that are more
The implementation of the ML-based solution in the
efficient and effective.
process of agent acquisition, upgrade, and activation has
It takes a certain approach to implementing this
improved the performance of PABs as well as BRILink
application in the operational workplace. The addition of
agents. Before the implementation of the ML-based
AI explainability and suggestion location features makes
approach, PABs relied predominantly on subjective
it easier for users to understand and follow up on
judgment, intuition, and random selection to determine
recommendations given in the application. Regular check
potential new agents to be acquired and existing agents to
points with users are also very helpful in understanding
be activated and upgraded. Consequently, many BRILink
problem findings and user needs for iteration
agents do not perform as expected and have relatively low
improvements
productivity. After the implementation of the ML-based
solution, PABs are assisted by being given high-quality
recommendation pipeline for acquisition, upgrade, and
ACKNOWLEDGMENT
activation.
Considering the novelty of this new system to PABs This research was supported by Digital Banking
and BRILink business and operations, there were initial Development & Operation Division and BRILink
concerns that this type of solution would be rejected by Network Division, PT Bank Rakyat Indonesia (Persero)
PABs due to their unfamiliarity with ML and the Tbk. We thank our colleagues from PT Bank Rakyat
complexity of an ML system. The solution to this concern Indonesia (Persero) Tbk who contributed on data
is by making the ML solution to be transparent (i.e. collection for experiments and reviewers for their
explainable) such that PABs can understand the reasoning productive feedbacks to enhance this work.
behind any recommendations and decisions made by the
ML system. Additionally, a human-centered approach is
required by making the ML solution to be easily used and
understood. Therefore, the ML model is complemented

1353
Authorized licensed use limited to: UNIVERSITAS GADJAH MADA. Downloaded on October 02,2023 at 04:56:27 UTC from IEEE Xplore. Restrictions apply.
 Proceedings of The 2022 IEEE International Conference on Industrial Engineering and Engineering Management (IEEM)

REFERENCES [10] D. Sola, C. Meilicke, H. van der Aa, and H.

Stuckenschmidt, “A Rule-Based Recommendation
[1] C. Nuryakin et al., “Financial Inclusion through Approach for Business Process Modeling,” in
Digital Financial Services and Branchless Banking: Advanced Information Systems Engineering, Cham,
Inclusiveness, Challenges and Opportunities,” p. 11, 2021, pp. 328–343.
2017. [11] D. Roy and K. Ghosh, “A Proposed Model of a
[2] H. Limanseto, “Akselerasi Capaian Target Inklusi Microcredit Institution: Break-Even Analysis,
Keuangan dalam Pengembangan Kemandirian Borrowing Group Creditworthiness and Risk
Ekonomi Pondok Pesantren,” 2022. Analysis,” p. 14.
https://www.ekon.go.id/publikasi/detail/4140/akseler [12] S. Sharma, D. Sarkar, and D. Gupta, “Agile
asi-capaian-target-inklusi-keuangan-dalam-pengemb Processes and Methodologies: A Conceptual Study,”
angan-kemandirian-ekonomi-pondok-pesantren vol. 4, no. 05, p. 7, 2012.
[3] P. Guerra and M. Castelli, “Machine Learning [13] F. Wang and Y. Xu, “Estimating O–D travel time
Applied to Banking Supervision a Literature matrix by Google Maps API: implementation,
Review,” Risks, vol. 9, no. 7, p. 136, Jul. 2021, doi: advantages, and implications,” Ann. GIS, vol. 17, no.
10.3390/risks9070136. 4, pp. 199–209, Dec. 2011, doi:
[4] F. Pallonetto, M. D. Rosa, F. Milano, and D. P. Finn, 10.1080/19475683.2011.625977.
“Demand response algorithms for smart-grid ready [14] A. Al Ali and S. Badi, “Exploring the Impacts of
residential buildings using machine learning Artificial Intelligence (AI) Implementation at
models,” Appl. Energy, vol. 239, pp. 1265–1282, Individual and Team Levels: A Case Study in the
2019, doi: UAE Government Sector,” Cham, 2022, pp.
https://doi.org/10.1016/j.apenergy.2019.02.020. 597–613.
[5] R. Wirth and J. Hipp, “CRISP-DM: Towards a [15] E. Brynjolfsson, A. A. Renshaw, and M. van Alstyne,
Standard Process Model for Data Mining,” p. 11. “The Matrix of Change: A Tool for Business Process
[6] S. Zhang, C. Zhang, and Q. Yang, “Data preparation Reengineering,” p. 38.
for data mining,” Appl. Artif. Intell., vol. 17, no. 5–6, [16] A. M. Carrington et al., “Deep ROC Analysis and
pp. 375–381, May 2003, doi: 10.1080/713827180. AUC as Balanced Average Accuracy to Improve
[7] G. Ke et al., “LightGBM: A Highly Efficient Model Selection, Understanding and Interpretation,”
Gradient Boosting Decision Tree,” p. 9. IEEE Trans. Pattern Anal. Mach. Intell., pp. 1–1,
[8] B. Gao and V. Balyan, “Construction of a financial 2022, doi: 10.1109/TPAMI.2022.3145392.
default risk prediction model based on the [17] T. Yang and Y. Ying, “AUC Maximization in the Era
LightGBM algorithm,” J. Intell. Syst., vol. 31, no. 1, of Big Data and AI: A Survey.” arXiv, Aug. 03,
pp. 767–779, Jun. 2022, doi: 2022. Accessed: Sep. 11, 2022. [Online]. Available:
10.1515/jisys-2022-0036. http://arxiv.org/abs/2203.15046
[9] Y. Xia, “A Novel Reject Inference Model Using
Outlier Detection and Gradient Boosting Technique
in Peer-to-Peer Lending,” IEEE Access, vol. 7, pp.
92893–92907, 2019, doi:
10.1109/ACCESS.2019.2927602.

1354
Authorized licensed use limited to: UNIVERSITAS GADJAH MADA. Downloaded on October 02,2023 at 04:56:27 UTC from IEEE Xplore. Restrictions apply.