International Journal of Business and Management Invention (IJBMI)

ISSN (Online): 2319-8028, ISSN (Print):2319-801X

www.ijbmi.org || Volume 10 Issue 2 Ser. III || February 2021 || PP 19-24

An IFRS 9 Framework for Model Validation

Mitoi Elena
Doctoral School of Accounting, Faculty of Accounting and Management Informatics, Bucharest University of
Economic Studies, Bucharest, Romania

Moldoveanu Valentin
Doctoral School of Accounting, Faculty of Accounting and Management Informatics, Bucharest University of
Economic Studies, Bucharest, Romania

Cazazian Rafaela
Doctoral School of Accounting, Faculty of Accounting and Management Informatics, Bucharest University of
Economic Studies, Bucharest, Romania

Turlea Codrut
Doctoral School of Accounting, Faculty of Accounting and Management Informatics, Bucharest University of
Economic Studies, Bucharest, Romania

ABSTRACT: With the coming into force of the IFRS 9 standard in January 2018 financial institutions have went
from an incurred loss model to a forward looking model for the computation of impairment losses. As such, the
IFRS 9 models use point-in-time (PIT) estimates of PDs and LGDs and provide a more faithful representation of
the credit risk at a given PIT as they are based on past experiences as well as the most recent and forecasted
economic conditions. However, given the short-term fluctuations in the macroeconomic conditions, the final
outcome of the Expected credit loss (ECL) models is highly volatile due to their sensitivity to the business cycle. In
order to prevent financial institutions’ over or under provisioning after the models have been developed, they need
to be adequately monitored and validated and if necessary re-calibrated in order to ensure that the outcome of the
models are accurate. The IFRS 9 standard has introduced the necessity to compute lifetime expected credit loss,
hence institutions had to use modelling techniques different to those used for the established Internal Ratings
Based (IRB) regulatory-capital estimation purposes. In an IFRS 9 context, a financial institution relies on rating
systems which are based on historic data to produce credit risk ranking systems. The resulted ratings are then used
in twelve months and lifetime PD estimation. Hence the lifetime PD models used in IFRS 9 are as good as the
underlying data used. Thus, financial institutions that use ratings as risk drivers for lifetime PD models must have
in place validation and monitoring tests to assess credit scoring models quality. As such, the paper focuses on the
validation of PD models under the IFRS 9 standard, presenting the complexity and challenges of developing the
PD models 9 and a selection of qualitative and quantitative techniques applicable in the monitoring or validation
processes.
KEYWORDS: IFRS 9, Financial Institution, Quantitative Validation Tests
---------------------------------------------------------------------------------------------------------------------------------------
Date of Submission: 05-02-2021 Date of Acceptance: 18-02-2021
---------------------------------------------------------------------------------------------------------------------------------------

I. INTRODUCTION
The Basel capital requirements are based on obligor-specific characteristics which are used in deriving the
key risk parameters: probability of default, loss given default and exposure at default used to determine unexpected
losses estimates, furthermore the IFRS 9 requirements also specify the same regulatory parameters could be used
for the computation of the expected credit losses upon applying specific adjustments i.e. integration of forward
looking variables and computation of lifetime estimates. Achim and Streza (2008) explain how the model
validation causes are similar to the errors identified throughout the audit process. This causes can be grouped into:
the failure to understand the business and the environments, the failure to assess management estimates and
overlay accurately, the lack of exercising professional skepticism, inadequate evaluation and quantification of
risks. Under the IFRS 9 framework it is expected that the output of the rating system is reviewed against realised
migrations. In order to ensure a strong validation process is in place, the assessment should be based both on an
out-of-time and out-of-sample assessment. A reliable validation process should be based on a comprehensive set of
statistical tests with a predefined set of thresholds and actions to be undertaken in case they have been breached.

DOI: 10.35629/8028-1002031924 www.ijbmi.org 19 | Page

 An IFRS 9 Framework for Model Validation

II. LITERATURE REVIEW

The Validation function is an independent and transparent function, similar to the internal audit function
as well as audit and assurance. Validation represents a complex process combining quantitative and qualitative
procedures performed in order to assess the model’s performance, challenge it and identify its limitations. The
importance of validation was noticed since the early stages of model implementation with the introduction of the
Basel requirements. As acknowledged by Giddens (1990) “not only the limits of, or the gaps in, expert knowledge,
but ... the very idea of expertise” play a crucial role throughout validation, hence complex mathematical models are
not enough to understand and present the global view of the model, its integration in the bank’s processes.
Currently, supervisors encourage banks to use quantitative models as a management tool and to be imbedded in the
business decisions process i.e. measuring risk, valuation of exposures, instruments, or positions, meeting financial
or regulatory reporting requirements, measuring compliance with internal limits or thresholds and performing
stress testing. Hence, an adequate validation governance framework is required in order to ensure an appropriate
level of oversight by the management body (Board of Directors, Supervisory Committee, Management
committees). The aforementioned framework is expected to cover the models in scope, the objectives, the policies
and methodologies as well as the infrastructure set in place to ensure its implementation.

III. RESEARCH METODOLOGY

The methodology of the scientific research that we will use in this paper combines the qualitative
research with the quantitative one, stating that the efficiency of the results obtained from the research would be
greater if an optimal combination between the qualitative research and the quantitative one was achieved, in
order to meet the objectives, set. The method we used in this paper is the documentation method in which we
went through the literature specialized in the field so that we could identify the relevant works on the matter.

IV. RESULTS
The number and complexity of models is increasing as the scope of application widens i.e. model are used
for the computation of capital requirement and impairment charges as well as stress testing and pricing.
Furthermore, with the introduction of the IFRS 9 in 2018 standard financial institutions faced a new challenge by
having to integrate accounting and risk data and building more complex models. The IFRS 9 standard focuses on
the classification and measurement of financial assets and financial liabilities, including the computation of
impairment losses. The main difference to the IAS 39 standard is that under IFRS 9 the impairment- expected
credit losses are computed taking into consideration all reasonable and supportable information, including forward-
looking information in order to the recognize both the 12 months expected credit losses for instruments whose
riskiness is similar to the data or recognition and lifetime expected credit losses for all remaining financial
instruments i.e. those for which a significant increases in credit risk since initial recognition was identified. The
assessment has to be performed for both the individual and collective impairment models. An integrated validation
framework should be set up in order to meet the IFRS 9 as well as the regulatory internal rating based approach
requirements. The core element of the validation framework is a solid internal governance which enables to sustain
the entire infrastructure, however this must be combined with experienced risk management professionals.
Based on the ECB Guide on Internal models and the GPPC requirements Validation activities can be split into
three types:
 Initial validation – occurs after the first development of a model, it is also performed for the evaluation
of material changes and potential extensions of existing models.
 Periodic validation – it is a mandatory periodic exercise (yearly or quarterly), however it can also be per
performed ad-hoc at regulators’ or external auditor’s demand.
 Monitoring –is performed on a more frequent basis (monthly or quarterly) and acts as an early warning
system for model deterioration.

A validation assessment should be both qualitative and quantitative in nature.

Qualitative validation/monitoring focuses on the analyses on documentation, feedbacks, interviews in order to
identify if elements are properly implemented as well as to ensure that quantitative methods are applied properly in
practice. When performing the assessment, the following should be covered:
 Data Quality – tests should be performed to assess the completeness and accuracy of the data,
furthermore the effectiveness of the checks and controls in place should be assessed.
 Default definition – the stability of the default definition over time should i.e. ensure consistency of the
modelling definition of default and the one used in the live environment.
 Relevance of rating process – tests should be performed to determine whether the process of assigning
the exposures to rating grades is still appropriate.
 Override Analysis – assessment of the extent of the involvement of expert judgment and overrides as
well as how it is assessed and validated

DOI: 10.35629/8028-1002031924 www.ijbmi.org 20 | Page

 An IFRS 9 Framework for Model Validation
 Environmental dynamics – the causal link between past events and the observed model performance
should be taken into consideration. Hence, the analysis of the potential effect of business, functional,
technical, regulatory and macroeconomic environmental factors may have on the models components,
such as target sample, predictors, model assumptions, methodological choices etc.
 Use-Test – assess whether the models are applied and used as intended.

In relation to the quantitative validation tests the following elements should be analyzed:
 Samples used for validation purposes – it should be different to the model development samples ensuring
that the out-of-sample and out-of-time principles are respected, furthermore one-off events must be
identified, assessed and excluded from the population.
 Consistency of the definition of default – the definition of default should be consistent throughout time
(in case of material differences, for the development and validation sample furthermore the same default
events should be considered for the computation of PD, LGD, and EAD/CCF.
 Model Discrimination – represents the capacity of the model to differentiate between defaulted and non-
defaulted exposures.
 Population Stability – most often it represents the comparison of data used in validation to the data used
for model development in order to assess if the model stable over time.
 Characteristic Stability – represents the assessment of the information value of each variable and the
impact on the model performance. In practice there is a trade-off between discriminatory power and
stability in case stability is severally affected, the discriminatory power is sacrificed, as stability is the
basis for statistical inference.
 Concentration Analysis – represents the assessment of large concentrations for particular deciles/credit
grades/pools, furthermore in can be extended to the analysis of large migrations over time.
 Staging analysis –the staging mechanism must be assessed (validated). This analysis could be done using
migration matrixes. Based on best practices, Stage 2 must be a transitory state that shouldn’t be
bypassed, hence transitions from stage 1 to 3, without many stage 2 transitions, should be considered as
an indication that the staging algorithm is inappropriate.
 Calibration or “Actual versus Expected” –represents the assessment of the model’s accuracy and the
model’s estimation error, this test is performed using a different sample than the modeling one. It can be
used for both validation and monitoring purposes.
 ECL back-testing – represents the most important test to be performed as it bring together the impact of
all the IFRS 9 model components. The ECL back testing must be performed both at portfolio level as
well at a grade level to ensure the loss estimates are accurate. The relative and absolute errors should be
analysed. Under this, collective and individual ECL is expected to be assessed separately.

The outcome of the validation process is the validation report which provides an overview of the overall model
assessment. In relation to the presentation of the outcomes, the traffic light approach is widely spread among
institutions as the outcome of each test is linked with the green, amber or red indicator.
In accordance with best practices and the EBA ECL guidelines the report itself should be given one of the
following ratings:
 Green –if the overall performance of the model is as expected and the model can be implemented subject
to the approval of the management body;
 Amber – if the model has a satisfactory performance, however further analysis should be performed in
order to understand and correct the identified deficiencies. The model can still be implemented after the
limitations and concerns have been assumed by being approved by the management body; and
 Red – the performance of the model is unsatisfactory i.e. the model is inadequate or has degraded, based
on the severity of the outcome a re-calibration or replacement of the model is proposed by the validation
department in order to mitigate inefficiencies.
Where thresholds are breached institutions must ensure that appropriate remediation actions are identified and
carried out.

4.1 Regulatory Requirements for validation under IFRS 9

The main regulatory requirements for IFRS 9 are set by Commission Regulation (EU) 2016/2067 of November
2016 (IFRS 9 Standard). In order to be compliant with the standard’s requirements on validation the following
requirements should be met:
 B5.5.52of the IFRS 9 standard– “An entity shall adjust historical data, such as credit loss experience, on
the basis of current observable data to reflect the effects of the current conditions and its forecasts of
future conditions that did not affect the period on which the historical data is based, and to remove the
effects of the conditions in the historical period that are not relevant to the future contractual cash flows.
[...] Estimates of changes in expected credit losses should reflect, and be directionally consistent with,
DOI: 10.35629/8028-1002031924 www.ijbmi.org 21 | Page
 An IFRS 9 Framework for Model Validation
changes in related observable data from period to period (such as changes in unemployment rates,
property prices, commodity prices, payment status or other factors that are indicative of credit losses on
the financial instrument or in the group of financial instruments and in the magnitude of those changes).
An entity shall regularly review the methodology and assumptions used for estimating expected credit
losses to reduce any differences between estimates and actual credit loss experience.”

Among the first institutions to provide additional guidance to the IFRS 9 standard, was the Basel
committee on banking supervision which published the Guidance on credit risk and accounting for expected credit
losses in December 2015. The document acknowledges the importance of the validation process: “A bank should
have policies and procedures in place to appropriately validate models used to assess and measure expected credit
losses.” Global Public Policy Committee published additional clarification on the key elements, the document
puts emphasis on the importance of the control framework and encourages institutions to focus on the estimation
and reporting of the ECL by establishing key performance indicators which can be used as tools for challenging
the model’s performance. The European Banking Authority Guidelines on Accounting for Expected Credit losses,
EBA/GL/2017/06 in 12 May 2017, brings additional clarifications on the governance arrangement on validation,
monitoring and review processes under “Principle 5 – ECL model validation”:
 An adequate governance should be established (policies and procedures) to ensure at a minimum the
following: accuracy and consistency of the models, risk rating systems and processes as well as an adequate
estimation of all relevant risk components (PD, LGD, EAD).
 Model validation should be carried out at model development, as well as after the development of the
model, through periodic validation and monitoring.
 The IFRS 9 models are expected to be updated frequently to ensure that changes in the macroeconomic
conditions are factored into the models to comply with the point in time requirements as well as the use of
most recent and updated information.

An adequate validation framework should include, but not be limited to, the following:
 An adequate governance process, with well-defined roles and responsibilities ensuring the function’s
independence and competences. An appropriate scope and methodology to ensure the robustness,
consistency and accuracy. As well as identify potential limitations of a model in order to address them in
a timely manner. The validation process should ensure a review of model inputs, design and outputs as
well as continue assessing its performance and fit for use through time.
 Model inputs: analysis of the quality, accuracy and reliability of data (historical, current and forward-
looking information) i.e. data is expected be relevant to the institutions’ current portfolios as well as
reflect the credit practices going forward and, to the extent possible, accurate, reliable and complete. The
validation process is seen as an additional layer, following model development to ensure that the data
used complies with the IFRS 9 requirements.
 Model design: The validation process assessed wheatear the underlying theory and statistical approach
used for the model development is appropriate and accepted based on best practices, regulatory and
accounting requirements and they are fit for use.
 Model output/performance: Institutions should develop internal threshold (in accordance with best
practices and the institution’s own risk appetite) in order to identify significant breaches in the
performance of individual parameters as well as the model’s outcome.

The validation framework should be clearly documented and reviewed (the review should be ensured by an
independent party and the findings should be reported to the management body and the audit committee on a
timely manner) on a regular basis.

4.2 IFRS 9 main challenges

The introduction of the IFRS 9 standard requires institutions to compute lifetime expected credit losses,
hence the requirements and complexity of both the modeling and validation of such models increases significantly.
Among the most common difficulties identified is the estimation of lifetimes losses for exposures with long
maturities for which only short historical time series are available, such as mortgages or for revolving exposures.
This estimates would be further used for the prediction of 12 months unexpected and losses as part of the
regulatory capital computations. Other opinions would consider a cycle of five or twenty years. The assessment of
the actual versus predicted losses is essential in understanding if the estimates are accurately depicting the riskiness
of the entire portfolio, as such it is important that the financial institution respects the out-of-time and out-of-
sample principles in order to avoid the overfitting of the parameters to the development sample, hence part of the
available data should be kept for validation purposes. The main challenges introduced by IFRS 9 are the
computation of the lifetime estimates and the staging concept. The rest of the paper present tests used to assess the
12 months and lifetime PDs. Under IAS 39 an exposure could find itself in two states: default or non-default,
DOI: 10.35629/8028-1002031924 www.ijbmi.org 22 | Page
 An IFRS 9 Framework for Model Validation
however IFRS 9 introduces the concept of stages and classifies exposures in 3 stages. Assets classified in stage 1
show no increase in credit risk since initial recognition, those classified as stage 2 present significant increase in
credit risk since origination, while the stage 3 covers defaulted assets. For stage 1 assets a 12 months expected
credit loss is computed, while for stages 2 and 3 a lifetime expected credit loss is computed. The elements that
possess difficulties is the definition and identification of the significant increase in credit risk (SICR). Even though
IFRS 9 prescribes the 30 days past due as a criteria for the recognition of the SICR, the criterion is rebuttable.
Under regulatory requirements as well as under the economic capital, the probability of default (PD) estimate the
likelihood that an obligor will be unable to meet its contractual payment obligations over a 12 months period,
however IFRS 9 requires in addition to the 12 months horizon an estimation of PD also for the lifetime of the
financial instrument, depending on its maturity. Based on the ECL formula, the PD is one of the main drivers of the
ECL alongside with the LGD (loss given default). The Expected Credit loss formula:

In order to estimate a lifetime PD, the easiest way is to rely on 12 month PDs.
The Population stability index (PSI) is the most common used test to check the representativeness of validation
sample. The PSI can be computed for each period used for the computation of the lifetime PD. The formula for
PSI is:

Where:
– Number of observations in the in of the development dataset;
– Number of observations in the in of the validation/monitoring dataset;
– Total number of observations in the development dataset;
– Number of observations in the validation/monitoring dataset;
In the case of the lifetime PD, each period must have a comparable difference in number of defaults with regard
to the previous period:

Where:
– Relative difference between two consecutive transition moments of the development sample
(cumulative or marginal);
– Relative difference between two consecutive transition moments of the validation sample
(cumulative or marginal);
– Transition percentage from period “I”. Can be a cell of a matrix used in Markov chain estimation
method. Can also be the cumulative empirical PD of a survival curve?
I – transition period. Start with a value of 2, we have no delta for the first period;
N – Number of transition periods;
The relative difference can also be computed for the first transition period.

Once it was determined that the population is comparable, the accuracy of PD estimation can be assessed
computing the mean squared error (MSE). The MSE is expected to be computed for each individual grade/pool.
The MSE measures the average squares of the errors or deviations generated by the difference between estimated
and observed PDs.

Where:
– Observed PD vector;
– Estimated PD vector;
I – period (year, month) of lifetime PD;
In the figure 1 below, two lifetime PD curves are presented. The one on the left (1) has a lower MSE than the one
on the right (2). Although the MSE nearly doubles when comparing the two graphs, for a better view of the actual
error, SE and cumulative SSE can be plotted. For a better scaling of results calculated .

DOI: 10.35629/8028-1002031924 www.ijbmi.org 23 | Page

 An IFRS 9 Framework for Model Validation
Figure 1: Lifetime PD curves

Source: Own calculations

In general, a smaller MSE means a more accurate model, however the acceptable deviation in terms of MSE is
defined according to each institutions lifetime PD modelling methodology.

V. CONCLUSIONS
Historically, supervisors have had a reactive attitude, they have followed the economic cycle, have
tightened the regulations following the economic crisis from 2008, as such both supervisors and banks have
realized the importance of model risk management. Given the importance of models and their increasing
complexity greater priority needs to be placed on ensure a thorough validation of the models outputs to ensure the
outcomes are robust and in line with the institutions’ risk profile and portfolio structure. The paper outlined the
main IFRS 9 validation requirements and standards as well as the main changes and challenges that banks face
with the introduction of the IFRS 9 standard focusing on the validation of the 12 months and lifetime PD risk
parameter. The paper describes how a robust and reliable PD validation framework that can be constructed by
financial institutions regardless of the calibration approach used for the development of the lifetime PD. Among
the elements represented are the calibration requirements compliant by maintaining the independence,
completeness, adequacy and soundness of validation. In this paper, we outline a Probability of Default Validation
Framework that we believe would satisfy the Internal Ratings-Based (IRB) approach of the Basel II Accord
(based on a quantitative testing approach only). Regular model validation is necessary for IFRS 9 compliance,
including monitoring of performance and stability.

BIBLIOGRAPHY
[1]. Adrian, T. and Shin, H. S. (2008) Liquidity, Monetary Policy and Financial Cycles, Current Issues in Economics and Finance, Vol.
14, No. 1.
[2]. Basel Committee on Banking Supervision (1988), International Convergence of Capital Measure ment and Capital Standards.
[3]. Basel Committee on Banking Supervision (2005), International Convergence of Capital Measure ment and Capital Standards: A
Revised Framework, BIS, Updated November 2005.
[4]. Basel Committee on Banking Supervision (2005a), Validation, Newsletter No. 4.
[5]. Basel committee on banking supervision (2015), Guidance on credit risk and accounting for expected credit losses
[6]. Blochwitz, S. and Hohl, S. (2011) Validation of Banks’ Internal Rating Systems: A Supervisory Perspective, in Engelmann, B. and
Rauhmeier, R., The Basel II Risk Parameters, Second Edition, Springer.
[7]. Board of Governors of the Federal Reserve System, Supervisory Guidance on Model Risk Management, SR Letter 11–7,
Washington, April 2011.
[8]. Daníelsson, J. (2002) The Emperor has No Clothes: Limits To Risk Modelling, Journal of Banking & Finance, Elsevier, Vol. 26,
No. 7, 1273–1296.
[9]. Derman, E. and Kani, I. (1994) Riding on a Smile, Risk, Vol. 7, 32–39.
[10]. European Banking Authority (2017), Guidelines on credit institutions’ credit risk management practices and accounting for
expected credit losses
[11]. European Banking Authority (2017), Guidelines on PD estimation, LGD estimation and the treatment of defaulted exposures
[12]. European Banking Authority (2016), Guidelines on the application of the definition of default under Article 178 of Regulation (EU)
No 575/2013
[13]. Global Public Policy Committee (2016), The implementation of IFRS 9 impairment requirements by banks
[14]. Giddens, A. (2012) The Consequences of Modernity, Stanford University Press, Stanford, CA, 1990. JP Morgan, Report of JP
Morgan Chase and Co. Management Task Force Regarding
[15]. International accounting standards Board (2014), International Financial reporting standard 9 Financial instruments
[16]. Organisation for Economic Co-operation and Development, OECD Principles of Corporate Governance, Paris, 2004.

Mitoi Elena, et. al, “An IFRS 9 Framework for Model Validation.” International Journal of Business
and Management Invention (IJBMI), vol. 10(02), 2021, pp. 19-24. Journal DOI- 10.35629/8028

DOI: 10.35629/8028-1002031924 www.ijbmi.org 24 | Page