Risk Control and Monitoring

Risk Management Life Cycle

IT Risk Identification

Risk and Control Monitoring and

IT Risk Assessment
Reporting

Risk Response and Mitigation

Control Monitoring & Reporting Objective

Continuousy monitor and report on

risk to stakeholders
Ensure the continued efficiency and
effectiveness of the IT risk
management strategy
Alignment to business objectives
 Key Topics
1. Define and establish key risk indicators (KRIs)
and thresholds based on available data, to
enable monitoring of changes in risk.
2. Monitor and analyze key risk indicators (KRIs)
to identify changes or trends in the IT risk
profile.
3. Report on changes or trends related to the IT
risk profile to assist management and
relevant stakeholders in decision making.
 Key Topics
4. Facilitate the identification of metrics and key
performance indicators (KPIs) to enable the
measurement of control performance.
5. Monitor and analyze KPIs to identify changes or
trends related to the control environment and
determine the efficiency and effectiveness of control.
6. Review the result of control assessments to
determine the effectiveness of the control
environment.
7. Report on the performance of, changes to, or trends
in the overall risk profile and control environment to
relevant stakeholders to enable decision making.
Learning Objectives

KRIs vs. KPIs

Describe data extraction, aggregation,
and analysis tools and techniques
Compare different monitoring tools
and techniques
Describe various testing and
assessment tools and techniques
The Objective
The risk response is designed and implemented based on a
risk assessment that was conducted at a single point in
time;
Risk changes:
• Controls can become less effective
• The operational environment may change
• New threats, technologies, and vulnerabilities may emerge
Because of the changing nature of risk and associated
controls, ongoing monitoring is an essential step of the risk
management lifecycle
Metrics and Key Risk Indicators
Measuring Effectiveness

The effect of risk response and

selected controls must be measurable
• Management overview
• Management support
• Due care and due diligence
• Compliance with regulations
Impact of Changes on Risk

Changes to the business, technology,

and projects can affect the risk profile
Risk practitioner reviews proposed
changes to determine the impact of
the changes on risk
Monitor trends
Compare results with performance
and risk objectives
 Key Risk Indicators

Compare to the risk

Measure the level of risk
thresholds

Alert to reaching or
approaching an unacceptable Tracking mechanism
level of risk
 Select meaningful set of controls
to monitor
KRI Selection • Consistent areas to measure
• Good indicators of health of
risk management program
• Areas that can be influenced
by management
 Specific
Based on a clearly understood goal

Measurable
Able to be measured

SMART Metrics Attainable

Realistic, based on important goals and values

Relevant
Directly related to a specific activity and goal

Timely
Grounded in a specific time frame
 Sample of Possible KRIs

Number of unauthorized
Breaches of Service Level
devices discovered on a
Agreements (SLAs)
network

Number of misconfigured
Time to deploy new patches
systems
KRI Effectiveness
Measure areas of higher risk
Measure areas that are relatively easy to
measure
• Reliability – a good predictor of risk
level
• Sensitivity – accurately reflect changes
in risk
• Repeatable – consistent measurement
to detect trends and patterns
 KRI Optimization

1 2
Collect and report on the Set KRI thresholds correctly
correct data

Data must be reliable, monitored and reported to management in a timely manner –

so that action can be taken expeditiously
 Key Performance Indicators (KPIs)

Determine how well a

Threshold that
process is performing Indicated likelihood of
indicates unacceptable
in enabling a goal to be reaching a goal
results
reached

Quantitative
Set benchmarks
measurement
 Sample KPIs

Network or system availability

Customer satisfaction levels
Number of complaints resolved on first
contact
Response to for data retreival
Number of employees that have
attended annual awareness sessions
Comparing KRIs and KPIs

KPIs indicate the threshold of

unacceptable results
KRIs are ‘tripwire’ that indicates a
measure is in danger of exceeding a
KPI
• A KRI is an early warning signal to
allow action to be taken before a
KPI is exceeded
Data Collection Sources
Data Collection Sources

Audit reports Interviews

Incident reports Security reports
User feedback Logs
Observation
 Logs
Capture and store data for analysis
Must be protected from alteration
- Investigate processes
- Contain sensitive data
Retained for a suitable length of time
Capture data
- In a timely way,
- Close to the source of the incident
- To enable analysis
 Log Analysis

Identify trends or Identify the

Identify
developing source of an
violations
attacks attack

Require time synchronization for log comparison

Goals of Monitoring

Create a culture of risk management

Continuous monitoring instead of periodic
monitoring
Feedback to improve risk response
Verify that controls are working correctly
and mitigating risk
Type Monitoring

Self-asssessment
Automated assessment
Third party audits
• Internal
• External
Effectiveness of Assessments

Depends on:
• Complete and accurate data
• Skill of analyst
• Management support and
response
• Continuous scheduled reviews –
not ad-hoc
 IS Audits
Provide an objective review of the
efficiency of IT operations
- Acquisition (the right product)
- Implementation
- Maintenance
- Disposal
Audit both technical and non-technical
aspects of the operations of
information systems
Vulnerability Assessment and Penetration
Tests
Vulnerability Assessments

Discover any potential vulnerabilities

that could be exploited
Report to management
- False positives
- False negatives
Finds, but does not exploit a
vulnerability
Vulnerability Assessment and Penetration
Tests

May be conducted by internal or

external teams
Try to simulate the methods used by
attackers
May be both technical and non-technical
- Physical security
- Social engineering
Penetration Test
Try to exploit a perceived vulnerability
Often based on the results of a
vulnerability assessment
May test:
- Applications
- Networks
- Physical
- People
- Incident management process
Results of Penetration Tests

Report provided to management

• Identify test procedures
• Identify any areas of concern
• Provide recommendation for
improvement
• Prioritize risk according to severity
Third Party Assurance and Reporting
 Third Party Assurance

Provide assurance to clients and

partners of:
- Compliance with best practice
- Standards
ISO - Maturity of risk management program
- Information security
SSAE 16
ISAE 3402
Maturity of Risk Management Process

Continuous Quantitatively
Optimization
improvement managed

Defined reliable Proactive

process management of risk
Monitoring Change

Alert to changes that could affect the

risk profile and the ability of the
organization to reach its goals
Annual review of monitoring an
reporting program
 Summary
• Proper and effective management of risk is
essential to protecting the assets of the
organization
• Risk management is a never-ending process
• IT risk and controls should be monitored
continuously to ensure that they are adequate
and effective