Doc 9883

Manual on Global Performance

of the Air Navigation System

Approved by the Secretary General

and published under his authority

First Edition — 2009

International Civil Aviation Organization

Published in separate English, French, Russian and Spanish editions by the
INTERNATIONAL CIVIL AVIATION ORGANIZATION
999 University Street, Montréal, Quebec, Canada H3C 5H7

For ordering information and for a complete listing of sales agents

and booksellers, please go to the ICAO website at www.icao.int

First edition 2009

ICAO Doc 9883, Manual on Global Performance of the Air Navigation System
Part I – Global Performance
Part II – Performance-based Transition Guidelines
Order Number: 9883
ISBN 978-92-9231-416-3

© ICAO 2009

All rights reserved. No part of this publication may be reproduced, stored in a

retrieval system or transmitted in any form or by any means, without prior
permission in writing from the International Civil Aviation Organization.
 AMENDMENTS

Amendments are announced in the supplements to the Catalogue of ICAO

Publications; the Catalogue and its supplements are available on the ICAO
website at www.icao.int. The space below is provided to keep a record of such
amendments.

RECORD OF AMENDMENTS AND CORRIGENDA

AMENDMENTS CORRIGENDA

No. Date Entered by No. Date Entered by

(iii)
 FOREWORD

This manual was produced by the ICAO Air Traffic Management (ATM) Requirements and Performance
Panel within the context of a performance-based global air navigation system.

The notion of a performance-based air navigation system emanated from sound industry practices external
to the aviation industry. As the aviation industry has evolved into a less regulated and more corporate environment, the
advantages of implementing a performance-based air navigation system are becoming increasingly apparent.

The performance-based approach (PBA) is based on the following principles:

— strong focus on desired/required results through adoption of performance objectives and targets;

— informed decision-making, driven by the desired/required results; and

— reliance on facts and data for decision-making.

Assessment of achievements is periodically checked through performance reviews, which in turn requires
adequate performance measurement and data collection capabilities.

Establishing a PBA requires knowledge-sharing, training and other specific expertise.

The expected result is a more efficient system through identified cost savings, reduction in the waste of
resources, more equitable charging practices, and more efficient provision of services. As the work effort is challenging
and requires a globally coordinated effort, the aviation community should be encouraged to follow a common approach
toward the development and implementation of a performance-based air navigation system.

The PBA can be applied in economic management, transition management, safety management, security
management, communications, etc. This manual is not replacing any existing ICAO guidance material in these areas,
rather it is to be seen as overarching and complementary guidance, introducing a set of performance management
terminology and techniques to assist the aviation community members in converging towards a globally harmonized and
agreed upon approach.

One of the objectives of this manual is to promote the use of harmonized terminology among ATM
community members applying the performance-based approach in their area of interest. This terminology is introduced
progressively, throughout the various chapters and sections of the document. Because some readers may not read the
document in its entirety or in sequential order, they may experience difficulty locating certain terminology
definitions. Consequently, an abbreviations and terminology list has been provided for quick-reference purposes in the
preliminary pages of this manual.

(v)
 ABBREVIATIONS AND TERMINOLOGY

1. ABBREVIATIONS

ACC Area control centre

ADS-B Automatic dependent surveillance — broadcast

AIRAC Aeronautical information regulation and control

ANS Air navigation system

ANSP Air navigation service provider

ATC Air traffic control

ATCO Air traffic controller

ATM Air traffic management

ATMRPP Air Traffic Management Requirements and Performance Panel

CFIT Controlled flight into terrain

CNS Communications, navigation and surveillance

DM Data mart

DW Data warehouse

ETL Extract, transform and load

FIR Flight information region

GANP Global air navigation plan

GPM Global Performance Manual

IATA International Air Transport Association

IFR Instrument flight rules

IMC Instrument meteorological conditions

ISO International Organization for Standardization

IT Information technology

(vii)
(viii) Manual on Global Performance of the Air Navigation System

KPA Key performance area

KPI Key performance indicator

MCDA Multi-criteria decision analysis

MDM Master data management

Mode S Mode select

OCD Operational Concept Document

OPS Operations

PBA Performance-based approach

PBN Performance-based navigation

PBTG Performance-based transition guidelines

PCA Process capability area

PIAC Peak instantaneous aircraft count

PIRG Planning and Implementation Regional Group

QoS Quality of Service

RASP Required air navigation system performance

RCP Required communication performance

RNP Required navigation performance

RSP Required surveillance performance

RTSP Required total system performance

SARPS Standards and Recommended Practices

SLA Service level agreement

SMART Specific, measurable, achievable, relevant and time-bound

SSR Secondary surveillance radar

SWOT Strengths, weaknesses, opportunities and threats

TFM Traffic flow management

TMA Terminal control area

Abbreviations and terminology (ix)

VFR Visual flight rules

VHF Very high frequency

VMC Visual meteorological conditions

2. TERMINOLOGY

Aggregation hierarchy. Aggregation hierarchies define how detailed (granular) performance data will be aggregated
into summary data, and vice versa, how to break down the summary data into details. The following is an
example of a sequence of aggregation levels: runway > airport > State > region > world (Chapter 2, 2.4.2,
Appendix C, 4.3.4 and 4.3.8, and Appendix D, 3.1 refer).

ATM community. The ATM community is defined in Appendix A of the Global Air Traffic Management Operational
Concept (Doc 9854). It comprises (in alphabetical order) the aerodrome community, the airspace providers, the
airspace users, the ATM service providers, the ATM support industry, the International Civil Aviation
Organization (ICAO), the regulatory authorities and States (Chapter 1, 1.3.2 refers).

Baseline performance. The term baseline performance refers to the performance, as described by a collection of
defined indicators, in a given set of years. This performance may be the planned baseline performance
(incorporating planned improvements), the measured baseline performance (based on historical data) or a
projected baseline performance (without planned improvements, sometimes referred to as a “do-nothing”
scenario). The projected baseline performance is expected to be used when analysing the need for planned
improvements (Chapter 2, 2.4.3 refers).

Classification scheme. A classification scheme defines how general object types and event types can be more
specifically characterized in terms of subtypes. For example: air vehicle < aircraft < aircraft category < aircraft
class < aircraft type < aircraft model < aircraft series < airframe.

Classification schemes allow the breakdown of summary data into details, and vice-versa, they can be used to
aggregate detailed (granular) performance data into summary data (Chapter 2, 2.4.2, Appendix C, 4.3.4 and
4.3.8, and Appendix D, 3.1 refer). See also: aggregation hierarchy.

Core data warehouse. The function of a core data warehouse is to facilitate data loading, integration and quality
assurance. So, the function of a core data warehouse is not to make the data available to its users. Instead, for
the users’ purposes, i.e. for performance data analysis and dissemination, separate data marts should be built
(Appendix C, 4.3.6 refers). See also: Data warehouse, Data mart.

Data management. Data management is the process of data collection, processing (including quality assurance),
storage and reporting in support of the performance-based approach.

In practical terms, it is about: how to set up the data acquisition process needed for performance monitoring;
how to aggregate performance data and exchange the data between planning groups; how groups can best
manage their information base in which performance data are stored; and how to organize performance
evaluations. Other subjects under this heading include global harmonization of such work, including definitions,
standards for reporting requirements, and information disclosure (Chapter 1, 1.4.6, Chapter 2, 2.7, Appendix A,
3.8 and Appendix B refer).

Data mart (DM). A data mart is a specialized version of a data warehouse. The key difference is that the creation of a
data mart is predicated on a specific, predefined need for a certain grouping and configuration of select data
(x) Manual on Global Performance of the Air Navigation System

from a core data warehouse. The primary function of data marts is to make data available to users, i.e. to
support performance data analysis and dissemination (Appendix C, 4.3.7 refers).

Data profiling. Data profiling is a technique used to support performance data quality assurance. It creates a “profile” or
picture of the composition of the whole set of data from different perspectives. Its role is to identify anomalies in
large quantities of incoming performance data, while it is still being held in the staging area (Appendix C, 4.3.11
refers).

Data set. A data set is a group of records in a fact table which logically belong together. By including a data set
dimension, performance data managers can load data for different versions, realities, assumptions or scenarios
in the same table. Likewise, data sets can be used to distinguish between measured values (indicators) and
targets.

Data cannot and should not be aggregated over data sets, but the data sets can be compared against each
other: for example, different forecast scenarios, or different forecast editions, or measured values against
targets. The data set dimension is an important instrument for performance evaluation (Appendix C, 4.3.9
refers). See also: Dimension table.

Data warehouse (DW). A data warehouse contains data which is built out of separate internal and/or external data
sources where the data is integrated in a consistent manner. The database design of a data warehouse favours
data analysis and reporting in order to gain strategic insight and to facilitate decision-making. A data warehouse
contains “fact tables” and “dimension tables” which may be arranged as a “star schema” or a “snowflake
schema” (Appendix C, 4.3.2 and 4.3.5 refer).

Dimension table. In data warehousing, the possible values for each of the dimensions used in the various fact tables
are stored in separate tables, called the dimension tables. Dimension tables contain a list of items called
dimension objects. These can be physical objects (e.g. airframes, airports, countries, regions), but also
intangible objects such as classes and categories (e.g. aircraft types, flight rules, wake turbulence categories,
months, severity classifications) (Appendix C, 4.3.4, 4.3.5 and 4.3.8 refer).

Extract, transform and load (ETL). The loading of data into a data warehouse is usually referred to by the term ETL
process (extract, transform and load). This is a process in data warehousing that involves extracting data from
outside sources, transforming it to fit performance monitoring needs, and ultimately loading it into the data
warehouse. ETL is how data actually gets loaded into the warehouse (Appendix C, 4.3.10 refers).

Fact table. Within a data warehouse, all performance data (e.g. the values of supporting metrics, performance indicators
and performance targets) are stored in fact tables. Each fact (described by a set of metrics and/or indicator
values) has a dimensional context which defines the scope covered by the value of the metrics/indicators
(Appendix C, 4.3.3 and 4.3.5 refer). See also: Dimension table.

Focus area. Within each KPA or PCA, a number of more specific areas — focus areas — are identified in which there
are potential intentions to establish performance management. Focus areas are typically needed where
performance issues have been identified. For example, within the capacity KPA one can identify airport
capacity, runway capacity and apron capacity as focus areas. Within the safety KPA, the list of focus areas
might include: accidents, incidents, runway incursions, safety management system maturity, etc. There may be
a need to define hierarchical groupings of focus areas (Chapter 2, 2.3.3 and Appendix A, Figure I-A-2 and 3.4
refer).

Global ATM performance hierarchy. This is a construct which is useful for illustrating (and reminding the reader) that
the performance-based approach can be applied at different levels, ranging from high-level socio-political
issues to lower-level technology. The hierarchy distinguishes between five levels: 1) political and socio-
economic requirements; 2) required air navigation system performance (RASP); 3) required total system
Abbreviations and terminology (xi)

performance (RTSP) at the level of ATM component functionality; 4) system performance requirements (e.g.
RNP, RCP); 5) technology performance (including standards, specifications) (Appendix A, 2 refers).

Indicator. See: Performance indicator.

Influence diagram. Influence diagrams are used to depict a chain of (assumed validated) cause-effect relationships.
They can be used in a simple way to depict a high-level, qualitative understanding of performance. When such
diagrams are used in a more complex way (supported by influence modelling tools), they follow precise
conventions which serve to document relationships between (supporting) metrics (via processes and
mechanisms), and to link these to well-defined performance indicators. Influence diagrams are useful as a tool
to develop and document an initial understanding of the performance behaviour of the system. They can also
serve as a communication tool to explain performance behaviour to a non-technical audience (Appendix D,
3.2.3 refers).

Key performance area (KPA). KPAs are a way of categorizing performance subjects related to high-level ambitions
and expectations. ICAO has defined 11 KPAs: safety, security, environmental impact, cost effectiveness,
capacity, flight efficiency, flexibility, predictability, access and equity, participation and collaboration,
interoperability (Chapter 2, 2.2.4, Appendix A, Figure I-A-2 and 3.3 refer).

Key performance indicator (KPI). See: Performance indicator.

Master data management (MDM). The function of maintaining a common framework of dimensions throughout
disparate IT systems and among a variety of stakeholders is called master data management (MDM)
(Appendix C, 4.3.8 refers).

Metric. See: Supporting metric.

Multi-criteria decision analysis (MCDA). Multi-criteria decision analysis is a discipline aimed at supporting decision-
makers who are faced with making numerous and conflicting evaluations. MCDA aims at highlighting these
conflicts and deriving how to arrive at a compromise in a transparent process (Chapter 2, 2.5.5 and Appendix B,
4.5 refer).

Performance-based approach (PBA). The performance-based approach is a decision-making method based on three
principles: strong focus on desired/required results; informed decision-making driven by those desired/required
results; and reliance on facts and data for decision-making. The PBA is a way of organizing the performance
management process (Chapter 1, 1.2.1 and Chapter 2, 2.1 refer). See also: Performance management process.

Performance case. The term performance case is used to denote the documentation that contains all the reasoning
and arguments used to demonstrate that the performance objectives (and performance targets) will be met. A
performance case can be seen as the combination of the various cases that together address and balance all
areas in which the ATM community has expectations, e.g. the safety case, together with the business case,
together with the environment case (Chapter 1, 1.4, Chapter 2, 2.5.3 and 2.5.5 and Appendix D, 3.2.7 refer).

Performance framework. A performance framework is the set of definitions and terminology describing the building
blocks used by a group of ATM community members to collaborate on performance management activities.
This set of definitions includes the levels in the global ATM performance hierarchy, the eleven key performance
areas, a set of process capability areas, focus areas, performance objectives, indicators, targets, supporting
metrics, lists of dimension objects, their aggregation hierarchies and classification schemes (Appendix A refers).
(xii) Manual on Global Performance of the Air Navigation System

Performance gap. The difference between the baseline performance and the target performance is referred to as the
“performance gap” (Chapter 2, 2.4.3 and Appendix A, 3.7 refer). See also: Baseline performance, Target
performance.

Performance indicator. Current/past performance, expected future performance (estimated as part of forecasting and
performance modelling), as well as actual progress in achieving performance objectives is quantitatively
expressed by means of indicators (sometimes called key performance indicators, or KPIs).

To be relevant, indicators need to correctly express the intention of the associated performance objective.
Since indicators support objectives, they should be defined having a specific performance objective in mind.

Indicators are not often directly measured. They are calculated from supporting metrics according to clearly
defined formulas, e.g. cost-per-flight-indicator = Sum(cost)/Sum(flights). Performance measurement is therefore
done through the collection of data for the supporting metrics (Chapter 1, 1.4.6, Chapter 2, 2.4.2, Appendix A,
Figure I-A-2 and 3.7, Appendix C, 4.3.3 and Appendix D refer).

Performance management process. This term refers to a repetitive or continuous process which applies the principles
of the performance-based approach to manage (generally improve) selected performance aspects of an
organization or system (i.e. the air navigation system). The process is executed through a sequence of well-
defined steps, which are described in Chapter 2, Figure I-2-1.

Examples of performance management processes are safety management, security management, and
capacity management.

Performance modelling. Within the context of the performance-based approach, the objective of modelling is not to
explain how the air navigation system works in terms of data flows, messages etc., but to construct ATM
performance models which help to — qualitatively and/or quantitatively — understand the cause-effect
relationships between performance variables, showing how individual performance objectives can be achieved
and how they interact (enhance or interfere) with each other. This enables understanding of how performance
levels can be achieved by operational improvements and what the trade-offs are.

Because performance is measured in terms of metrics and indicators, it follows that the variables in
performance models should include all metrics from which the indicators are derived, as well the indicators
defined in the performance framework (Appendix D, 3.1 refers).

Performance monitoring. Performance monitoring is the act of collecting performance data in support of the
performance review (Chapter 2, 2.7 refers). See also: Performance review.

Performance objective. Within focus areas, the potential intention to establish performance management is “activated”
by defining one or more performance objectives. These define — in a qualitative and focused way — a desired
trend from today’s performance (e.g. improvement). A distinction is made between generic objectives and
instantiated objectives.

Generic objectives specifically focus on what has to be achieved, but do not make statements about the when,
where, who or how much. For example “improve safety” is not specific enough to be an objective, whereas
“reduce the total number of accidents” and even more specifically “reduce the number of CFIT accidents” would
qualify as performance objectives. Because at the level of generic objectives no mention is made about the
when, where and who, it does not make sense to try to associate numbers (indicator values or targets) with this
level.

Instantiated objectives add the when, where, who and how much to the generic objectives. Instantiated
objectives can have indicator values and associated targets (Chapter 2, 2.3.3, Appendix A, Figure I-A-2, 3.5
and 3.6 refer).
Abbreviations and terminology (xiii)

Performance review. Performance review is the act of analysing performance data in support of performance
management (Chapter 2, 2.7 refers).

Performance target. Performance targets are closely associated with performance indicators: they represent the values
of performance indicators that need to be reached or exceeded to consider a performance objective as being
fully achieved (Chapter 2, 2.4.3, Appendix A, Figure I-A-2 and 3.6 refer).

Priority. The notion of priorities is used to ensure that the performance-based approach is applied at opportune
moments when and where it will bring desired benefits, and to maintain acceptable performance levels in KPAs,
which are critical for meeting general expectations and realizing strategic ambitions (Appendix B, 3 refers).

Process capability area (PCA). Whereas KPAs zoom in on a particular type of performance outcome (e.g. safety,
capacity, etc.), PCAs slice the performance subject from a different angle, focussing on the quality, maturity
and completeness of the performance management processes that ATM community members have put in
place. The process-oriented perspective helps to diagnose weaknesses in the existing performance
management processes, after which specific initiatives can be taken (performance based, i.e. with appropriate
objectives, indicators and targets) to improve these processes and/or their deployment across the ATM
community (Appendix A, 5 and Appendix D, 3.2.2 refer).

Risk management. The systematic application of management policies, procedures and practices to the tasks of:
establishing the context of, identifying, analysing, evaluating and treating risks; monitoring the implementation
of treatments; and communicating about risk.

Staging area. A staging area is a part of a data warehouse where raw source data is placed, prior to quality checking,
transformation and subsequent loading into fact tables and dimension tables (Appendix C, 4.3.10 and 4.3.11
refer).

Stakeholder. In this document, a stakeholder is any ATM community member who has an interest in, or is involved in,
ATM performance management.

Supporting metric. Supporting metrics are used to calculate the values of performance indicators. For example, cost-
per-flight-indicator = Sum(cost)/Sum(flights). Performance measurement is done through the collection of data
for the supporting metrics (e.g. this leads to a requirement for cost data collection and flight data collection)
(Chapter 2, 2.4.2, Appendix A, Figure I-A-2, 3.8, Appendix B, 3, Appendix C, 4.3.3 and Appendix D refer).

SWOT analysis. SWOT analysis is a business management term used to denote the analysis of a system or
organization with the aim of developing an inventory of present and future Strengths, Weaknesses,
Opportunities and Threats that may require performance management attention (Chapter 2, 2.3.2 and
Appendix D, 3.2.7 refer).

Target performance. See: Performance target.

Taxonomy. See: Classification scheme.

Trade-off. The role of trade-offs is explained as an instrument for choosing the appropriate — that is, the most balanced
— solution within a given set of priorities, when different options are available, but each option has different
advantages and disadvantages in terms of performance impact (Chapter 2, 2.5.5 and Appendix B, 4 refer).

_____________________
 PART I

GLOBAL PERFORMANCE
 TABLE OF CONTENTS

Page

Chapter 1. Introduction ..................................................................................................................................... I-1-1

1.1 About this document ............................................................................................................................ I-1-1
1.2 The performance-based approach and its advantages ....................................................................... I-1-3
1.3 Purpose and applicability of the performance-based approach ........................................................... I-1-4
1.4 The way to success ............................................................................................................................. I-1-7

Chapter 2. The performance-based approach step-by-step .......................................................................... I-2-1

2.1 Introduction .......................................................................................................................................... I-2-1
2.2 Step 1: Define/review scope, context and general ambitions/expectations ......................................... I-2-3
2.3 Step 2: Identify opportunities, issues and set (new) objectives ........................................................... I-2-5
2.4 Step 3: Quantify objectives .................................................................................................................. I-2-8
2.5 Step 4: Select solutions to exploit opportunities and resolve issues .................................................... I-2-13
2.6 Step 5: Implement solutions ................................................................................................................ I-2-22
2.7 Step 6: Assess achievement of objectives .......................................................................................... I-2-23
2.8 Repeat the process ............................................................................................................................. I-2-26

Chapter 3. Getting started ................................................................................................................................ I-3-1

3.1 Introduction .......................................................................................................................................... I-3-1
3.2 Develop a good understanding of the performance-based approach .................................................. I-3-2
3.3 Assess currently used approach ......................................................................................................... I-3-2
3.4 Define implementation priorities .......................................................................................................... I-3-3
3.5 Start with a limited scope..................................................................................................................... I-3-3
3.6 Establish commitment and collaboration ............................................................................................. I-3-4

Chapter 4. What to expect from the Appendices in this document .............................................................. I-4-1
4.1 Appendix A – Performance framework ................................................................................................ I-4-1
4.2 Appendix B – Priorities, trade-offs and risks ........................................................................................ I-4-1
4.3 Appendix C – Data and information management ............................................................................... I-4-2
4.4 Appendix D – Performance analysis and modelling ............................................................................ I-4-2
4.5 Appendix E – Metrics and indicators ................................................................................................... I-4-3

Chapter 5. References ....................................................................................................................................... I-5-1

Appendix A. Performance framework .............................................................................................................. I-App A-1

1. Purpose ............................................................................................................................................... I-App A-1
2. Performance hierarchy ........................................................................................................................ I-App A-2
3. Measurement taxonomy ...................................................................................................................... I-App A-4
4. The need for a structured view of the air navigation system................................................................ I-App A-8
5. Performance process maturity ............................................................................................................. I-App A-9

Appendix B. Priorities, trade-offs and risks .................................................................................................... I-App B-1

1. Introduction .......................................................................................................................................... I-App B-1
2. Reference to the performance management process ......................................................................... I-App B-1

I-(i)
I-(ii) Manual on Global Performance of the Air Navigation System

3. Priorities .............................................................................................................................................. I-App B-2

4. Trade-offs ............................................................................................................................................ I-App B-3
5. Risks.................................................................................................................................................... I-App B-6

Appendix C. Data and information management ............................................................................................ I-App C-1

1. Introduction .......................................................................................................................................... I-App C-1
2. Issues driving the set-up of the performance monitoring process ....................................................... I-App C-1
3. Data feeds and information disclosure ................................................................................................ I-App C-2
4. Performance data management: basic concepts ................................................................................ I-App C-4
5. Performance evaluation....................................................................................................................... I-App C-13

Appendix D. Performance analysis and modelling ........................................................................................ I-App D-1

1. Introduction .......................................................................................................................................... I-App D-1
2. Role within the performance management process ............................................................................ I-App D-1
3. Practical guidance for performance modelling ..................................................................................... I-App D-2

Appendix E. Metrics and indicators ................................................................................................................. I-App E-1

1. Commonality in ATM performance indicators ...................................................................................... I-App E-1
2. KPA 01 — Access and equity.............................................................................................................. I-App E-1
3. KPA 02 — Capacity ............................................................................................................................. I-App E-2
4. KPA 03 — Cost effectiveness ............................................................................................................. I-App E-3
5. KPA 04 — Efficiency ........................................................................................................................... I-App E-4
6. KPA 05 — Environment....................................................................................................................... I-App E-4
7. KPA 06 — Flexibility ............................................................................................................................ I-App E-5
8. KPA 07 — Global interoperability ........................................................................................................ I-App E-5
9. KPA 08 — Participation by the ATM community ................................................................................. I-App E-5
10. KPA 09 — Predictability ...................................................................................................................... I-App E-6
11. KPA 10 — Safety ................................................................................................................................ I-App E-6
12. KPA 11 — Security .............................................................................................................................. I-App E-7

Figures

Figure I-1-1. Role of the global performance manual .......................................................................................... I-1-2

Figure I-2-1. General performance management process ................................................................................... I-2-2
Figure I-2-2. Expected performance of candidate solutions (example)................................................................ I-2-18
Figure I-2-3. Selection of solutions ...................................................................................................................... I-2-20
Figure I-2-4. Scope of performance monitoring and review ................................................................................. I-2-24
Figure I-A-1. Performance hierarchy ................................................................................................................... I-App A-2
Figure I-A-2. Illustration of measurement taxonomy ............................................................................................ I-App A-4
Figure I-B-1. Illustration of trade-offs between key performance indicators (KPIs) .............................................. I-App B-6
Figure I-B-2. Risk management principles ........................................................................................................... I-App B-7
Figure I-C-1. Fact and dimension tables: snowflake schema .............................................................................. I-App C-7

Tables

Table I-2-1. Selection of solutions (example) ...................................................................................................... I-2-19

Table I-A-1. Example of process capability areas (PCAs) ................................................................................... I-App A-9
Table I-C-1. Example fact table ........................................................................................................................... I-App C-5

_____________________
 Chapter 1

INTRODUCTION

1.1 ABOUT THIS DOCUMENT

1.1.1 Context

1.1.1.1 Organizations in many industries have long since discovered the value of a performance-based approach
(PBA) for:

— improving the effectiveness of the day-to-day economic management of their business;

— channelling efforts towards meeting stakeholder expectations and improving customer satisfaction;
and

— managing change in a dynamic environment.

Likewise, the air traffic management (ATM) industry can reap significant benefits from adopting a performance-based
approach.

1.1.1.2 For the best results, ATM community members need to cooperate in a performance-based manner. For
this reason, ICAO supports and encourages the global adoption of performance management techniques, which is part
of the transition towards a performance-based global air navigation system (ANS) as envisaged in the Global Air Traffic
Management Operational Concept 1 (Doc 9854) and the related Manual on Air Traffic Management (ATM) System
Requirements2 (Doc 9882).

1.1.1.3 As illustrated in Figure I-1-1, one can distinguish between basic (common) performance management
terminology and techniques, and their application in economic management, transition management, safety
management, and other applications.

1.1.1.4 This manual addresses the basic performance management terminology and techniques that are the
“common denominator” between all performance planning/management applications. The specifics of each application
are addressed in other related documents, including but not limited to the:

— Manual on Air Navigation Services Economics (Doc 9161)

— Global Air Navigation Plan (Doc 9750)

— Safety Management Manual (SMM) (Doc 9859)

1. The global ATM operational concept presents a vision for an integrated, harmonized, and globally interoperable air navigation
system planned up to 2025 and beyond.
2. This manual identifies requirements where a significant change to operating practices will be required to transition to the global
ATM operational concept.

I-1-1
I-1-2 Manual on Global Performance of the Air Navigation System

Transition management
Economic management

Safety management

Other applications
Application of
performance management
techniques

Basic and common

Global Performance Manual performance management
techniques

Figure I-1-1. Role of the global performance manual

1.1.2 Using this manual

1.1.2.1 The methods and procedures described in this manual have been compiled from experience gained in the
successful development and management of performance management activities in various sectors of both the aviation
and non-aviation industries.

1.1.2.2 This manual targets a wide audience ranging from ICAO experts, Planning and Implementation Groups
(PIRGs), State aviation regulators, operators and service providers. It also aims to address all levels of personnel from
senior management to front-line workers.

1.1.2.3 The manual also covers a wide range of subjects; therefore, users are encouraged to first read Chapters 1
to 5 before focusing on their specific areas of interest. It is possible that more specific ICAO guidance material is
available in your area of interest. If that is the case, you are invited to use this manual to ensure that the performance-
based perspective is duly taken into account.

1.1.2.4 This manual is aimed at those personnel who are responsible for designing, implementing and managing
effective performance management activities. Users should find sufficient information herein for the justification, initiation
and operation of a viable performance management process in their organization. The manual is not prescriptive.
However, based on an understanding of the philosophy, principles and practices discussed herein, organizations should
be able to develop an approach to performance management suited to their local conditions.

1.1.3 Content and structure

1.1.3.1 This manual contains guidance material intended to:

— provide “getting started” assistance to those ATM community members who are (relatively)
inexperienced in this subject; and

— assist the experienced ATM community members in converging towards a globally harmonized and
agreed upon approach.
Part I — Global Performance
Chapter 1. Introduction I-1-3

1.1.3.2 The main body of the document summarizes the three most frequently asked questions about the
performance-based approach:

— What is it for? The basic principles, advantages, purpose, applicability and success factors are
explained in Chapter 1 (Introduction). This chapter should help you in deciding where and when to
implement (or modify) the performance-based approach in your organization.

— What is it really? The performance-based approach is a decision-making method. Chapter 2 (The

performance-based approach) provides a step-by-step description with an example. The purpose of
this chapter is to show that the approach can be broken down into manageable, easy-to-understand
steps. By following these steps, ATM community members should gain confidence in their ability to
apply the approach in a successful manner, and benefit from participating in a globally harmonized
approach.

— How do I start? Some guidelines on how to get yourself (your PBA “project”) organized are offered in
Chapter 3 (Getting Started).

1.1.3.3 Appendices provide more specific or technical guidance on selected subjects. Chapter 4 provides an
overview (abstract) of the contents of these appendices.

1.2 THE PERFORMANCE-BASED APPROACH AND ITS ADVANTAGES

1.2.1 Principles

In general terms, the performance-based approach is based on the following three principles:

— Strong focus on desired/required results. Instead of prescribing solutions, desired/required

performance is specified. The attention of management is shifted from a resource and solution-centric
view (How will we do it?) towards a primary focus on desired/required performance results (What is
the outcome we are expected to achieve?). This implies determining the current performance
situation, what the most appropriate results should be, and clarifying who is accountable for achieving
those results.

— Informed decision-making, driven by the desired/required results. This means working “backwards”
from the “what”—the primary focus—to decisions about the “how”. “Informed decision-making”
requires that those making the decisions (decision-makers) develop a good understanding of the
mechanisms which explain how drivers, constraints, shortcomings, options and opportunities influence
(i.e. contribute to, or prevent) the achievement of the desired/required results. Only then can decisions
— in terms of priorities, trade-offs, selection of solutions and resource allocation — be optimized to
maximize the achievement of the desired/required (performance) results.

— Reliance on facts and data for decision-making. In the performance-based approach the
desired/required results, as well as the drivers, constraints, shortcomings and options, are expressed
in quantitative terms and in a qualitative way. The rationale for this is that “if you can’t measure it, you
can’t manage it”, i.e. unless you measure something, you don’t know if it is getting better or worse.
When facts and data are used, they should be relevant and reflect reality. This requires the adoption
of a performance measurement culture. It also necessitates associated investments in data collection
and management.
I-1-4 Manual on Global Performance of the Air Navigation System

1.2.2 Advantages

The advantages of the performance-based approach are:

— it is result-oriented, allows customer focus and promotes accountability;

— policy-making becomes transparent when the goals are publicly stated in terms of performance
outcome rather than solutions;

— the shift from prescribing solutions to specifying desired/required performance also gives more
freedom and flexibility in selecting suitable solutions, which in turn is a catalyst for more cost-
effectiveness. Furthermore, solutions can be easily adapted in a diverse and changing environment.

— exclusive bottom-up approaches (“technology-driven approach” and “solutions searching for a problem
to solve”) are easier to avoid;

— reliance on anecdotal evidence can be replaced by a rigorous scientific approach employing

quantitative and qualitative methods;

— the focus on desired/required results helps decision-makers set priorities, make the most appropriate
trade-offs, choose the right solutions and perform optimum resource allocation;

— organizations are successful in reaching goals, i.e. the general effect of the approach is that it ensures
improved predictability of benefits;

— it is worth the investment: the adoption of a performance-based approach typically results in cost-
savings (cost avoidance) significantly higher, by comparison, than the cost of applying the approach.

1.3 PURPOSE AND APPLICABILITY OF THE PERFORMANCE-BASED APPROACH

1.3.1 Purpose

1.3.1.1 The performance-based approach is a pragmatic tool for ATM community members involved in:

— policy-making;

— regulation;

— transition planning (planning changes to the system);

— system design and validation (developing changes to the system);

— management (economic and operational); and

— continuous improvement (optimizing the system).

1.3.1.2 The basic terminology, methods and techniques explained in this manual are intended to effect better
collaboration between the above activities and audiences, resulting in a “balanced total system approach”.

1.3.1.3 This “balanced total system approach” enables the ATM community to better work together across all
disciplines to:
Part I — Global Performance
Chapter 1. Introduction I-1-5

— develop a consensus on “the direction to follow” with the air navigation system;

— manage the required change in a dynamic environment; and

— improve the effectiveness of the day-to-day operational and economic management, thereby meeting
long-term and short-term stakeholder expectations and improving customer satisfaction.

1.3.2 Applicability

The performance-based approach (PBA) provides benefits in almost every subject area and can be applied by almost
every audience at any level of detail. The list below gives an overview of the applicability from a variety of perspectives.
It also delineates the applicability and target audience for this manual.

— The PBA can and should be used in subject areas such as:

• safety, security, environmental impact of air transport and ATM;

• economic performance of airspace users, airports and air navigation service providers;

• operational performance (including quality of service) of flight operations, airport operations and
the provision of air navigation services;

• human performance and social factors within the air navigation system; and

• performance of technical systems within the air navigation system.

— It can and should be used in all of the following planning and management activities:

• policymaking;

• planning;

• research, development and validation;

• economic management; and

• operational management.

— It can and should be used by all ATM community members:

• air navigation service providers (ANSPs);

• airports;

• airspace users;

• manufacturers;

• regulators;

• States; and
I-1-6 Manual on Global Performance of the Air Navigation System

• ICAO (including panels and planning groups).

— It applies to the management of performance at a wide variety of levels of detail or aggregation:

• from a geographical perspective: at the level of local performance (States), regional performance,
or global performance;

• from a time period perspective: at the level of momentary performance (real time), or more
aggregated, i.e. hourly, daily, weekly, monthly, quarterly, seasonal, annual, or multi-year
performance results;

• from an operational perspective: at the level of parts of operations (e.g. performance of individual
flight phases), individual operations (e.g. gate-to-gate performance of individual flights), or the
aggregate of operations (e.g. for optimizing the collective performance of groups of flights);

• from a stakeholder aggregation perspective: at the level of individual operational units/entities

(e.g. specific ACCs), individual stakeholder organizations (e.g. specific ANSPs), or stakeholder
segments (e.g. collective performance of ANSP groups or all ANSPs).

1.3.3 Interdependencies and the need for collaboration

1.3.3.1 There are interdependencies between all of the above perspectives. In the past, these interdependencies
have not always been recognized, resulting in non-optimum performance of the air navigation system as a whole. For
example:

— subject areas have developed into distinct performance-based approaches resulting in different
terminology and other variations; this makes it difficult to detect the commonalities and to manage
interdependencies and integrate results;

— insufficient coordination between policymaking, planning, research/development/validation, economic

management, and operational management;

— where insufficient coordination between ANSPs, airports, airspace users, manufacturers, regulators
and ICAO takes place, the result is a fragmented air navigation system;

— insufficient coordination at the local level, regional level, and global level leads to less than ideal
interoperability and to geographic differences in terms of performance and maturity;

— a fragmented approach from an operational perspective (no gate-to-gate and en-route to en-route)
leads to less than optimum flight efficiency and airport operations efficiency.

1.3.3.2 To resolve such issues and achieve better results, all ATM community members need to cooperate in a
performance-based manner across geographic boundaries, across operational (e.g. flight phase) boundaries, across
subject areas and across planning and management activity boundaries.

1.3.4 Performance management application examples

There are many areas in which the migration to a performance-based approach has already started. Examples include:

— transition planning at global, regional and local levels;

Part I — Global Performance
Chapter 1. Introduction I-1-7

— economic performance management for ANSPs;

— performance-based navigation (PBN); and

— safety management.

1.4 THE WAY TO SUCCESS

1.4.1 Once an organization (or State, region, etc.) decides to adopt the performance-based approach in a
particular subject area, it must acknowledge that a number of elements are essential to the successful application of the
approach. These elements are:

— commitment;

— agreement on goals;

— organization;

— human resources and knowledge/expertise;

— data collection, processing, storage and reporting;

— collaboration and coordination; and

— cost implications.

1.4.2 Commitment

1.4.2.1 The decision-makers (e.g. senior management) need to be convinced that the approach will bring real
value. After all, it is their tool because the approach supports result-oriented management, i.e. helps to achieve real
results based on a “performance case” containing solid, quantified arguments, as outlined in Part II of this manual.

1.4.2.2 Commitment from the senior management will ensure that:

— the effort and money spent on the collection and analysis of performance data are actually used to
improve the effectiveness of decision-making; and

— decision-making is effectively supported by the availability of trustworthy, meaningful data.

1.4.2.3 In other words, it is the responsibility of senior management to ensure a healthy balanced relationship
between performance-based decision-making, and performance data collection and analysis.

1.4.3 Agreement on goals

The main principle of the performance-based approach is a strong focus on desired/required results. The ability to reach
consensus on the desired outcome of performance management in terms of performance results to be achieved (i.e. to
agree on objectives and targets), is a basic prerequisite for the successful application of the approach.
I-1-8 Manual on Global Performance of the Air Navigation System

1.4.4 Organization

1.4.4.1 The process which underlies the performance-based approach is introduced in Chapter 2. The steps of
that process require a number of roles and responsibilities. This means that it must be clear who (or which organization)
is accountable and responsible for:

— defining (regulatory and management) objectives;

— setting targets (regulatory and management);

— defining indicators;

— gathering performance data;

— managing data quality; and

— performance review.

1.4.4.2 Overlaps and gaps in these responsibilities are to be avoided. Depending on the complexity, the
organizational aspects will need to be addressed at different levels:

— in simple cases, this means assigning roles and responsibilities to individuals;

— in more complex situations (application in large organizations), the roles and responsibilities should be
reflected in the organizational structure; and

— at State and regional levels, specific institutional arrangements may be required (setting up statistical
offices, performance review organizations, governance structures, such as regulatory bodies).

1.4.5 Human resources and knowledge/expertise

1.4.5.1 Applying the performance-based approach requires a certain amount of effort; hence, it is necessary that
the organizational structure is sufficiently staffed.

1.4.5.2 In simple cases, performance management roles and responsibilities may be added to the job description
of existing staff. With increasing workload, successful application of the approach will require dedicated staff, which is
able to spend 100 per cent of their time on the subject. In particular, data collection and management, and also
performance review are all labour-intensive.

1.4.5.3 Likewise, it is essential that the required knowledge/expertise is available: successful application of the
performance-based approach requires a certain culture and skills which may not be readily available in the organization.
It must be ensured that the staff involved has a suitable background and receives additional training as required3.

3. Required knowledge/expertise includes an indepth understanding of ATM performance and the surrounding issues of data
collection, information management, performance modelling, performance review, forecasting, etc. The team should include system
analysts, statisticians, database specialists, etc.
Part I — Global Performance
Chapter 1. Introduction I-1-9

1.4.6 Data collection, processing, storage and reporting

1.4.6.1 Data collection, processing, storage and reporting are fundamental to the performance-based approach.

1.4.6.2 It should not be assumed that all data needed for applying the performance-based approach are simply
available “somewhere” and only needs to be copied. Although re-use of data prepared by others is sometimes possible,
the data reporting chain always starts at the “grass-roots level”, and properly setting up and managing the entire chain is
an integral part of the approach.

1.4.6.3 Establishing a data reporting chain usually involves participation from many ATM community members.
Their willingness to participate requires the establishment of a performance data reporting culture (i.e. to consider the
challenges surrounding safety event reporting), a capability to successfully manage disclosure and confidentiality
aspects, and deciding on a case-by-case basis which approach works best: mandatory or voluntary (incentive-based)
reporting.

1.4.6.4 In the end, data will be condensed into a few indicators which represent the high-level knowledge about
the performance of the system, but prior to that, data from a variety of sources will need to be collected, quality-checked
and stored. This requires initiatives at the technical level: harmonization and standardization of reporting requirements,
and investments in appropriate information technology (IT) infrastructure.

1.4.6.5 The above-mentioned harmonization and standardization of reporting requirements is needed to allow
consistent interpretation of data across different reporting sources. This in turn is a prerequisite for meaningful
benchmarking (comparison of reporting sources) and aggregation of data (calculation of totals across reporting sources).

1.4.7 Collaboration and coordination

1.4.7.1 The performance-based approach is never applied in isolation. There is always interaction within a context,
consisting of other subject areas, other stakeholders, other geographical areas, higher or lower aggregation levels, other
planning and management activities, etc.

1.4.7.2 Collaboration and coordination is needed to:

— come to an agreed vision on the expected results;

— ensure that everyone delivers their part of (their contribution to) the required performance;

— ensure that everyone uses a compatible approach, method and terminology; and

— ensure that everyone’s data can be integrated and aggregated to calculate overall indicators and
assess system performance at a higher aggregation level.

Paragraph 1.3.1.3 gave some examples on what can happen if there is insufficient emphasis on collaboration and
coordination.

1.4.8 Cost implications

Many of the above-mentioned success factors have cost implications: in all but the simplest cases, the PBA will require
specialized organizations, organizational units or staff working on the subject. In addition, a dedicated IT infrastructure
may be required.

_____________________
 Chapter 2

THE PERFORMANCE-BASED APPROACH STEP-BY-STEP

2.1 INTRODUCTION

2.1.1 The performance-based approach is a way of organizing the performance management process. Many
variations to this process are in use today. They are all based on a similar philosophy and principles which are covered
in more detail in the following paragraphs.

2.1.2 Without intending to be prescriptive, this chapter presents a “generalized” version of this process, together
with a step-by-step description and a worked-out (hypothetical) example. Its purpose is to show that — no matter in
which subject area it is applied — the approach can be broken down into manageable, easy-to-understand steps. By
systematically following these steps, ATM community members will gain confidence in their ability to apply the approach
in a successful manner, and benefit from participating in a globally harmonized approach.

2.1.3 Figure I-2-1 outlines the general sequence of steps in the performance management process. It serves as
general guidance.

2.1.4 There is a clear relationship between the three principles presented in 1.2.1 and the steps of the process:

— strong focus on desired/required results: is covered by Steps 1 and 2;

— informed decision-making, driven by the desired/required results: is dealt with by Steps 4 and 5;

— reliance on facts and data for decision-making: is enabled by Steps 3 and 6.

2.1.5 The remainder of this chapter explains Figure I-2-1 in more detail. More specifically, the steps cover the
following activities:

— Step 1: Define/review scope, context and general ambitions/expectations

• Step 1.1: Define scope

• Step 1.2: Define context

• Step 1.3: Identify ambitions and expectations

— Step 2: Identify opportunities, issues and set (new) objectives

• Step 2.1: Develop a list of present and future opportunities and issues that require performance
management attention

• Step 2.2: Focus efforts by defining and prioritizing performance objectives as needed

I-2-1
I-2-2 Manual on Global Performance of the Air Navigation System

1.
6. Define/review
Assess scope, context and
achievement general ambitions
of objectives and expectations

5. 2.
Implement Identify
solutions opportunities, issues
and set (new) objectives

4. 3.
Select solutions to Quantify
exploit opportunities objectives
and resolve issues

Figure I-2-1. General performance management process

— Step 3: Quantify objectives

• Step 3.1: Define how progress in achieving performance objectives will be measured and which
data are required to do so

• Step 3.2: Define the desired speed of progress in terms of baseline and target performance

— Step 4: Select solutions to exploit opportunities and resolve issues

• Step 4.1: Select the decisive factors to reach the target performance

• Step 4.2: Identify solutions to exploit opportunities and mitigate the effects of the selected drivers
and blocking factors

• Step 4.3: Select a sufficient set of solutions

— Step 5: Implement solutions

— Step 6: Assess achievement of objectives.

Part I — Global Performance
Chapter 2. The performance-based approach step-by-step I-2-3

2.2 STEP 1: DEFINE/REVIEW SCOPE, CONTEXT AND

GENERAL AMBITIONS/EXPECTATIONS

2.2.1 The purpose of Step 1 is to reach a common agreement on the scope and (assumed) context of the
“system” on which the performance management process will be applied, as well as a common view on the general
nature of the expected performance improvements.

2.2.2 Step 1.1: Define scope

2.2.2.1 In practice, there is not just one global, all-encompassing application of the performance management
process, but many simultaneous — and often interrelated — applications at more specialized and localized levels.
Chapter 1, 1.3.2 provides an indication of the possibilities for reduced scope.

2.2.2.2 Scope definition is important to avoid misunderstandings, in particular about the performance
(improvement) which can be expected within the given scope. For example, the possibilities for managing safety or
environmental impact vary depending on whether one considers only the role of ATM or approaches the subject at the
level of the entire air transport system (which includes, for example, changes in fleet composition, engine technology,
etc.).

2.2.2.3 By defining the scope of the performance management activity, the limits of responsibility and
accountability are also defined.

Example — Step 1.1

Organization X is the air navigation service provider (ANSP) in flight information region
(FIR) YYYY. The organization will be establishing a capacity management process as part
of an ongoing performance-based approach. The initial scope is defined as follows:

• Time period: traffic forecasting and issue analysis will be done for a 15-year time
horizon. The planning period (selection of solutions) will be initially limited to a 5-
year horizon (to support development of the business plan).

• Key performance areas: the main focus of the planning process will be on ATM
capacity.

• Geographically: the planning process will plan and implement ATM capacity in
FIR YYYY, but only en-route airspace is considered. Terminal control area (TMA)
capacity and airport capacity are initially not included. Airport expansion planning
(e.g. construction of new airports, runways, terminals) does not fall within the
scope.

• Traffic: the scope includes domestic, international and overflying traffic. However,
the scope of the planning process is limited to instrument flight rules (IFR) traffic.
Capacity management for visual flight rules (VFR) traffic is initially not included.
I-2-4 Manual on Global Performance of the Air Navigation System

2.2.3 Step 1.2: Define context

Once the scope is defined, it is necessary to make clear assumptions on what is “surrounding” the performance
management activity. This includes clarifying what the strategic fit is within a larger (parent scope) performance
management activity, with whom there is a need to coordinate and collaborate, and what the external drivers and
constraints are for the scope.

Example — Step 1.2

The capacity planning process for FIR YYYY fits within the regional planning process
(ICAO region ZZZ), which looks after the network effect. Organization X is responsible for
avoiding that FIR YYYY becomes the capacity bottleneck in region ZZZ. Regional
planning is also responsible for managing the VHF frequency allocation plan.

The geographical context includes neighbouring airspace, as well as the TMAs and
airports within FIR YYYY (because they were excluded from the scope).

The main external driver for capacity planning is the expected traffic growth. The
magnitude of growth and the expected changes in traffic patterns are documented in a
medium- and long-term traffic forecast. Physical airport capacity is a potential external
constraint.

2.2.4 Step 1.3: Identify ambitions and expectations

2.2.4.1 Within a given scope, the purpose of identifying general ambitions and expectations is to develop a
strategic view on the (performance) results that are expected. The term “expectation” refers to desired results from an
external perspective. The term “ambition” indicates that the desired results refer to an internal initiative.

2.2.4.2 For example, in ATM, the performance-based approach can be used to better meet society’s aviation
expectations, as well as improve the business performance of airlines, service providers, etc. To achieve this, one could
identify ambitions and expectations with regard to the performance of flight operations, airspace/airport usage and air
navigation services in areas such as:

— safety;

— security;

— environmental impact;

— cost effectiveness;

— capacity;

— flight efficiency;

— flexibility;
Part I — Global Performance
Chapter 2. The performance-based approach step-by-step I-2-5

— predictability;

— access and equity;

— participation and collaboration; and

— interoperability.

These are the eleven key performance areas (KPAs) as identified in the Global Air Traffic Management Operational
Concept (Doc 9854).

2.2.4.3 Achievable performance in the areas listed in 2.2.4.2 is made possible by the following enabler levels:

— services and procedures;

— human resources;

— physical infrastructure;

— systems and technology; and

— regulation and standardization.

2.2.4.4 The performance-based approach can and should be applied at each of these enabler levels for the
purposes of understanding the impact on the eleven KPAs. For example, for the systems and technology level, the focus
includes technical performance characteristics such as service/system availability, continuity, reliability, integrity,
resilience, maintainability, scalability etc. An important part of the PBA is the development of cause-effect relationships
between these technical performance characteristics and the higher level 11 KPAs.

Example — Step 1.3

The general expectation of the ATM community with regard to capacity is that the air
navigation system in FIR YYYY will meet airspace user demand at peak times and
locations by minimizing restrictions on traffic flow. To respond to future growth, capacity
must increase, along with corresponding increases in efficiency, flexibility, and
predictability while ensuring that there are no adverse impacts on safety or the
environment. The air navigation system must be resilient to service disruption, and the
resulting temporary loss of capacity.

2.3 STEP 2: IDENTIFY OPPORTUNITIES, ISSUES AND SET (NEW) OBJECTIVES

2.3.1 The purpose of Step 2 is to develop a detailed understanding of the performance behaviour of the system
(this includes producing a list of opportunities and issues), and to decide which specific performance aspects are
essential for meeting the general expectations. The essential performance aspects are those which need to be actively
managed (and perhaps improved) by setting performance objectives.
I-2-6 Manual on Global Performance of the Air Navigation System

2.3.2 Step 2.1: Develop a list of present and future opportunities and
issues that require performance management attention

2.3.2.1 Based on the scope, context and general ambitions/expectations which were agreed to during the previous
step, the system should be analysed in order to develop an inventory of present and future opportunities and issues
(weaknesses, threats) that may require performance management attention.

This part of the process is generally known as the SWOT (strengths, weaknesses, opportunities and threats) analysis.

— Strengths are (internal) attributes of a system or an organization that help in the realization of
ambitions or in meeting expectations.

— Weaknesses are (internal) attributes of a system or an organization that are a detriment to realizing
ambitions or meeting expectations.

— Opportunities are external conditions that help in the realization of ambitions or in meeting
expectations.

— Threats 1 are external conditions that are a detriment or harmful to realizing ambitions or meeting
expectations.

2.3.2.2 Note that what may represent strengths with respect to one ambition or expectation may be weaknesses
for another one. The term “issues” is used in this document to refer to weaknesses as well as threats.

2.3.2.3 A good understanding of the opportunities and issues should be developed early in the process, to provide
background information for deciding which performance objectives to set, what to measure and how/where to change
the system. When possible (second or later iteration in the process), advantage should be taken of the results of Step 6:
Assess achievement of objectives. In general, this activity should take place as part of the forecasting and performance
review.

2.3.2.4 Once the strengths, weaknesses, opportunities and threats are identified, action can be taken to target and
exploit or remove these factors, thereby leading to performance improvements directly related to meeting the
expectations.

Example — Step 2.1

Organization X has conducted a SWOT analysis on the air navigation system in FIR YYYY.
The conclusions have been summarized as follows:

• Strengths: currently, there is no staff shortage.

• Weaknesses:

• In the northern part of the FIR:

1. Term used in the general (business) sense of the word; not just to be seen within the context of security.
Part I — Global Performance
Chapter 2. The performance-based approach step-by-step I-2-7

— Currently there is no radar coverage. ATC services are provided in that area
using procedural control. At present, this is sufficient to handle the traffic levels
there, but within a few years procedural control and lack of surveillance will
become the blocking factors.

• In the southern part of the FIR the anticipated problems are:

— controller workload may become a blocking factor.

— traffic density is already very high, and frequency shortage may ultimately
prevent the ANSP from splitting sectors to increase capacity.

— the high number of aircraft simultaneously present in the airspace will also lead
to a degradation of surveillance performance (SSR capacity limits will be
exceeded).

— a wave of controller retirements is expected in ten years, leading to a staff

shortage.

• Opportunities: ADS-B technology has matured, and this is potentially a cost-effective

solution to the surveillance problem in the northern part of the FIR.

• Threats: in the coming years, various events could take place which could cause
traffic patterns of intercontinental flights to change. This could lead to a severe drop
in the volume of overflying traffic. There is a risk that this could happen after capacity
has been increased. Since the service provider obtains a significant part of the
revenue from user charges, this risk creates an unfavourable investment climate.

2.3.3 Step 2.2: Focus efforts by defining and prioritizing performance objectives as needed

2.3.3.1 The purpose of this activity is to focus and prioritize the application of the performance-based approach:

— focus is necessary to aim general expectations into specific performance objectives, which in turn will
be the basis for deciding on improvement actions.

— prioritization is required because, even though the scope of the process has already been limited, in
practice, not everything can and/or needs to be performance managed.

— prioritization is supported by risk management which helps identify the risks that are most urgent or
must be avoided, those that should be transferred or reduced, and those that are reasonable to retain.

2.3.3.2 Focusing the performance-based approach is a two-stage process:

— Within each KPA, identify a number of specific areas — focus areas — in which there are potential
intentions to establish performance management. Focus areas are typically needed where issues
have been identified (see Step 2.1). For example, within the capacity KPA one can identify airport
I-2-8 Manual on Global Performance of the Air Navigation System

capacity, runway capacity and apron capacity as focus areas. Within the safety KPA, the list of focus
areas might include: accidents, incidents, runway incursions, safety management system maturity, etc.
There may be a need to define hierarchical groupings of focus areas.

— Within focus areas, the potential intention to establish performance management is “activated” by
defining one or more performance objectives. These define — in a qualitative and focused way — a
desired trend in today’s performance (e.g. improvement); they specifically focus on what has to be
achieved, but do not make statements about the when, where, who or how much. These objectives
may be developed iteratively with the development of indicators. Significant analysis of historical data,
performance modelling or simulation may be required to understand the necessary objectives. For
example, “improve safety” is not specific enough to be an objective, but represents a starting point.
Further analysis would reveal that “reduce the total number of accidents” and even more specifically
“reduce the number of CFIT accidents” would qualify as performance objectives. Because at this level
of detail no mention is made about the when, where and who, it does not make sense to try to
associate numbers (indicator values or targets) at this level. That is done during the next step of the
process.

2.3.3.3 Prioritizing the performance-based approach means that performance objectives will only be defined in
those focus areas where a real (present or anticipated) need for action and improvement has been identified (preferably
through analysis of historical or projected performance data).

Example — Step 2.2

As stated in Step 1.1, the scope of the new performance management process was
defined as covering en-route ATM capacity for IFR flights.

Focus of performance management: one focus area is defined within the capacity KPA:
en-route ATM capacity for IFR flights.

Specific improvements: traffic growth will lead to higher airspace throughput, but also to
a higher number of aircraft simultaneously present in the airspace. Considering the results
of the SWOT analysis, the service provider decides that there is a need for two separate
performance objectives:

• Objective 1: increase en-route throughput that can be handled during typical

heavy traffic time-periods.

• Objective 2: increase the number of aircraft that can be simultaneously

accommodated in en-route airspace.

2.4 STEP 3: QUANTIFY OBJECTIVES

2.4.1 The principle of “reliance on facts and data for decision-making” implies that objectives should be specific,
measurable, achievable, relevant and time-bound (SMART). The purpose of Step 3 in the process is to ensure that
these aspects are properly addressed.
Part I — Global Performance
Chapter 2. The performance-based approach step-by-step I-2-9

2.4.2 Step 3.1: Define how progress in achieving performance objectives

will be measured and which data are required

2.4.2.1 This section explains briefly that as part of the performance-based approach there is a need for defining:

— indicators;

— the metrics underpinning those indicators; and

— common definitions for data aggregation and event classification.

2.4.2.2 It also addresses the measurement granularity and the need for harmonization.

2.4.2.3 Current/past performance, expected future performance (estimated as part of forecasting and performance
modelling), as well as actual progress in achieving performance objectives is quantitatively expressed by means of
indicators (sometimes called Key Performance Indicators, or KPIs).

2.4.2.4 Indicators are not often directly measured. They are calculated from supporting metrics according to clearly
defined formulas, e.g. cost-per-flight-indicator = Sum(cost)/Sum(flights). Performance measurement is therefore done
through the collection of data for the supporting metrics (e.g. leads to a requirement for cost data collection and flight
data collection).

2.4.2.5 Indicators need to be defined carefully:

— To be relevant, they need to correctly express the intention of the performance objective. Since
indicators support objectives, they should not be defined without having a specific performance
objective in mind.

— They need to be expressed in terms of supporting metrics for which there is adequate data availability.

2.4.2.6 When there is a problem with data availability, there are two possibilities:

— set up the appropriate data reporting flows and/or modelling activities, to ensure all supporting metrics
are populated with data as required to calculate the indicator(s) associated with the objective; or

— if this is not possible, aim for a different kind of performance improvement, by choosing a different
performance objective, as constrained by data availability.

2.4.2.7 Note that the need for an indicator lasts only as long as the corresponding performance objective exists.

2.4.2.8 Alternatively, the need for supporting metrics (such as the number of flights) lasts much longer because
metrics are seldom indicator-specific, i.e. they are typically used to calculate a variety of indicators. When deciding which
data to collect, a sufficiently broad spectrum of supporting metrics will have to be considered.

2.4.2.9 Data collection should take place at the most detailed level of granularity that can be afforded because the
availability of detailed data greatly increases the effectiveness of the performance-based approach.

2.4.2.10 Common aggregation hierarchies and classification schemes (taxonomies) are then used to condense the
detailed supporting metrics into clearly scoped summary indicators.
I-2-10 Manual on Global Performance of the Air Navigation System

2.4.2.11 To conclude: in a collaborative environment in which many stakeholders contribute to the achievement of
objectives and/or have performance reporting obligations, it is important to harmonize not only the definition of indicators
and supporting metrics, but also the scope definitions, e.g. aggregation hierarchies and classification schemes
(taxonomies).

Example — Step 3.1

The service provider has chosen to use the following indicators:

— To measure Objective 1:

• Throughput demand as a number of IFR movements per hour. Defined as the

number of IFR flights requiring entry to the airspace volume during a given one
hour period.

• Throughput capacity as a number of IFR movements per hour. Defined as the

number of IFR flights which may enter the airspace without causing excessive
ATC staff workload, and therefore without impeding safety.

• Number of sectors. Defined as the number of sectors that are open in an

airspace volume (FIR or part thereof) during typical busy hours.

— To measure Objective 2:

• PIAC demand as a peak instantaneous aircraft count (PIAC). Defined as the

number of IFR flights simultaneously present in the airspace at a given moment
in time.

• PIAC capacity as a peak instantaneous aircraft count (PIAC). Defined as the

number of IFR flights which can be simultaneously present in the airspace
volume at a given moment in time, without causing system overload (e.g.
surveillance, flight data processing).

To calculate the above demand indicators, data will have to be collected at the level of
individual flights. The following supporting metrics are needed:

• Entry time. Defined as the date and time at which a flight is entering the airspace
volume.

• Exit time. Defined as the date and time at which a flight is leaving the airspace
volume.

For each flight, entry and exit times will have to be calculated and/or recorded for every
airspace volume for which calculation of the indicators is desired.

Calculation of indicators:

• Throughput demand: the number of flights with an entry time within the one-hour
period for which the indicator is calculated.
Part I — Global Performance
Chapter 2. The performance-based approach step-by-step I-2-11

• PIAC demand at time T1: PIAC demand at time T0 plus the number of flights with
an entry time within the T0-T1 period minus the number of flights with an exit time
within the T0-T1 period.

Aggregation hierarchy:

• The FIR is subdivided into two sector groups (north and south), which in turn consist
of ATC sectors.

• Indicators are calculated for sectors, sector groups and the FIR as a whole.

2.4.3 Step 3.2: Define the desired speed of progress in terms of baseline and target performance

2.4.3.1 The above-mentioned performance indicators are the quantifiers for how well performance objectives have
been achieved.

2.4.3.2 Performance targets are closely associated with performance indicators: they represent the values of
performance indicators that need to be reached or exceeded to consider a performance objective as being fully achieved.
Note that performance targets can be set as a function of time (e.g. to plan yearly improvement); they can also vary by
geographic area, stakeholder, etc. In addition, targets can be set at different levels: local, regional or global.

2.4.3.3 Once the scope of a target has been agreed, it becomes clear where and at which level performance
management will need to be applied, between which stakeholders the achievement of the objective needs to be
coordinated, and who will need to be involved in trade-off decisions. The term target is used with different meanings:

— current versus future: when the aim of the objective is to improve current performance over time, the
term “target” refers to a future desired or required performance level;

— real versus design specifications: when the aim of the objective is to manage real performance so as
to stay within pre-defined limits, the term “target” refers to design specifications.

2.4.3.4 Performance targets may be set with different intentions, for example:

— as a strategic design target, to support transition planning;

— as a recommendation or incentive to promote the need for action and accelerate improvements;

— as a legal requirement;

— as a performance level which needs to be achieved to enable other performance improvements;

— as a mandatory performance requirement which is necessary for safety reasons;

— to gain access to certain airspace or receive certain levels of service, etc.

The above examples are included to illustrate that targets can also be set to provide guidance and should
not just be seen as an instrument for enforcement.
I-2-12 Manual on Global Performance of the Air Navigation System

2.4.3.5 To understand how challenging it is to reach a target, one should know the baseline performance. The
difference between the baseline and the target is called the performance gap. In a “current versus future” application,
the size of the gap is often expressed as a percentage of the baseline performance (e.g. 10 per cent improvement
needed to reach the target). For “real versus design specifications” applications, targets are usually expressed as
absolute values, without reference to a baseline.

2.4.3.6 The determination of the baseline performance (calculation of baseline indicator values) is done during the
previous iteration of the process. It is one of the results of Step 6: Assess achievement of objectives. This is part of the
performance review.

2.4.3.7 The time available to achieve performance objectives is always limited. Therefore, targets should always
be time-bound.

2.4.3.8 The target and the time available to reach the target determine the required speed of progress for the
performance objective. Care should be taken to set targets so that the required speed of progress is realistic. Target
setting is used as a tool by managers, policymakers, regulatory bodies and standardization organizations. Targets can
have far reaching consequences depending on how challenging they are and how serious they are taken.

2.4.3.9 In the air navigation system, appropriate decision-making/policymaking processes need to be in place to
collaboratively agree on performance objectives, performance indicators and the values of performance targets at the
local, regional and, where required, global levels.

Example — Step 3.2

Baseline: today’s performance has been determined to be:

— Northern part of the FIR:

• Number of sectors: 5
• The sectors have a capacity of 15 movements/hour, with a typical busy hour
demand of 10 movements/hour
• PIAC capacity of the sector group is 40 aircraft, with a typical busy hour PIAC
demand of 25 aircraft

— Southern part of the FIR:

• Number of sectors: 4
• Sector capacity is 30 movements/hour, with a typical busy hour demand of 25
movements/hour
• PIAC capacity of the sector group is 100 aircraft, with a typical busy hour PIAC
demand of 80 aircraft

The traffic growth forecast for the next 15 years is:

— Northern part of the FIR: traffic density is expected to triple (x3)

— Southern part of the FIR: traffic density is expected to double (x2)

Target setting: it is decided to adopt the future typical busy hour demand as the capacity
target. This results in the following capacity targets (absolute values: baseline demand
multiplied by growth factor):
Part I — Global Performance
Chapter 2. The performance-based approach step-by-step I-2-13

— Northern part of the FIR:

• Sector capacity: 10 × 3 = 30 movements/hour
• PIAC sector group capacity: 25 × 3 = 75 aircraft

— Southern part of the FIR:

• Sector capacity: 25 × 2 = 50 movements/hour
• PIAC sector group capacity: 80 × 2 = 160 aircraft

The capacity gap (for the “do nothing” scenario) is the difference between the target and the
baseline performance:

— Northern part of the FIR:

• Sector capacity gap: 30 – 15 = 15 movements/hour
• PIAC sector group capacity gap: 75 – 40 = 35 aircraft

In terms of relative values (gap divided by baseline), the required capacity increase is:

— Northern part of the FIR:

• Sector capacity: 15/15 = 100% increase
• PIAC capacity: 30/40 = 75% increase

— Southern part of the FIR:

• Sector capacity: 20/30 = 66% increase
• PIAC capacity: 60/100 = 60% increase

2.5 STEP 4: SELECT SOLUTIONS TO EXPLOIT

OPPORTUNITIES AND RESOLVE ISSUES

2.5.1 The purpose of this step is to apply the principle of “informed decision-making, driven by the
desired/required results”.

2.5.2 It combines the knowledge of baseline performance, opportunities and issues with the performance
objectives and targets, in order to make decisions in terms of priorities, trade-offs, selection of solutions and resource
allocation. The aim is to optimize the decisions to maximize the achievement of the desired/required (performance)
results.

2.5.3 Step 4.1: Select the decisive factors to reach the target performance

2.5.3.1 This part of the process is sometimes called performance gap analysis.

2.5.3.2 As a result of Step 2, a qualitative inventory of present and future opportunities and issues that may require
performance management attention is already available (see 2.3.2).
I-2-14 Manual on Global Performance of the Air Navigation System

2.5.3.3 This list now needs to be analysed in a performance oriented way, to assess (i.e. quantify) the impact of
drivers, constraints, impediments, etc., on the objective(s) under consideration. In other words: to what extent, when and
under which conditions do these contribute to (or prevent) the required performance improvements.

2.5.3.4 When analysing blocking factors for runway capacity improvement for example, it may turn out that for a
given airport (example: single runway without parallel taxiway), the dominant blocking factor is runway occupancy time,
rather than wake vortex separation minima. Knowing this, it is clear that solutions that reduce runway occupancy time
will contribute to runway capacity enhancement, whereas solutions which reduce wake vortex separation minima will not
contribute to the achievement of the objective in this particular example. Likewise, at some airports the dominant
constraining factor may be runway capacity, but elsewhere it may be gate and apron capacity.

2.5.3.5 In order to make progress in reaching an objective, the dominant factors first need to be undertaken. So
the outcome of this activity is a selection and prioritization of opportunities and issues. This can be seen as the
development of a “performance strategy” for the achievement of a given objective: working “backwards” from
expectation related objectives, it cascades performance requirements down to a selection of subordinate, enabling
objectives and targets (e.g. to improve airport capacity, we first need to improve runway capacity, for which we first need
to reduce runway occupancy time).

2.5.3.6 This part of the process:

— eliminates/defers issues that do not immediately or significantly affect the achievement of objective(s);

— helps to maximize effectiveness if performance improvements have to be realised with limited

resources (e.g. budget, manpower);

— creates a “traceability chain”, and/or a “performance case” which explains what will be improved and
how much, prior to the selection of solutions; and

— progresses the decision-making to the point where it is appropriate to start thinking in terms of
available solutions (options).

Example — Step 4.1

Analysis of drivers: after collaborative checking of the forecasting assumptions and models,
there is a shared belief between the service provider of FIR YYYY, the neighbouring ANSPs
and the regional planners that traffic will indeed grow as forecasted.

At present, in both the northern and southern parts of the FIR, there is some spare capacity.
After analysis of the data, it is decided that:

• no capacity increases are required for the next five years.

Then:

• procedural control will be the dominant blocking factor in the northern part of the FIR.
This will have to be addressed as a matter of priority;

• controller workload may become an issue; and

Part I — Global Performance
Chapter 2. The performance-based approach step-by-step I-2-15

• degradation of surveillance performance (SSR capacity limits exceeded) will be the

next blocking factor affecting capacity in the southern part of the FIR.

The following issues are classified as lower priority, because their limiting effect on capacity
enhancements lies much further into the future:

• frequency shortage; and

• staff shortage due to air traffic controller retirements.

Assessment of the investment risk associated with a possible drop in the volume of overflying
traffic:

• probability assessment: at present, analyses of future demand, route structures and

airspace availability in neighbouring countries seem to indicate that the probability of
traffic flow re-routing is rather low for the next ten years; and

• severity assessment: the effect of a drop in the volume of overflying traffic could be
lessened by finding more cost-effective ways of increasing capacity.

Opportunity analysis: after analysing how other regions have implemented ADS-B, the service
provider is confident that ADS-B is a viable alternative to other surveillance technologies.

2.5.4 Step 4.2: Identify solutions to exploit opportunities and mitigate the effects
of the selected drivers and blocking factors

2.5.4.1 At this stage, decision-makers need to know their options for mitigating pre-identified issues and therefore
to exploit available opportunities. This part of the process is about establishing the list of options, i.e. defining the
“solution space” which is at the disposal of decision-makers for optimizing the achievement of performance objectives.

2.5.4.2 In the above example, for the objective “reducing runway occupancy time”, the list of possible
solutions/options may include:

— building extra taxiways to avoid the need for backtracking or to eliminate the need for runway
crossings;

— building high speed runway exits to give more options for vacating the runway, thereby reducing
runway occupancy time; and

— equipping aircraft with “brake-to-vacate” 2 technology, which enables pilots to select a runway exit
while the aircraft is making its landing approach. This increases the predictability of runway occupancy
time, which in turn allows reducing the separation minima on final approach. The latter will lead to
increased capacity if runway occupancy time is the dominating blocking factor.

2. Brake-to-vacate uses the auto-flight, flight controls, and auto-brake systems to regulate deceleration after touchdown. This allows
the aircraft to reach a specified exit at the correct speed under optimum conditions.
I-2-16 Manual on Global Performance of the Air Navigation System

2.5.4.3 The list of solutions relates to the list of issues. In this example, each solution addresses a different issue
(which may or may not be present at a given airport), but they all contribute to the same performance objective.

2.5.4.4 When the task is to improve the effectiveness of the day-to-day economic and operational management,
the list of options will most likely be populated with off-the-shelf solutions and best practices, i.e. solutions which are
readily available.

2.5.4.5 When working with longer time horizons (during transition planning), a number of the options or operational
improvements may still be in their research, development and trials phases, meaning that decision-makers will have to
work with a “living” list of options, which are still surrounded by a certain degree of uncertainty.

2.5.4.6 In any case, decision-makers need to gain a good understanding of the strategic fit, the benefits, cost and
feasibility of each option for operational improvement. Therefore, the description of the operational improvements in the
list needs to show that they have been developed from different complementary perspectives. To produce this list of
options, the performance-based approach should be applied at each level.

— they must fit within high-level strategy and policy orientations;

— they must be on the transition path towards the operational concept/concept of operation;

— they must take into account the architecture in which the technical system enablers (e.g. flight data
processing systems, CNS systems) will need to fit;

— they need to be associated with a baseline (“before” state) from which they can be deployed and will
start delivering benefits;

— they depend on the feasibility/timing of developing and deploying the required enablers. Typically this
will provide information on earliest availability;

— a safety and human factors assessment is required to have sufficient confidence that the operational
improvement is feasible from a human factors and safety perspective, and that a list of issues that first
need to be addressed during the development life cycle is raised;

— the expected performance enhancement contribution needs to be known. More specifically, issues
that need to be resolved (or opportunities exploited) under which circumstances should be explicitly
specified, so as to quantify the resulting performance improvement. The performance improvement is
to be shown first in terms of affected supporting metrics, which can then be expressed as performance
indicator changes, which in turn are linked to performance objectives, focus areas, expectations and
key performance areas. The preferred way of assessing expected performance is through modelling
(see Appendix D). If this is for some reason not possible, a pragmatic way forward is through expert
judgement with consensus building; and

— the performance assessment should also have identified all relevant side effects. Information on side
effects is needed for trade-off decision-making. This covers disadvantages such as:

• increased cost;

• increased resource consumption; and

• unintended performance reduction in other areas.

Part I — Global Performance
Chapter 2. The performance-based approach step-by-step I-2-17

2.5.4.7 The Global Air Navigation Plan (Doc 9750) is a useful source for developing the list of options/operational
improvements. Use can also be made of work already developed in other regions.

2.5.4.8 In those cases where a list of options/operational improvements was already developed during a previous
planning cycle, the task consists of updating the list to take the latest developments into account. In the case of
transition planning — where the process may be executed only once every five years — “updating the list” means
significantly “refining the list”.

Example — Step 4.2

The issues which were identified as having to be mitigated as a matter of priority are:

• capacity limits associated with procedural control in the northern part of the FIR;

• controller workload in the southern part of the FIR; and

• the SSR capacity limits in the southern part of the FIR.

Candidate solutions (in terms of operational improvements and enablers) have been
identified by the service provider as:

• moving from procedural control to radar control (required enabler: surveillance

coverage);

• sector splitting (horizontal and vertical);

• replacement of SSR technology by a system with higher PIAC capability; and

• choice of surveillance technologies:

• SSR;
• Mode-S radar; and
• ADS-B.

2.5.5 Step 4.3: Select a sufficient set of solutions

2.5.5.1 This is the part of the process where decisions are made based on which solution(s) to implement. The
following information is available to support decision-making:

— definition of system/expectation scope and context;

— the required results in terms of performance objectives and targets (in some cases, for a certain date,
in other cases, as an evolution through time, specifying a “required speed of progress”, e.g. four per
cent improvement per year);

— prioritized issues and opportunities, and their impact on performance; and

— an overview of candidate solutions and their capability to resolve issues and exploit opportunities, in
terms of:
I-2-18 Manual on Global Performance of the Air Navigation System

• list of operational improvements; and

• associated enablers (services and procedures, human resources, systems and technology, and
regulation and standardization).

2.5.5.2 It is within this framework that decisions have to be taken. The nature of the decision and the method best
applied depend on the situation, as explained below.

2.5.5.3 Assume an example in which one has identified a number of candidate solutions to increase runway
capacity. The requirement is to exceed a certain capacity target while staying below a certain cost target. This is
illustrated in Figure I-2-2.

2.5.5.4 For each candidate solution, the expected runway capacity and associated cost have been assessed
during the previous step. With this information, the various candidate solutions can be positioned in the capacity/cost
diagram of Figure I-2-2.

Cost

Target
C performance F

E
D
A
B
Current
performance

Capacity
Figure I-2-2. Expected performance of candidate solutions (example)

2.5.5.5 What is the decision to be taken and which method is to be used? Depending on which subset of the
solutions is available, the answer will vary. Your project may be faced with any combination of the situations described in
Table I-2-1.
Part I — Global Performance
Chapter 2. The performance-based approach step-by-step I-2-19

Table I-2-1. Selection of solutions (example)

Case Solutions Decision

1 Only A, B, C, and F are available. None of the proposed solutions is satisfactory. Decide whether to:
None of these solutions meets — continue searching for solutions which address the selected issues
both targets. and opportunities and meet all targets;
— focus on different issues and opportunities (this assumes that the
objectives can be achieved using a different method or by setting
different priorities);
— relax the target(s) (which makes it easier to achieve the
objective(s), but the impact of this would need to be assessed); or
— abandon the performance objective(s) altogether (this also requires
an impact assessment).

2 D and E are under development Decide what can be done to advance the availability date of D and E. In
and not yet available when many cases, availability constraints are imposed by enabler deployment
needed. and this would imply a need to accelerate the deployment of certain
enablers.

3 Only D is available and/or meets There are no alternatives to choose from. Accept solution D.
all targets.

4 Only D and E are available. Both candidate solutions exceed the targets. Select the “best buy”. Is it
They are mutually exclusive. worth choosing E instead of D? A commonly used method for making
the decision is multi-criteria decision analysis (MCDA).

5 Only D and E are available. D and E may be complementary in delivering performance benefits
They can be applied (improving performance in different places, at different times of the day,
simultaneously. under different conditions). Decide whether to combine them into one
“implementation package”.

6 Only D and E are available. It may make sense to first use solution D, and after a while (some
years) to replace D by E to achieve better performance. Decide how to
include D and E in the “road map” or “deployment sequence”.

7 D is available, but its capacity will This is the case where D and G interfere with each other from a
suffer if solution G (which performance perspective. Trade-off considerations will need to be part
improves flexibility, not shown in of the decision process. The aim is to take a balanced decision.
Figure I-2-2) is also applied. See Appendix A for more information about trade-offs.

An overview of how the seven cases above relate to each other is given in Figure I-2-3.
I-2-20 Manual on Global Performance of the Air Navigation System

Outcome of
candidate
solution
evaluation

Zero One Multiple

options option options

Solutions Mutually Solutions

No suitable
not yet exclusive not mutually
solution
mature solutions exclusive

Options
Options
have mixed
reinforce
performance
performance
impact

Solutions for Solutions for

simultaneous sequential
deployment deployment

1 2 3 4 5 6 7
Try to find other Try to advance Accept Select Define Define Select
solutions or modify availability date this solution “Best Buy” “Implementation deployment balanced
performance of solutions (use multi- Packages” sequence combination of
objectives / targets criteria decision and define solutions using
analysis) deployment trade-off
criteria considerations

Figure I-2-3. Selection of solutions

Part I — Global Performance
Chapter 2. The performance-based approach step-by-step I-2-21

2.5.5.6 Depending on the nature of the project, the output of this process step is either a single preferred solution,
or a road map of selected solutions (combined into implementation packages and sequenced into a deployment
sequence), accompanied by an initial performance case that describes those issues which are resolved and
opportunities exploited, together with the expected costs and benefits in terms of performance improvement towards the
specified targets.

Example — Step 4.3

Different solutions are chosen for the northern and southern parts of the FIR. This is due to
the fact that the targets as well as the baseline system are different. The description below
explains the rationale behind the decisions.

Operational improvements selected for the northern part of the FIR:

• Since the target for sector capacity is 30 movements/hour, which is equal to the
baseline capacity of the sectors in the southern part of the FIR, sector splitting is not
necessary. The number of sectors will remain unchanged, but their capacity will be
increased by 100 per cent, and this will be achieved by moving from “procedural”
control to “radar” control. The choice of supporting surveillance technology is
described in the section “enablers”.

Operational improvements selected for the southern part of the FIR:

• In this example there are no options available that allow sector capacity to be
increased beyond today’s 30 movements/hour; however, we need a 66 per cent
increase. This will be achieved by reorganizing the sectorization in the southern part
of the FIR to increase the number of sectors from four to seven, with a capacity of 30
movements/hour each. Independent of that, the problem of degraded surveillance
performance under conditions of higher traffic density will still have to be resolved.

Selection of enablers:

• Sector splitting requires the installation of three more controller workstations.

Computer system/software changes are not necessary: the flight data processing and
tracking systems have sufficient spare capacity.

• There is no need for recruitment of additional controllers: it is determined that the

extra human resource requirement can be accommodated through more efficient
rostering.

• Surveillance coverage is required in the northern part of the FIR. Options: SSR,
Mode-S radar, and ADS-B.

• In the southern part of the FIR, SSR will become inadequate and will have to be
replaced. Options: Mode-S radar and ADS-B.
I-2-22 Manual on Global Performance of the Air Navigation System

• The service provider has determined that having different surveillance systems in the
north and the south is undesirable for various reasons (cost-effectiveness, airspace
user equipage, etc.). This rules out SSR as an option for the north.

• Therefore, the shortlist of options for the entire FIR is Mode-S radar and ADS-B. After
more detailed evaluation, the service provider decides to select ADS-B. Under the
given circumstances, it is a more cost-effective solution with sufficient potential for
long-term capacity.

2.6 STEP 5: IMPLEMENT SOLUTIONS

2.6.1 Step 5 is the execution phase of the performance management process. This is where the changes and
improvements that were decided upon during the previous step are organized into detailed plans, implemented, and
begin delivering benefits.

2.6.2 Depending on the nature and magnitude of the change, this could mean:

— In the case of small-scale changes or day-to-day management:

• assigning management responsibility for the implementation to an individual;

• assigning responsibility and accountability for reaching a performance target to an individual or

organization;

— In the case of major or multi-year changes:

• refining the road map of selected solutions into a detailed implementation plan, followed by the
launching of implementation projects.

• Ensure that each individual implementation project is operated in accordance with the
performance-based approach. This means launching and executing the performance
management process at the level of individual projects. Each project derives its scope, context
and expectations (see Step 1 of the process) from the overall implementation plan.

Example — Step 5

At this stage of the process, a plan is developed for the three main projects:

• The re-sectorization project

• The ADS-B implementation project (covering both the airspace user and ground
segments)

• The SSR decommissioning project

Part I — Global Performance
Chapter 2. The performance-based approach step-by-step I-2-23

The required completion date for the first two projects is more than five years into the future
(recall that no capacity shortage was expected during the first five years), but because
acquisition, installation and pre-operational testing will require several years, the projects need
to start soon.

SSR actual decommissioning will be planned in detail and take place a number of years later;
there will be a transition period during which SSR and ADS-B will co-exist in the southern part
of the FIR.

2.7 STEP 6: ASSESS ACHIEVEMENT OF OBJECTIVES

2.7.1 The purpose of Step 6 is to continuously keep track of performance and monitor whether performance
gaps are being closed as planned and expected.

2.7.2 First and foremost, this implies data collection to populate the supporting metrics with the data needed to
calculate the performance indicators. The indicators are then compared with the targets defined during Step 3 to draw
conclusions on the speed of progress in achieving the objectives.

2.7.3 This step includes monitoring progress of the implementation projects, particularly in those cases where
the implementation of solutions takes several years (as in our example), as well as checking periodically whether all
assumptions are still valid and the planned performance of the solutions is still meeting the (perhaps changed)
requirements.

2.7.4 With regard to the review of actually achieved performance, the output of Step 6 is simply an updated list
of performance gaps and their causes. In practice, the scope of the activity is often interpreted as being much wider and
includes recommendations to mitigate the gaps. This is then called performance monitoring and review, which in
addition to Step 6 includes Steps 1, 2 and 3 of the performance management process. This is illustrated in Figure I-2-4.

2.7.5 For the purpose of organizing performance monitoring and review, the task can be broken down into five
separate activities:

— data collection;

— data publication;

— data analysis;

— formulation of conclusions; and

— formulation of recommendations.

2.7.6 Data collection

2.7.6.1 There are essentially two major categories of data feeds that performance monitoring and review will deal
with:
I-2-24 Manual on Global Performance of the Air Navigation System

1.
6. Define/review
Assess scope, context and
achievement general ambitions
of objectives and expectations

5. 2.
Implement Identify
solutions opportunities, issues
and set (new) objectives

4. 3.
Select solutions to Quantify
exploit opportunities objectives
and resolve issues

Figure I-2-4. Scope of performance monitoring and review

— data which are captured by automatic means and forwarded in electronic form with little or no human
intervention. This type of data feed is typical for high volume streaming measurement data and
automated database-to-database exchanges; and

— manual reporting of information (electronically or on paper). This requires human effort to collect,
interpret, analyse, structure and otherwise prepare the data for reporting. Typical for low-frequency,
complex information data feeds in which the performance monitoring organization receives processed
information (forms, reports) instead of raw data feeds.

2.7.6.2 To establish data feeds in each KPA, the following steps need to be undertaken:

— identify information needs;

— identify potential suppliers of data;

— ensure information disclosure by candidate data suppliers; and

Part I — Global Performance
Chapter 2. The performance-based approach step-by-step I-2-25

— manage the data feeds on an ongoing basis.

2.7.6.3 More information on each of these steps can be found in Appendix B (Data and Information Management).

2.7.7 Data access and publication

2.7.7.1 The performance review can begin once the required data (performance targets and current/anticipated
values for performance indicators) are available. The first activity in this process is data publication.

2.7.7.2 With proper ATM community participation in place, ATM performance will be evaluated by two different
groups:

— performance specialists (e.g. analysts from designated ATM performance review organizations); and

— people with a generally high level of interest in ATM performance.

2.7.7.3 Each group has its own specific need for access to ATM performance data, which should be satisfied by
appropriate data access and publication means.

2.7.7.4 People with a general interest in ATM performance will wish to see executive level, quality-controlled data
and draw their own conclusions, at which point, the need arises to make certain performance data publicly available in
the interest of transparency. A capability is therefore required which enables them to monitor the current situation
against the performance targets, and to provide them with the general trends, the “big picture” and their own
performance in comparison with others. This need can be satisfied by publishing high-level performance indicator
indices. These indices are periodically updated and generally allow limited or no interactivity by the user.

2.7.7.5 In addition, analysts from designated ATM performance review organizations are tasked with gaining an in-
depth understanding of ATM performance and finding causes and effects. Their work is an integral part of the
performance management process described earlier. Their data needs can be satisfied by publishing selected data in
performance assessment databases which are designed to suit the analysts’ needs. These databases should allow high
levels of interactivity (querying and analysis).

2.7.8 Data analysis

2.7.8.1 At the data analysis stage, the performance review organization should ensure that the data are already
quality-checked. Rather than struggling with data quality issues, analysts should be able to focus on their main task:
performance review.

2.7.8.2 Analysts need to examine the reasons for (good/poor) performance, and explain these to decision-makers,
while gaining a better insight into past, current and future ATM performances.

2.7.8.3 To that effect, analysts will compare performance indicators against performance targets, identify
performance evolution trends, analyse historical evolution of performance, and find relationships (correlations) between
performance indicators, supporting metrics, etc. They will look at the “big picture” (annual totals and averages,
performance indicators summarized during the planning cycle) and break down the data into very detailed levels to find
the causes of performance gaps and the reasons for trade-offs. Analysts will also make use of various modelling
techniques to increase their understanding of system performance (see Appendix D, 3.2.3).

2.7.8.4 As a side effect of data analysis, analysts will be able to propose performance objectives, define new
performance indicators and identify data needs.
I-2-26 Manual on Global Performance of the Air Navigation System

2.7.9 Formulation of conclusions

2.7.9.1 After completing the data analysis, analysts are expected to document the conclusions for each KPA.
Normally, these conclusions contain an assessment of the current and expected future performance for each
performance objective. Alternatively, a conclusion could be that the available data are insufficient for a meaningful
performance review.

2.7.9.2 Typically, the conclusions are published in a performance review report.

2.7.10 Formulation of recommendations

2.7.10.1 An integral part of the performance review process is the formulation of recommendations. These should
be derived from the conclusions and also be included in the performance review report.

2.7.10.2 Recommendations should focus on how to meet ATM community expectations through agreed upon
performance objectives, performance indicators and performance targets. When an evaluation indicates inconsistency
between ATM community expectations and performance objectives, performance indicators and performance targets,
recommendations may include:

— the need to set or change performance objectives;

— the need to (re-)define performance indicators; and

— the need to set or change performance targets.

2.7.10.3 Recommendations will typically fall into the following categories (non-exhaustive list):

— the need to improve performance data collection;

— suggested initiatives aimed at closing identified performance gaps;

— suggestions to accelerate or delay performance improvements based on anticipated evolution of traffic

demand and predicted performance indicator trends; and

— setting up task forces, defining action plans, etc., with a view to beginning the implementation process.

2.7.11 Positioning of performance review within the overall process

It is recommended that the performance monitoring and review activity is sufficiently integrated into the overall
performance planning process to ensure that the conclusions and recommendations serve as direct input for Step 4 of
the process, while simultaneously maintaining a degree of independence from the other parts of the process in order to
ensure a sufficient level of objectivity and impartiality.

2.8 REPEAT THE PROCESS

2.8.1 Performance management is a closed loop process

2.8.1.1 The performance management process is intended to be a closed-loop process. Step 6 has identified
deficiencies, i.e. cases where performance is not as expected, despite the implementation of changes designed to
achieve performance improvements. These deficiencies are to be acted upon by starting the next iteration of the process.
Part I — Global Performance
Chapter 2. The performance-based approach step-by-step I-2-27

2.8.1.2 One may be surprised to note that this implies revisiting Step 1: Define/review scope, context and general
ambitions/expectations. It is strongly recommended not to skip this because the performance management scope,
context and general expectations may be subject to continuous change.

2.8.2 How, when and how often to execute the process?

The periodicity of the process greatly depends on where in the air navigation system and lifecycle it is applied.
Depending on the nature of your project/activity you could be responsible for any of the following:

— Ensuring performance of concepts and systems:

• during concept validation; and

• during and/or after implementation (e.g. pre-operational tests, acceptance tests).

— Application of regulatory methods to organizations, people and systems:

• legal requirements, rules and regulations;

• certification and licensing; and

• inspection and oversight.

— Annual performance review:

• performance is evaluated in annual cycles, in a reactive way;

• followed by actions in subsequent years to correct performance deficiencies.

— Proactive, collaborative performance planning:

• medium-term planning processes (annual cycles);

• strategic planning/seasonal scheduling (seasonal cycles); and

• pre-tactical planning (e.g. daily cycles).

_____________________
 Chapter 3

GETTING STARTED

3.1 INTRODUCTION

3.1.1 Chapter 1, 1.1.2 already indicated that this manual contains guidance material intended to:

— provide “getting started” assistance to those ATM community members who are (relatively)
inexperienced in applying the performance-based approach; and

— assist the experienced ATM community members in converging towards a globally harmonized and
agreed upon approach.

3.1.2 This chapter is primarily for the benefit of the first group of readers. ATM community members, who
consider starting up their first “performance management project” and are not sure where to start, will find in the next
sections some practical tips of an organizational nature:

— develop a good understanding of the performance-based approach;

— assess currently used approaches;

— define implementation priorities;

— start with a limited scope; and

— establish commitment and collaboration.

3.1.3 After the initial start with limited scope, it is expected that ATM community members would progressively
increase the scope to systematically cope with all performance issues of importance to them.

3.1.4 Within the context of “getting started”, a review of the subjects presented in Chapter 1, 1.4 is
recommended:

— commitment;

— agreement on goals;

— organization;

— human resources and knowledge/expertise;

— data collection, processing, storage and reporting;

— collaboration and coordination; and

— cost implications.

I-3-1
I-3-2 Manual on Global Performance of the Air Navigation System

3.2 DEVELOP A GOOD UNDERSTANDING OF THE

PERFORMANCE-BASED APPROACH

3.2.1 When you start your first “performance management project”, there is likely going to be a diverging
understanding amongst the participants of what the “performance-based approach” exactly means, in particular, if
organizations and people are involved which had little prior experience with the subject.

3.2.2 Alternatively, there may be prior experience, in which case the challenge is to adapt and improve the
existing procedures.

3.2.3 No matter which of the above situations applies to your organization, there is a need to develop a sufficient
level of know-how. While Chapter 2 provides a general overview with a worked-out example, there is also a set of
appendices which provide more specific or technical guidance on selected subjects.

3.2.4 A good way of learning and adopting the performance-based approach is by “hands-on” experience.
Choose a pilot project with a limited scope (i.e. addressing a single KPA, a specific objective, and with limited data
requirements) and try to apply the different steps of the performance management process. The experience will allow
you to attempt a more challenging subject next time.

3.2.5 Another way of introducing performance know-how in your organization is by first establishing a
performance review function. After this group is recognized for its expertise and has laid the foundation in terms of data
collection and analysis capabilities, it will be easier to introduce the performance-based approach at policymaking and
decision-making levels.

3.2.6 For a more sizeable project, one in which a large number of people will be involved and which needs to be
well planned, the following approach is suggested:

— Ensure that the project has a core team. The responsibility of the core team will be to influence, coach
and assist the wider group of participants, ranging from the political/management level down to the
suppliers of data. To be able to do that, the members of the core team should develop a thorough
understanding of the performance-based approach as described in this manual. This will take some
time: training will be required, tools may need to be evaluated and adopted, and the core team needs
to get the opportunity for hands-on experience (e.g. small pilot project).

— Within the core team, establish a technical support group which has an in-depth understanding of ATM
performance and the surrounding issues of data collection, information management, performance
modelling, performance review, forecasting, etc. This group should be comprised of systems analysts,
statisticians, database specialists, etc.

— Develop and execute a plan to establish acceptance and basic understanding of the PBA among the
wider group of participants. This should, at a minimum, cover the subjects covered by Chapters 1
and 2 of this manual. The “wider group of participants” includes those decision-makers who will be
involved in defining objectives, setting targets, selecting solutions, etc.

3.3 ASSESS CURRENTLY USED APPROACH

3.3.1 In many cases, organizations will already have applied a type of performance-based approach, have
access to certain data, and have already used objectives and targets to trigger or guide changes.
Part I — Global Performance
Chapter 3. Getting started I-3-3

3.3.2 Prior to launching any performance management project, it is suggested to baseline your current
performance management maturity. Make an inventory of what you already have, identify what is missing, and identify
where there is a need to align your current approach with this manual.

3.3.3 For example, you may notice that you have been using a performance-based approach for safety, capacity
and cost-effectiveness, but not as yet for security, environmental aspects, etc. You may have good demand and
capacity data, but may, for example, be weak in collecting safety performance metrics. You may have been using
targets, but perhaps have less developed performance assessment capabilities, etc.

3.3.4 You may wish to use Chapter 1, 1.3.2 (Applicability) and the bullet list in Chapter 2, 2.8.2 (How, when and
how often to execute the process?) as checklists to scope the assessment exercise.

3.3.5 The results of this assessment form the basis for deciding what to add and/or change, and for defining
implementation priorities.

3.4 DEFINE IMPLEMENTATION PRIORITIES

3.4.1 Implementing the performance-based approach in one full scope is usually not an option. A gradual
introduction, with step-by-step extension of the scope offers a much higher chance of success.

3.4.2 By prioritizing the results of your baseline assessment, you will be able to develop a phased
implementation plan for the performance-based approach.

3.4.3 There may be (external) pressure to put the most important or most urgent items at the top of your project
list. However, when your organization has little prior experience, there is not yet widespread commitment to the
approach, or there is limited availability of performance data, in which case it will be wise to start the first project with a
limited, low-risk scope.

3.5 START WITH A LIMITED SCOPE

3.5.1 When limiting the scope of the first performance management project, it is recommended to focus on a
particular KPA, or even a particular focus area within a KPA. In other words, you should start with a limited number of
performance objectives and targets. It is also wise to choose a project which allows you to start with a limited
geographical scope, a limited set of ATM community members, etc.

3.5.2 No matter how limited the “getting started” scope is, ultimately the full performance management process,
i.e. all six steps should be implemented. Only then will the benefits of the approach materialize.

3.5.3 After a successful start with a limited scope, you will be able to use your “lessons learned” to gradually:

— extend the approach to other performance areas;

— use more and better data;

— get a better handle on trade-offs;

— become more effective in achieving desired results;

I-3-4 Manual on Global Performance of the Air Navigation System

— integrate your performance management process into a larger geographical scope (e.g. regional,
global); and

— undertake subjects which involve more stakeholders.

3.5.4 This manual does not prescribe the order in which the scope should be extended. That would entirely be
determined by your phased implementation plan, which is the result of your prioritization process.

3.5.5 As the scope increases, you will need to consider adding resources and adjusting roles and responsibilities
among the various participants (see Chapter 1, 1.4.4).

3.6 ESTABLISH COMMITMENT AND COLLABORATION

3.6.1 Developing know-how in a core team and defining a phased implementation plan with well-scoped projects
is only the first part of “getting started”.

3.6.2 Before really launching any performance management project, it is necessary to get properly organized. In
practice, this requires establishing internal commitment and collaboration with external stakeholders.

3.6.3 Commitment is a matter of:

— getting the decision-makers (e.g. senior management) behind the approach (see also
Chapter 1, 1.4.2);

— defining/adjusting roles and responsibilities (organizational aspects, see Chapter 1, 1.4.4);

— getting the necessary internal support for information/data collection, processing, storage and
reporting (see Chapter 1, 1.4.6); and

— securing the necessary budget (see Chapter 1, 1.4.8).

3.6.4 Collaboration: the project will not exist in isolation. Because “doing less together is better than doing more
apart” you should be prepared to collaborate with a number of external parties (see also Chapter 1, 1.4.7). Who these
parties are depends on the scope and context of the project. It is suggested that for each step of the performance
management process (see Chapter 2), you know who the relevant external stakeholders are and take the necessary
steps to establish collaboration and coordination.

_____________________
 Chapter 4

WHAT TO EXPECT FROM THE APPENDICES

IN THIS DOCUMENT

Note.— The previous chapters have given a high-level overview of the performance-based approach. For
a selected set of subjects, it was deemed useful to include more specific or technical guidance in this manual. Such
material has been placed in the appendices. This chapter provides a brief introduction.

4.1 APPENDIX A — PERFORMANCE FRAMEWORK

4.1.1 Appendix A, 2, introduces the five-level global ATM performance hierarchy, which is a construct that is
useful for illustrating (and reminding the reader) that the performance-based approach can be applied at different levels,
ranging from high-level socio-political issues to lower-level technology issues. The section also points out how the
different levels have a performance impact on each other, which should be carefully managed.

4.1.2 Appendix A, 3, explains the intended meaning and use of terms such as key performance area (KPA),
focus area, performance objective, performance indicator, supporting metrics, etc. These play a key role in the various
steps of the performance management process illustrated in Chapter 2, Figure I-2-1.

4.1.3 Appendix A, 4, stresses the need for a harmonized, structured view of the air navigation system to enable
the harmonization of performance indicators and supporting metrics (see also Appendix E). The development of a
structured description requires an understanding of the entities, activities and interactions involved in the air navigation
system. The section recognizes that only those entities, activities and interactions need to be considered which are
relevant for the harmonization of the required indicators and metrics.

4.1.4 Appendix A, 5, introduces the notion of process capability areas (PCA). Whereas the above mentioned
KPAs focus on a particular type of performance outcome (e.g. safety, capacity), PCAs focus on the quality, maturity and
completeness of the performance management processes that ATM community members have implemented. The
process-oriented perspective helps to diagnose weaknesses in the existing performance management processes, after
which specific initiatives can be taken (i.e. with appropriate objectives, indicators and targets) to improve these
processes and/or their deployment across the ATM community.

4.2 APPENDIX B — PRIORITIES, TRADE-OFFS AND RISKS

4.2.1 Appendix B, 2, recapitulates the Chapter 2 statements about the role of priorities, trade-offs and risks in the
various steps of the performance management process.

4.2.2 Appendix B, 3, elaborates on the use of priorities, not just for favouring one type of performance over
another (e.g. safety has highest priority in aviation), but also as a general instrument for distinguishing important from
less important issues during the application of the performance-based approach itself. It also stresses the point that
priorities can change over time and can vary in different parts of the world.

I-4-1
I-4-2 Manual on Global Performance of the Air Navigation System

4.2.3 In Appendix B, 4, the role of trade-offs is explained as an instrument for choosing the appropriate — that is,
the most balanced — solution within a given set of priorities, when different options are available, but each option has
different advantages and disadvantages in terms of performance impact.

4.2.4 Appendix B, 5, explains that risk management plays a role in performance management when dealing with
rare events or when there is uncertainty associated with influencing factors — while recognizing that the occurrence of
such events can have a severe impact on meeting the performance expectations. Risk management applies in all KPAs,
but as a technique it is mainly used in the safety and security areas.

4.3 APPENDIX C — DATA AND INFORMATION MANAGEMENT

4.3.1 This appendix contains practical guidance on:

— how to set up the data acquisition process needed for performance monitoring;

— how to aggregate performance data and exchange it between planning groups;

— how groups can best manage their information base in which performance data is stored; and

— how to organize performance evaluations.

Other subjects which are touched upon include global harmonization of such work, including definitions, standards for
reporting requirements, and information disclosure.

4.3.2 Appendix C, 2, introduces the reader to the fact that performance monitoring is a complex task which
should not be underestimated.

4.3.3 Guidance on how to establish data feeds and information disclosure is given in Appendix C, 3.

4.3.4 Appendix C, 4, explains that “data warehousing” techniques are the recommended approach for organizing
performance data storage, quality management and exchange. This subject is elaborated on in technical detail for two
reasons: 1) because experience has shown that relatively few people in the ATM community are sufficiently familiar with
this subject and its associated terminology, and 2) that data and information management is the most expensive and
labour intensive part of the whole performance-based approach. Therefore “doing things right the first time” in this area
is a major contributor for the cost effectiveness of the approach.

4.3.5 Appendix C, 5, deals with the actual use of performance data once it is available and quality checked. This
use is generally called “performance evaluation” or “performance review”. It consists of four separate activities which are
described as: data access and publication, data analysis, formulation of conclusions, and formulation of
recommendations.

4.4 APPENDIX D — PERFORMANCE ANALYSIS AND MODELLING

4.4.1 Appendix D, 2, reiterates the role that performance analysis and modelling play in the performance-based
approach.

4.4.2 Appendix D, 3, provides some practical guidance for performance modelling.

Part I — Global Performance
Chapter 4. What to expect from the Appendices in this document I-4-3

4.4.3 Within the context of the performance-based approach, particular interest should be shown for those
models and modelling techniques which support an improved understanding of ATM performance, and support the
choice of ATM changes aimed at improving performance.

4.4.4 Because performance is measured in terms of metrics and indicators, it follows that the variables in
performance models should include all metrics from which the indicators are derived; also, the indicators should be
defined in the performance framework.

4.4.5 Appendix D, 3.2, presents a suggested checklist for developing and using a performance model. It also
explains the difference between influence models, analytical models and simulation models.

4.5 APPENDIX E — METRICS AND INDICATORS

4.5.1 This appendix was written with the intent to globally harmonize (a number of) ATM performance indicators.

4.5.2 Appendix E, 1, describes the results of a comparison of ATM performance indicators used in two
organizations. This analysis was done for the 11 KPAs. The result of this investigation yielded no identical indicators.
However, some commonality was identified in certain indicators. For these, the steps required to reach consensus on a
common set of indicators were identified. For those KPAs where no commonality was found, a few examples are
presented from yet a third source.

4.5.3 The appendix illustrates the challenges associated with the development of a standardized set of
performance metrics and indicators.

4.5.4 The reader is encouraged to make the connection between Appendix E and the subjects discussed in
Appendix A, 3 and Appendix C, 4.3.

_____________________
 Chapter 5

REFERENCES

1. Global Air Traffic Management Operational Concept (Doc 9854)

2. Manual on Air Traffic Management System Requirements (Doc 9882)

3. Manual on Air Navigation Services Economics (Doc 9161)

4. Global Air Navigation Plan (Doc 9750)

5. Safety Management Manual (SMM) (Doc 9859)

_____________________

I-5-1
 Appendix A

PERFORMANCE FRAMEWORK

1. PURPOSE

1.1 The performance-based approach described in this document provides decision-makers with a consistent
method to apply when implementing changes to the air navigation system. However, there are additional aspects of a
performance-based approach for which consistency brings about advantages. This appendix provides constructs
covering several of these aspects:

— a performance hierarchy that helps to describe how performance changes at one level can influence
high-level objectives (see 2);

— a measurement taxonomy that describes how metrics are defined within each key performance area
(KPA) (see 3);

— a structured view of the air navigation system required for explicit and consistent definitions of metrics
(see 4); and

— a description of the performance process maturity level (see 5).

1.2 Together, these constructs are called a performance framework. In essence, a performance framework is
the set of definitions and terminology describing the building blocks used by a group of ATM community members to
collaborate on performance management activities.

1.3 Consistency in the above areas brings about the following advantages:

— End-to-end performance — In the globally interoperable air navigation system, the ability to
consistently describe and understand the performance impacts across multiple ANSPs provides a
basis from which end-to-end performance can be understood and improved through the collaborative
application of the performance-based approach.

— Benchmarking — Consistency in performance description facilitates a better understanding of

attainable levels of performance through comparison.

— Best practices — The ability to consistently understand performance facilitates the establishment of
best-practices for delivering improvements in air navigation system performance. When faced with
performance gaps, the application of best practices to resolve these gaps provides a proven approach
for performance improvement, since knowledge gained at one point in the global air navigation system
is knowledge gained across the whole system.

— Accountability — Knowledge of benchmarks and comparable performance allows individual ANSP

and regions to hold themselves accountable when decisions are made to improve local performance.

I-App A-1
I-App A-2 Manual on Global Performance of the Air Navigation System

— Consistency in requirements — With aircraft operating in an increasingly global environment,

performance requirements imposed on aircraft can be defined more consistently.

— Service delivery reporting — A global framework and consistent service performance reporting
allows operators to determine the level of expected air navigation service delivery performance in a
consistent manner.

2. PERFORMANCE HIERARCHY

2.1 A five-level global ATM performance hierarchy is illustrated in Figure I-A-1. The five levels represent
different views of the air navigation system performance (e.g. external, functional, system, technological) from the
highest-level political and socio-economic views to the lowest-level technological views.

2.2 What the hierarchy shows is that the actual performance of one level impacts the actual performance of
the level above it. Conversely, performance requirements of one level affect the requirements of the level below it. While
the performance between layers is related, one should not expect a simple mathematical relationship. The hierarchy is
simply meant to help in explaining how operational improvements affect the performance of the air navigation system.

2.3 As a simple example, consider a technology change that improves the positional accuracy of each
surveillance report. This can be considered a change at level 5 with performance specifications for this technology.
Incorporation of this technology into surveillance systems with specific design characteristics (e.g. update rates, software
algorithms, reliability) provides a level of surveillance performance at level 4. In order to deliver separation services, the
air navigation system must consider many other systems (e.g. navigation, communication) in conjunction with
procedures and human factors. The performance of the separation services would be considered at level 3.
Improvement in separation service performance at level 3 can then lead to better performance in areas such as safety,
capacity and efficiency at level 2.

Level 1: Socio-political and economic

requirements

ATM system
Level 2: Expectations — RASP expectations
(CONOPS)

Level 3: ATM component People

functionality — RTSP procedures
Infrastructure

Level 4: System requirements System design

(e.g. RNP, RCP)

Level 5: Technologies (including standards and Technology

specifications)

Figure I-A-1. Performance hierarchy

Part I — Global Performance
Appendix A. Performance framework I-App A-3

2.4 The example illustrates that the performance hierarchy allows a consistent description of benefit
mechanisms. This description shows how performance improvements can cascade through the different levels of the air
navigation system and impact the ATM community performance expectations. Through the establishment of metrics on
performance at each level, one can understand if the anticipated operational performance is being achieved at each
level of the hierarchy, providing information necessary to better understand what is happening and take corrective action,
if necessary.

2.5 The five levels of the hierarchy are described in more detail below.

2.6 Level 1: Socio-political and economic requirements

The societal expectations from the air navigation system can be influenced by socio-political and economic conditions.
Changes in this environment can lead to sudden shifts in expectations levied on the air navigation system. For example,
one can understand how events can cause an increase in focus in such areas as security or environmental performance.

2.7 Level 2: Expectations RASP (required air navigation system performance)

This level represents the outcome of the air navigation system as measured by indicators describing the system-level
performance within the 11 KPAs. The collection of targets for these indicators represents the required air navigation
system performance. It is expected that these would evolve over time. The actual air navigation system performance
refers to the performance as measured by the collection of indicators.

2.8 Level 3: ATM component functionality, RTSP (required total system performance)

This expresses the performance of the functions and services that the air navigation system provides. It is by performing
these that the air navigation system can deliver the outcomes at level 2. For example, by performing functions (e.g. flow
management, separation assurance) the air navigation system delivers a level of safety, capacity and environmental
impact. The concept of required total system performance (RTSP) can be applied at this level. When considering a given
air navigation system, a combination of required system performance can be required to achieve a desired level of
function and service performance.

2.9 Level 4: System requirements

The air navigation system is broken down into specific system performance requirements. These include both ground-
based and airborne systems. Requirements on airborne systems to achieve a level of service delivery would be
considered at this layer (e.g. RNP). Many combinations of system performance may be possible to deliver the same
RTSP at level 3.

2.10 Level 5: Technologies

At this level, the technologies implemented by the systems are considered. Improvements in technology can impact the
performance of multiple systems based upon common technology. However, two systems with different designs based
upon identical technology may have a different system-level performance at level 4.
I-App A-4 Manual on Global Performance of the Air Navigation System

3. MEASUREMENT TAXONOMY

3.1 A structured method for the breakdown of KPAS into performance indicators and targets is illustrated in
Figure I-A-2. Each step is described in further subsections below.

3.2 The breakdown starts with eleven KPAs matching the global performance expectations in the Global Air
Traffic Management Operational Concept (Doc 9854). It may be useful to aggregate these eleven areas into groups of
related performance areas for ease of communication outside of the ATM community. These can be aggregated
according to various criteria such as degree of visibility or between tightly coupled performance areas.

3.3 Global categorization framework

3.3.1 Eleven KPAs are defined, one per ICAO Operational Concept Document (OCD) expectation. KPAs are
named after their corresponding expectation.

3.3.2 KPAs are defined/updated in the performance-based approach during Step 1.3: Identify ambitions and
expectations.

What
KPAs Eleven standard KPAs Global categorization framework
(very general)

What Focus
(less general) areas What will performance policy focus on?

Definition of Generic What will be improved?

performance policy objectives (specific definition of what will be improved)

Application of When
Instantiated Performance
Where
performance policy objectives targets

stakeholder-specific
Who

Time, location and

Quantification of current/expected performance Performance Indicator
(data aggregation + indicator value calculation) indicators values

Data collection and forecasting Supporting

Data
metrics

Definitions and qualitative statements

Numbers with a context (time, location, etc.)

Figure I-A-2. Illustration of measurement taxonomy

Part I — Global Performance
Appendix A. Performance framework I-App A-5

3.3.3 The KPAs are described in more detail below. The text is taken from Appendix D of the Global Air Traffic
Management Operational Concept (Doc 9854).

— Access and equity. A global air navigation system should provide an operating environment that
ensures that all airspace users have the right of access to ATM resources needed to meet their
specific operational requirements; and ensures that the shared use of the airspace for different
airspace users can be achieved safely. The global air navigation system should ensure equity for all
airspace users that have access to a given airspace or service. Generally, the first aircraft ready to
use the ATM resources will receive priority, except where significant overall safety or system
operational efficiency would accrue or national defence considerations or interests dictate by providing
priority on a different basis.

— Capacity. The global air navigation system should exploit the inherent capacity to meet airspace user
demand at peak times and locations while minimizing restrictions on traffic flow. To respond to future
growth, capacity must increase, along with corresponding increases in efficiency, flexibility, and
predictability while ensuring that there are no adverse impacts to safety giving due consideration to the
environment. The air navigation system must be resilient to service disruption and the resulting
temporary loss of capacity.

— Cost effectiveness. The air navigation system should be cost effective, while balancing the varied
interests of the ATM community. The cost of service to airspace users should always be considered
when evaluating any proposal to improve ATM service quality or performance. ICAO guidelines
regarding user charge policies and principles should be followed.

— Efficiency. Efficiency addresses the operational and economic cost effectiveness of gate-to-gate flight
operations from a single-flight perspective. Airspace users want to depart and arrive at the times they
select and fly the trajectory they determine to be optimum in all phases of flight.

— Environment. The air navigation system should contribute to the protection of the environment by
considering noise, gaseous emissions, and other environmental issues in the implementation and
operation of the global air navigation system.

— Flexibility. Flexibility addresses the ability of all airspace users to modify flight trajectories dynamically
and adjust departure and arrival times thereby permitting them to exploit operational opportunities as
they occur.

— Global interoperability. The air navigation system should be based on global standards and uniform
principles to ensure the technical and operational interoperability of air navigation systems and
facilitate homogeneous and non-discriminatory global and regional traffic flows.

— Participation by the ATM community. The ATM community should continuously be involved in the
planning, implementation, and operation of the system to ensure that the evolution of the global air
navigation system meets the expectations of the community.

— Predictability. Predictability refers to the ability of the airspace users and air navigation service
providers to provide consistent and dependable levels of performance. Predictability is essential to
airspace users as they develop and operate their schedules.

— Safety. Safety is the highest priority in aviation, and ATM plays an important part in ensuring overall
aviation safety. Uniform safety standards and risk and safety management practices should be applied
systematically to the air navigation system. In implementing elements of the global aviation system,
I-App A-6 Manual on Global Performance of the Air Navigation System

safety needs to be assessed against appropriate criteria, and in accordance with appropriate and
globally standardized safety management processes and practices.

— Security. Security refers to the protection against threats, which stem from intentional (e.g. terrorism)
or unintentional (e.g. human error, natural disaster) acts affecting aircraft, people or installations on
the ground. Adequate security is a major expectation of the ATM community and of citizens. The air
navigation system should therefore contribute to security and should be protected against security
threats. Security risk management should balance the needs of the members of the ATM community
who require access to the system with the need to protect the air navigation system. In the event of
threats to aircraft or threats using aircraft, ATM shall provide responsible authorities with appropriate
assistance and information.

3.4 The focus of performance policy

3.4.1 Focus areas are defined within each KPA to identify and delineate the broad areas in which there are
intentions to establish a performance policy via the definition of generic objectives. Focus areas may be defined as a
result of a high-level analysis indicating areas where performance must be addressed in any given KPA. For example, in
the safety KPA, focus may be in such areas as CFIT accidents, runway incursions, or mid-air collisions for general
aviation aircraft.

3.4.2 Focus areas are defined and/or updated in the performance-based approach during Step 2.2: Focus efforts
by defining and prioritizing performance objectives as needed.

3.5 Definition of performance policy

3.5.1 Each expectation should be reached through meeting a set of specific, measurable, achievable, relevant
and timely (SMART) objectives.

3.5.2 Generic objectives are an expression of performance policy by defining — in a qualitative but focused
way — a desired trend from today’s performance (e.g. improvement). They specifically focus on what has to be achieved,
but do not make statements about the when, where, or who. Because at this level, no mention is made about the when,
where and who, it does not make sense to try to associate numbers (indicator values or targets) at this point.

3.5.3 Generic objectives are defined and/or updated in the performance-based approach during:

— Step 2.2: Focus efforts by defining and prioritizing performance objectives as needed.

3.6 Application of performance policy

3.6.1 Once generic objectives have been described, these must be precisely defined and numerical targets must
be set. These precisely defined objectives are labelled “instantiated objectives”. The instantiated objectives deal with the
when, where and who.

3.6.2 Starting with the generic objectives previously defined, the instantiated objectives limit the scope by
describing the applicable ATM planning environment. For example, each instantiated objective is focused within a given
geographic area, time period or other criteria. At the highest level, the geographic area may be a planning region as a
whole, and the time period may be a year or a multi-year period. Other scope-limiting criteria may include the type of
flight rules (i.e. IFR vs VFR).
Part I — Global Performance
Appendix A. Performance framework I-App A-7

3.6.3 Once instantiated objectives are defined, one requires a means of knowing when the objective has been
met. This is accomplished through the establishment of a set of targets on numerical performance indicators. It is
expected that reaching the set of numerical targets corresponds to meeting the instantiated objective. Performance
targets can only be specified after indicators have been defined (described in 3.7). These must not only reflect the
achievement of the objective, but must also be defined so as to be reachable.

3.6.4 Instantiated objectives and performance targets are defined and/or updated in the performance-based
approach during:

— Step 2.2: Focus efforts by defining and prioritizing performance objectives as needed.

— Step 3.2: Define the desired speed of progress in terms of baseline and target performance.

3.7 Quantification of performance

3.7.1 Instantiated objectives require precisely defined numerical performance indicators. These serve to
establish quantitative measures that, collectively, will indicate progress towards achieving an objective. Performance
indicators should precisely describe how the indicators should be derived using supporting metrics. This includes
considerations such as limits on scope, statistical derivation, or other mathematical derivation.

3.7.2 Performance indicators are defined in the performance-based approach during:

— Step 3.1: Define how progress in achieving performance objectives will be measured and which data
will be required to do so.

3.7.3 A performance indicator can be used to obtain indicator values by measuring actual or historical data and
applying the definition of the indicator. Indicator values can also be obtained by using forecast data to obtain projected
performance. Thus, indicator values can represent historical, current or projected performance within the scope of a
given geographical area, time period or other scope-limiting criteria.

3.7.4 Investigation of projected performance under a scenario without improvements will provide the
performance gaps. These are evaluated by determining the indicator values on projected data. This corresponds to the
following step in the performance-based approach:

— Step 4.1: Select the decisive factors to reach the target performance.

3.7.5 Once performance gaps have been identified, a set of candidate improvements can be applied to a
forecast (e.g. through simulation, modelling or analysis). This investigation will provide the indicator values with the
projected improvements corresponding to the projected performance once various improvements are implemented. This
is required as part of the following step in the performance-based approach:

— Step 4.2: Identify solutions to exploit opportunities and mitigate the effects of the selected drivers and
blocking factors.

3.7.6 A comparison of the projected performance to the targets can be used during:

— Step 4.3: Select a sufficient set of solutions.

I-App A-8 Manual on Global Performance of the Air Navigation System

3.7.7 Assuming that a set of solutions is successfully selected and implemented (e.g. Step 5), the future system
must be monitored to determine if the objectives have been achieved. At a future date, the comparison of current
performance (indicator values on current, actual data) to the required targets will be used to measure progress towards
meeting the objective. This corresponds to the following step in the performance-based approach:

— Step 6: Assess achievement of objectives.

3.8 Data collection and forecasting

3.8.1 Supporting metrics are required to compute the performance indicators. The supporting metrics define
which data need to be collected and/or forecasted to calculate values for the performance indicators. Definitions must be
sufficiently precise to allow individuals to duplicate the exact measurement without “insider knowledge”.

3.8.2 Associated with each definition is the actual data which is used to create the indicator values.

3.8.3 Supporting metrics are defined in the performance-based approach during:

— Step 3.1: Define how progress in achieving performance objectives will be measured and which data
will be required to do so.

3.8.4 Forecasting of data is required for gap analyses during:

— Step 4.1: Select the decisive factors to reach the target performance.

3.8.5 Additional forecast data is required for evaluation of candidate solutions during:

— Step 4.2: Identify solutions to exploit opportunities and mitigate the effects of the selected drivers and
blocking factors.

3.8.6 Data for supporting metrics are collected in the performance-based approach during:

— Step 6: Assess achievement of objectives.

4. THE NEED FOR A STRUCTURED VIEW OF THE AIR NAVIGATION SYSTEM

4.1 The prior description of a measurement taxonomy highlighted the need for precise definitions in the areas
of instantiated objectives, performance indicators and supporting metrics. This level of definitional precision can only be
achieved through a consistent and structured description of the air navigation system. This structured description
requires an understanding of the entities, activities and interactions involved in the air navigation system.

4.2 Performance indicators and metrics are typically defined through the combination of activities, entities and
temporal scope; for example, the number of events applicable to specific entities (i.e. daily number of flights entering a
specific airspace volume). Statistical derivation may also be applied to these measurements (i.e. average daily count).

4.3 Through consistent definitions of entities, activities and interactions, one can begin to define performance
indicators and metrics in a harmonized manner. Agreement in this area would represent a first step in the development
of harmonized performance indicators. It is through consistency and definitional precision across the global air
navigation system that the benefits described in 1 (Purpose) can have maximum impact. Inconsistencies and lack of
precision between any geographical areas will provide barriers to achieving the benefits described. It should be noted
Part I — Global Performance
Appendix A. Performance framework I-App A-9

that agreement on performance indicators and metrics in some areas may not require a structured description of the
entire air navigation system.

4.4 A harmonized structured view of the air navigation system is also important for harmonized performance
data collection and storage, a subject described in more technical detail in Appendix C, 4 (Performance data
management: basic concepts). Only those areas in which data collection and storage will be established require a
structured view. If this is not developed and harmonized prior to the establishment of data collection and storage, those
responsible for the development of data collection and storage will have to create a structure.

5. PERFORMANCE PROCESS MATURITY

5.1 The purpose of performance management is to ensure a better performance outcome. For this reason,
embarking on the performance-based approach should certainly focus on the evaluation and improvement of the
performance of the air navigation system. It is recognized, that this performance evaluation and improvement will be
conducted through a performance management process tailored to the needs of the individual organization. However, if
this performance management process is poor, it is likely that the outcome of the process will be suspect. For this
reason, an assessment of the performance management process itself provides a structured mechanism to ensure the
quality of the process.

5.2 In a manner analogous to the derivation of KPAs for the air navigation system, the evaluation of the
performance process maturity can be described in terms of focus areas, performance objectives and indicators in
various process capability areas (PCAs). Since it is expected that organizations will tailor the process to their needs,
these PCAs will not likely be uniform like the KPA. However, an example of various PCAs is provided in Table I-A-1.

5.3 For example in the Safety KPA, the rules and processes PCA would be used to define focus areas
regarding:

— availability and maturity of safety regulation; and

— existence and maturity of safety management systems.

5.4 As previously mentioned, such focus areas serve to diagnose weaknesses in the performance
management process (in this example, safety management), followed by specific (performance-based) initiatives to
improve the process or its deployment across involved stakeholders.

Table I-A-1. Example of process capability areas (PCAs)

PCA Title Subjects for which the maturity should be monitored

Systemic and Policy and To what extent does a well-defined, mature, politically accepted policy for
institutional objectives each KPA exist, and is it adequately translated into performance
capability areas objectives?

Indicators and To what extent have well-defined, mature, politically accepted indicators
targets and targets been agreed? Do the indicators properly reflect the intent of
the performance objectives, and do they result in clear and suitable criteria
to determine when and where the performance objectives have been
achieved?
I-App A-10 Manual on Global Performance of the Air Navigation System

PCA Title Subjects for which the maturity should be monitored

Rules, Is a consistent, coordinated performance management process

processes and implemented throughout the organization or planning region? Across all
organizational KPAs?
excellence
Is it supported by an appropriate set of rules, regulations, laws, processes,
procedures and practices?

Does this include a performance data reporting process in which all

relevant stakeholders participate?

Is the organization or planning region organized in such a way that its

strategic, pre-tactical and tactical planning processes guide the making of
impact assessments and trade-off decisions, as well as the planning of
desired performance, and do these processes provide a general
framework which prevents or at least minimizes deviation from top-level
policy and the performance targets?

Operational Planning To what extent are the strategic, pre-tactical and tactical operational plans
capability areas developed and optimized in function of the targets to be achieved, while
respecting the needs of other KPAs and other stakeholders?

Execution To what extent are sufficient resources and management capabilities

available to ensure that the strategic, pre-tactical and tactical operational
plans can be successfully executed?

Under nominal conditions (no significant unexpected events occur after

the plans have been developed), are the plans consistently executed in
such a way that the actual performance is closely following the plan?

Detection During execution of the strategic, pre-tactical and tactical operational

plans, to what extent are mechanisms used to ensure that deviation from
planned performance is predicted and detected as early as possible?

Is information regarding such developing deviations immediately made

available for mitigation purposes?

Mitigation To what extent is the performance management process successful in

mitigating the effects of developing deviations from planned performance?

Outcome During execution — under nominal, degraded and disrupted conditions,

and taking the corrective effects of detection and mitigation into account
— to what extent, and how often does actual performance meet the
agreed targets?

Is the performance management process able to measure the actually

achieved performance at the required level of granularity, with a high
degree of accuracy and completeness?

Is sufficient supporting data available to develop a thorough understanding

of causes and effects?
Part I — Global Performance
Appendix A. Performance framework I-App A-11

PCA Title Subjects for which the maturity should be monitored

Impact To what extent is the performance management process able to measure

and assess the impact of performance on the value chain of the air
navigation system (distribution of costs and benefits amongst all
Stakeholders), and on a wider scale, determine its impact on society,
environment and the economy at large?

Recovery When unexpected events disrupt the operation of the air navigation
system, with an impact on performance, to what extent is the performance
management process able to re-establish normal performance levels with
a minimum of delay?

Post-operational Evaluation To what extent is performance review addressing performance in an

capability areas adequate manner, and how well is it able to develop “lessons learned”?

Improvement To what extent is the performance management process able to respond

to the ‘lessons learned’ which are the result of the performance review, i.e.
use these to achieve continuous improvement?

_____________________
 Appendix B

PRIORITIES, TRADE-OFFS AND RISKS

1. INTRODUCTION

This appendix points to references in Chapter 2, as an introduction to discussing the reasons for priorities, and the
consequent need for trade-offs. Also explained is the integral role that risk management plays in performance
management.

2. REFERENCE TO THE
PERFORMANCE MANAGEMENT PROCESS

Chapter 2 addresses priorities, trade-offs and risks in the following performance management process steps (only
relevant text is quoted here):

— Step 2.1: Develop a list of present and future opportunities and issues that require performance
management attention

• Develop a good understanding of the opportunities and issues is to be done early in the process
to provide background information for deciding which performance objectives to set, what to
measure and how/where to change the system.

• The result of the issue analysis should include risks: risk (probability/severity) analysis plays a
role in performance management when there is uncertainty associated with certain events or
factors, while recognizing that their occurrence can have a severe impact on meeting the
performance expectations.

— Step 2.2: Focus efforts by defining and prioritizing performance objectives as needed

• Prioritization is required because — even though the scope of the process has already been
limited — in practice, not everything can and needs to be performance managed.

• Prioritization is supported by risk management which helps identify the risks that are most urgent
or must be avoided, those that should be transferred or reduced, and those that are reasonable to
retain.

• Prioritizing the performance-based approach implies that performance objectives will only be
defined in those focus areas where a real (present or anticipated) need for action and
improvement has been identified.

I-App B-1
I-App B-2 Manual on Global Performance of the Air Navigation System

— Step 3.2: Define the desired speed of progress in terms of baseline and target performance

• Once the scope of a target has been agreed, it becomes clear where and at which level
performance management will need to be applied, between which stakeholders the achievement
of the objective needs to be coordinated, and who will need to be involved in trade-off decisions.

— Step 4.1: Select the decisive factors to reach the target performance

• The outcome of this activity is a selection and prioritization of opportunities and issues.

• This part of the process eliminates/defers issues that do not immediately or significantly affect the
achievement of objective(s).

• To make progress in reaching an objective, the dominant factors need to be undertaken first.

— Step 4.2: Identify solutions to exploit opportunities and mitigate the effects of the selected drivers and
blocking factors

• Decision-makers need to gain a good understanding of the strategic fit, the benefits, cost and
feasibility of each option.

• Their expected performance enhancement contribution needs to be known.

• The performance assessment should also have identified all relevant side effects. Information on
side effects is needed for trade-off decision-making. This covers disadvantages such as
increased cost, increased resource consumption, unintended performance reduction in other
areas, etc.

— Step 4.3: Select a sufficient set of solutions

• In many cases, options interfere with each other (performance is positively impacted by one
solution, and negatively by another). Trade-off considerations will need to be part of the decision
process. The aim is to make a balanced decision.

3. PRIORITIES

As can be seen above, the notion of priorities appears in several parts of the performance management process, for
different reasons:

— To ensure that the performance-based approach is applied at opportune moments when and where it
will bring desired benefits.

• An initial (largely qualitative) analysis is made of the potential issues and opportunities — a
corresponding list of focus areas is defined with each KPA.

• Issues and opportunities are prioritized according to external need, impact, risk, available
resources, urgency, etc. The result is that performance objectives, indicators and targets are only
defined for a subset of the focus areas.
Part I — Global Performance
Appendix B. Priorities, trade-offs and risks I-App B-3

• Priorities can change over time. This can be triggered by the magnitude of performance gaps
(e.g. in the late 1980s escalating delays in Europe raised the priority of capacity management), by
external events (e.g. the events of 11 September 2001 in the United States raised the priority of
security issues), by changing expectations, ambitions, etc. As a consequence, additional focus
areas, new objectives, different indicators, changed targets, etc., will need to be defined as time
passes by.

— To maintain acceptable performance levels in KPAs which are critical for meeting general
expectations and realizing strategic ambitions.

• Safety is always a priority in aviation.

• Other KPAs become a priority when local circumstances so dictate.

• Because expectations and ambitions may differ depending on time, location, stakeholder, etc.,
KPA prioritization will also be different.

4. TRADE-OFFS

4.1 Performance trade-offs to achieve a balanced result

4.1.1 Air navigation system performance management covers a broad spectrum of subjects: safety, security,
environment, capacity, flexibility, predictability, etc. Many of these areas are mutually interdependent, which means that
improving the performance in one area can come at the price of reduced performance in another area. This type of
dilemma leads to the need for a “balanced approach” to performance.

4.1.2 A “balanced approach” is the result of trade-off decision-making between the various performance
objectives and targets.

4.1.3 Preferably, innovative solutions should be chosen which are able to overcome the need for (some of the)
trade-offs. History is full of examples where trade-offs were once necessary due to certain (technical or operational)
limitations. Together with the elimination of those limitations, the need for trade-offs has disappeared.

4.1.4 However, if trade-offs are unavoidable, there is a need to make decisions based on priorities between the
objectives and targets.

4.1.5 Different levels of trade-off challenges can be distinguished:

— “Easy”: it is possible to reach all targets, but maybe we need to accept somewhat lower (but still
satisfactory) performance in some areas in order to reach the target in other areas. Despite the fact
that all targets are met, one should still select a set of solutions which delivers balanced performance.

— “Difficult”: not all targets can be reached simultaneously. One has to decide which performance
aspects have priority (e.g. safety). These are not subject to trade-off. Note that performance priorities
may depend on circumstances. For the non-priority objectives, the combination of solutions should be
chosen in order to reach a balanced performance, despite the fact that some targets will not be met.
I-App B-4 Manual on Global Performance of the Air Navigation System

— “Mission impossible”: the available solutions will improve performance, but no targets can be reached.
When this happens, one should re-evaluate the targets. Despite the fact that targets are not reached,
the combination of solutions should deliver a balanced performance, just like in the previous cases.

4.2 Optimum balance as a performance objective

Frequently, the term “balanced approach to performance” implies the aim to achieve some “optimum performance”
across different performance areas, objectives, metrics, etc. In the spirit of the performance-based approach, such an
aim should be treated as a true performance objective, with its own indicator. Typically, this indicator takes the form of a
performance index, a weighted score, or the monetized cost and/or benefit of all other performance aspects.

4.3 Trade-offs between key performance areas (KPAs)

4.3.1 Examples of the need to balance performance between KPAs include:

— Flight efficiency versus capacity: objectives related to providing flight trajectories closer to user
preferred trajectories may have to be balanced against the objective of increasing capacity.

— Flexibility versus capacity: airspace users’ ability to modify flight trajectories or arrival and departure
times may come at the expense of the capacity of the air navigation system.

— Access versus capacity: the access of all aircraft, irrespective of their equipage or size, to a certain
airspace or airport can have an impact on the capacity provided.

— Financial cost-effectiveness versus quality of service (flight efficiency, flexibility and predictability): the
need to reduce the cost of providing ATM capacity may have to be balanced against the need to limit
the cost of delay due to capacity shortages.

4.3.2 To improve overall performance when there are interdependencies, one must first determine if there are
conflicting objectives that need to be balanced. When conflicting objectives emerge, techniques from multi-criteria
decision-making (MCDM) should be applied (a technique also mentioned in Chapter 2, Figure I-2-3). While a detailed
treatment of methods in this area are beyond the scope of this manual, some of the types of techniques that can be
considered are shown below:

— Development of a common performance metric across multiple objectives. One example is the
expression of all objectives into a “common currency” to be optimized (see also 4.2).

— Techniques allowing decision-makers to rank preferences between alternatives with multiple known
performance attributes. Consistency in ranking is verified, and numerical techniques consider the
relative rankings to provide a best alternative. An example of this technique is the analytical hierarchy
process (AHP).

— When not all performance attributes can be quantified, multi-attribute utility methods can be applied.
Part I — Global Performance
Appendix B. Priorities, trade-offs and risks I-App B-5

4.3.3 When considering trades between key performance indicators (KPIs), several concepts are illustrated in
Figure I-B-1. Differences between two alternatives can result in an improvement or degradation in the values of different
KPIs as shown on the radar plot (left diagram in Figure I-B-1). Decision-makers must determine which choices represent
acceptable trades. To illustrate how this trade may be accomplished, consider the trade between two areas of
performance. Alternatives may lead to multiple operating points as shown in the figure; however, a Pareto frontier will
exist indicating those operating points with best performance in an area assuming fixed performance in the others. In the
air navigation system, there are many examples of this type of trade. Examples include: fuel consumed versus delay or
level of inequity versus system-wide delay.

4.4 Trade-offs between parts of the air navigation system

4.4.1 This question arises whenever the achievement of an overall target has to be accomplished via the
allocation of resources. It is about deciding whether to improve the performance of all system elements in a comparable
manner, or whether to introduce differential treatment in order to maximize the collective benefits and minimize collective
costs.

4.4.2 For the sake of illustrating this point, consider for example a planning region (performance scope),
consisting of States, stakeholders and aircraft (system elements). In order to improve the overall safety performance of
the region by a certain percentage and with a constrained regional budget (parameters which are part of the target
performance), it may not be realistic to attempt equipping all aircraft with the required avionics. In this example, one will
have to make a trade-off in the local allocation of budget: which part of the fleet to equip and which part not to equip, in
order to maximize the overall safety benefits, i.e. achieve the target set at regional level.

4.4.3 There is also a connection with the subject of so-called “network effects”. In matters of performance, the
total system is not always equal to the sum of its parts. A typical example is capacity planning at the regional level.
Because of traffic flow patterns, bad performance in one part of the system may affect performance elsewhere; only
removal of bottlenecks will lead to real overall network (system) performance improvement.

4.4.4 This kind of trade-off decision-making has a relationship with the subject of prioritization.

4.5 Suggested approach

4.5.1 To deal with the issue of performance trade-offs in an effective way, the subject should be approached in a
careful, stepwise fashion:

— During Step 2.2 and Step 3.2 of the performance management process: Start without immediately
attempting to include trade-off considerations. This simplifies the discussions and permits a better
focus on what should be achieved in each KPA to meet the expectations. During this step, one should
define the various performance objectives and set initial (pre-trade-off) targets.

— During Step 4.2 of the performance management process: As part of identifying candidate solutions to
mitigate the effects of selected drivers and blocking factors, the expected performance of each
solution should be assessed. At the same time, lateral interdependencies between KPAs and network
effects leading to trade-offs need to be analysed and discussed within the ATM community.

— During Step 4.3 of the performance management process: Using the shared understanding developed
during the previous step, multi-criteria decision analysis (MCDA) techniques can be applied to select a
suitable set of solutions. At this stage, the combined performance of the entire solution set should be
assessed. In some cases, this may reveal that the initial targets are incompatible and that priorities
need to be defined and a trade-off choice needs to be made.
I-App B-6 Manual on Global Performance of the Air Navigation System

KPI 1 Trade between KPIs

1
1
KPI 11 0.75 KPI 2
0.9

0.5
0.8
KPI 10 KPI 3 0.7
0.25
0.6

KPI 2
0.5
0
0.4
KPI 9 KPI 4
0.3
0.2 Pareto frontier
0.1
KPI 8 KPI 5
0
0 0.2 0.4 0.6 0.8 1
KPI 7 KPI 6
KPI 1

Option 1
Option 2

Figure I-B-1. Illustration of trade-offs between key performance indicators (KPIs)

4.5.2 To summarize: after the initial target setting, the performance management process must take account of
the identified interdependencies and trade-off aspects. Where the simultaneous meeting of different targets is not
possible, the balance between targets must be adjusted so that they reflect an acceptable and feasible compromise. The
aim is to ensure that the total set of agreed targets reflects society’s priorities in the high-level expectations.

5. RISKS

5.1 Risk management is an integral part of performance management. A significant amount of literature is
available on this subject, including for example in the Safety Management Manual (SMM) (Doc 9859).

5.2 The objective of risk management is to reduce different risks related to a pre-selected domain to the level
accepted by the ATM community or society in general. It may refer to numerous types of threats caused by environment,
technology, humans, organizations and politics. On the other hand, it involves all means available for humans, or in
particular, for a risk management entity (person, staff, and/or organization).

5.3 Risk management plays a role in performance management when dealing with rare events or when there
is uncertainty associated with influencing factors — recognizing that the occurrence of events can have a severe impact
on meeting the performance expectations.

5.4 Risk management applies in all KPAs, but as a technique it is mainly used in the safety and security areas.

5.5 During risk management one quantifies the likelihood (or frequency) and severity of impact (or
consequence) of identified risks in order to prioritize risk response activities. Risk management helps identify the risks
that are most urgent or must be avoided, those that should be transferred or reduced, and those it is reasonable to retain.
Part I — Global Performance
Appendix B. Priorities, trade-offs and risks I-App B-7

5.6 These principles are shown in Figure I-B-2.

Identified Identified
source threat

Identified
risk
Risk identification

Certain Catastrophic
Likely Significant
Risk Risk
Possible Moderate
probability severity
Unlikely Minor
Remote Insignificant

Acceptable
Risk
Tolerable
acceptance
Unacceptable
Risk assessment

Accepted / Tolerated / Retained

Managed Controlled / Treated / Mitigated / Reduced
risk Avoided / Terminated / Eliminated
Transferred to another party
Risk treatment

Figure I-B-2. Risk management principles

_____________________
 Appendix C

DATA AND INFORMATION MANAGEMENT

1. INTRODUCTION

1.1 At its most basic level, a performance-based approach simply needs to apply the basic principles on data
collection as set out in Chapter 3. However, as the performance management process matures, it will be necessary to
incrementally enhance the data collection, quality management and reporting mechanisms in order to continually
improve the process.

1.2 Air navigation system performance management at its most developed level is very much a data driven
process, with data quality and interoperability being critical success factors. Therefore, this appendix places significant
emphasis on the practical aspects of (performance) data management. By its very nature, this is a technical subject, and
it is natural that this is not only discussed using terminology familiar to ATM experts, but also requires the definition and
use of some IT (Information Technology) terminology.

1.3 This appendix contains practical guidance on how to set up the data acquisition process needed for
performance monitoring, how to aggregate performance data and exchange it between planning groups, how groups
can best manage their information base in which performance data is stored, and how to organize performance
evaluation. Other subjects include global harmonization of such work, including definitions, standards for reporting
requirements, and information disclosure.

2. ISSUES DRIVING THE SET-UP OF THE PERFORMANCE MONITORING PROCESS

The act of performance monitoring implies the need to establish a flow of performance data originating from all parts of
the air navigation system that are being monitored. This data then needs to be cleaned, integrated and recorded in a
repository for further processing. For instance, to conduct the performance evaluation, calculation of summaries and
corresponding performance indicators, comparison with targets, etc., must be completed. While this sounds simple,
practice has shown that there are numerous issues which complicate this task. This is illustrated by the following non-
exhaustive list:

— Typically a large number of ATM community members are involved in the performance data reporting
process. Any decision to collect data has wide ranging consequences at the “roots level”, because
each of the participants must be convinced that it is worthwhile to spend the extra effort required. Even
if everyone is willing to participate, the data collection process will fail if it is not properly supported by
common definitions, standards for reporting requirements, and information disclosure agreements.
There may also be legal, organizational and managerial impediments to the reporting of performance
data.

— Issues can exist with the routing of data. For historical, legal, organizational and managerial reasons,
some data reporting flows are unnecessarily complex: from the point of origin until the central
repository where performance evaluation takes place, the data may pass through many “hands”

I-App C-1
I-App C-2 Manual on Global Performance of the Air Navigation System

(organizations), each interpreting, aggregating, filtering and transforming the data. Typically each
“station” in the data supply chain sees only its immediate suppliers and immediate users. The end
result of such a situation is an unmanageable overall data supply chain, and a high risk of poor data
flow continuity, quality, timeliness and completeness.

— Some of the data may be quite sensitive, and the suppliers of that data must trust that no
inappropriate use is made of their data. This is certainly true for financial (cost effectiveness) and
safety-related data. The existence of mechanisms to ensure confidentiality is one of the major issues
here.

— Data completeness — availability of data to ensure a defined level of coverage in all dimensions — is
often difficult to achieve across time, geography and other classifications (e.g. limited historical data,
incomplete geographical data and limited availability of VFR traffic statistics).

— One must always be wary of the risk that the information base can easily and unknowingly become
contaminated with bad data. Failing to keep the data quality up to standards will have negative
downstream effects on the entire performance-based approach.

— Although the data will finally be aggregated to a few KPIs, it is important that at the source, the data
are captured at a sufficiently detailed (usually the highest possible) level of granularity. Combined with
the fact that performance monitoring is an ongoing activity which generates a continuous influx of data
into the repository, it can easily be understood that performance data repositories can grow to
significant sizes. Tens to hundreds of Gigabytes for a group of States forming a planning region is a
realistic estimate. This creates special Information Technology (IT) challenges. It is important that the
IT staff supporting the performance monitoring process is aware of this, which helps them to choose
the right approach and make the right technological choices.

— It is certainly the intention that performance monitoring will be ongoing for a long period of time.
Therefore, the information base will become populated with historical data covering several decades.
Likewise, the time horizon for strategic performance planning implies that the repository will contain
forward-looking planning data spanning up to two decades, with new versions of such data being
generated every few years. For comparison purposes, it is important to have continuity and stability in
the data collection process. The definition of numerical metrics and the categories for which data will
be collected must be able to stand the test of time. Therefore, decisions causing changes in the data
collection process (i.e. introduction of new/different metrics and/or starting to measure at a
new/different level of granularity) should be carefully justified.

— Systematic performance monitoring does have a price: system-wide collection, cleaning,

transformation, storage, evaluation and usage of performance data add up to a significant cost in
terms of effort (required staff), investments and goodwill from all participants. For this reason, senior
management support and proper funding among all parties are critical factors for the success of the
performance-based approach.

3. DATA FEEDS AND INFORMATION DISCLOSURE

3.1 There are essentially two major categories of data feeds that performance monitoring deals with:

— Data which are captured by automatic means, and forwarded in electronic form with little or no human
intervention. This type of data feed is typical for high volume streaming measurement data and
automated database-to-database exchanges.
Part I — Global Performance
Appendix C. Data and information management I-App C-3

— Manual reporting of information (electronically or on paper). This requires human effort to collect,
interpret, analyse, structure and otherwise prepare the data for reporting. Typical for low-frequency,
complex information data feeds in which the performance monitoring organization receives processed
information (e.g. forms, reports) instead of raw data feeds.

3.2 To establish data feeds in each KPA, the following steps need to be undertaken:

— identify information needs;

— identify potential suppliers of data;

— ensure information disclosure by candidate data suppliers; and

— manage the data feeds on an ongoing basis.

3.3 The information needs are driven by the list of agreed performance indicators. As explained in Appendix A,
3.8, the values of performance indicators are usually calculated from input and output metrics. It is a data flow for the
contributing input/output metrics that needs to be established. It is recognized that there are costs (and time lags)
associated with system modifications required for establishing these data flows within and between the participating
ATM community members. Furthermore, costs associated with frequent system modifications can greatly outweigh a
single well-timed change. Consequently, this calls for a clear definition and timeline of required metrics leading to
information requirements incorporated into system development activities. While these dictate the minimum set of
information requirements, one should also recognize that there are financial and programmatic risks associated with
collecting and storing only this minimum set. These risks stem from evolving information requirements to support future
performance innovations.

3.4 Once the information needs are established, some research is necessary to identify potential suppliers.
This should not be limited to those parties which today already have the required information in their possession, but
include those who can be reasonably expected to adapt their data capture procedures and systems to satisfy the
performance data needs. In addition, one should not overlook the possibility of acquiring the desired data from
commercial sources, where possible. For cost-efficiency reasons, the effort spent by other parties (potential data
suppliers) to collect, clean and integrate data should be leveraged to the maximum extent possible, while ensuring that
known data quality standards are applied as needed.

3.5 Next, the candidate suppliers need to be approached to arrange information disclosure. Disclosure can be
by paid subscription or commercially available data, on a voluntary collaborative basis, but might require legislation to
establish a framework of legal obligations to supply the required information (particularly in KPAs such as safety,
security, cost effectiveness). This is the stage at which institutional and legal barriers need to be identified and removed.
It is also the stage where a good climate can be established and a “no-blame” reporting culture can grow. Data suppliers
must receive guarantees that their data will not be used in an inappropriate way (e.g. legal action against them,
commercially sensitive information passed on to competitors, abuse of data to create security threats, etc.).

3.6 When possible, written service level agreements (SLAs) should be concluded between the performance
monitoring organization and the data suppliers. These should specify the contents, format, start and duration of the data
flow, its scope and quality, frequency and timeliness, means of exchange, permitted use of the data, required
investments on both sides, who pays for what, etc.

3.7 Finally, the performance monitoring organization should actively manage the data feeds from all suppliers:
monitor the data delivery calendar, coordinate with data suppliers when data are not delivered as required, be prepared
to receive and process the data on time, perform overall workflow planning covering all data feeds, etc.
I-App C-4 Manual on Global Performance of the Air Navigation System

4. PERFORMANCE DATA MANAGEMENT: BASIC CONCEPTS

4.1 Performance monitoring and evaluation has been practiced in industries other than ATM for several
decades. In particular, in the area of data management, it is appropriate to rely on existing IT terminology, practices and
“know-how” rather than to invent and promote ATM-specific solutions. This section of the manual is intended to assist
the ATM community in choosing a well-proven data management approach.

4.2 In particular, the attention of the readers is drawn to the fact that the terminology, data quality management
techniques and database design philosophies for performance data repositories are quite different from those
traditionally used for operational or administrative applications. Over the past decades, the IT industry has developed a
mature body of knowledge regarding performance data management. It is however a specialized niche market within the
overall database management industry. For this reason, it should not automatically be assumed that staff members who
have a general understanding of database management are also sufficiently familiar with the specifics of performance
data management. Therefore, this section presents an overview of those specifics as a starting point for those needing
to familiarize themselves with this technical subject.

4.3 Guidance on the design of performance data storage

4.3.1 ATM performance data will need to be stored in suitable data storage facilities. For efficiency and
interoperability reasons, it is recommended that all ATM performance data storage is based on industry-standard data
warehousing concepts. This implies that the data storage facilities are implemented using concepts such as data
warehouse, data mart, fact table, dimension table, etc. The following pages introduce the main highlights of the data
warehousing approach.

4.3.2 Data warehouses

4.3.2.1 Repositories and information bases used for performance management are commonly referred to as “data
warehouses”.

4.3.2.2 A data warehouse differs from an operational database in the following ways:

— A data warehouse accumulates historic data. An operational database contains current “live” data
only.

— Data warehouses hold much larger amounts of information than operational databases.

— A data warehouse contains data which are built out of separate internal and/or external data sources
where the data are integrated in a consistent manner. In contrast, an operational database is not
loaded with data from external sources. Its contents change as a result of the processing of
operational transactions.

— The database design of a data warehouse favours data analysis and reporting in order to gain
strategic insight and to facilitate decision-making. The design of an operational database is optimized
to support (real-time) transaction processing.

4.3.3 Fact tables

4.3.3.1 Within the data warehouse, all performance data (e.g. the values of input/output metrics, performance
indicators and performance targets) are stored in “fact tables”.
Part I — Global Performance
Appendix C. Data and information management I-App C-5

4.3.3.2 A fact table is a data warehousing concept. It consists of the facts (e.g. measurements, metrics) of a
business process (see Table I-C-1).

4.3.3.3 Each fact (described by a set of metrics and/or indicator values) has a dimensional context which defines
the scope covered by the value of the metrics/indicators. In the example above, the metrics are “FL TOP” and “DIST
FLOWN” and “FLT DUR”, whereas the five dimensions in the fact table are “DEP APT”, “ARR APT”, “DEP TIME”, “ARR
TIME” and “ACFT TYPE”.

4.3.4 Dimension tables

4.3.4.1 In data warehousing, the possible values for each of the dimensions used in the various fact tables are
stored in separate tables called the “dimension tables”.

4.3.4.2 Dimension tables contain a list of items called “dimension objects”. These can be physical objects (e.g.
airframes, airports, countries, regions), but also intangible objects, classes and categories (aircraft types, flight rules,
wake turbulence categories, months, severity classifications, etc.).

4.3.4.3 These tables also contain information that defines the aggregation hierarchies for the dimensions. An
example of this is the aggregation (grouping) of time into hours, hours into days, days into months, and months into
years. The hierarchies define for each dimension how performance metrics, indicators and targets contained in fact
tables can be summarized and/or calculated at higher aggregation levels, and how high-level performance summaries
can be the starting point for breaking down the performance data into more granular levels.

Table I-C-1. Example fact table

DIMENSIONAL CONTEXT FACTS / METRICS

DEP ARR DEP ARR ACFT FL DIST FLT

APT APT TIME TIME TYPE TOP FLOWN DUR

LFPG LIPZ 01/01/2003 08:55 01/01/2003 10:20 MD82 310 511 65

LFSB LEAL 01/01/2003 08:55 01/01/2003 10:55 E145 370 726 113

LSZH LTBA 01/01/2003 08:55 01/01/2003 11:36 A321 390 1015 141

EDDF KMIA 01/01/2003 08:55 01/01/2003 18:28 B744 380 4268 558

EDDF KDFW 01/01/2003 08:55 01/01/2003 19:32 A343 340 4578 622

LTBA LTAU 01/01/2003 09:00 01/01/2003 10:13 B734 290 353 58

LEBL LEMH 01/01/2003 09:00 01/01/2003 09:57 MD87 170 186 45

ESSV ESKN 01/01/2003 09:00 01/01/2003 09:40 C208 20 84 35

LFML DTTA 01/01/2003 09:00 01/01/2003 10:20 A319 370 481 70

EHAM HAAB 01/01/2003 09:00 01/01/2003 16:45 B752 390 3225 450

LSGG EGGP 01/01/2003 09:00 01/01/2003 10:50 B733 320 571 100

EDDM LIMC 01/01/2003 10:05 01/01/2003 11:04 CRJ2 290 299 39

EDDM EDDW 01/01/2003 10:05 01/01/2003 11:18 CRJ7 320 357 53

I-App C-6 Manual on Global Performance of the Air Navigation System

4.3.4.4 Dimensions can have multiple aggregation hierarchies. For example, airframes can be aggregated to
aircraft models, which are in turn aggregated to (classified as) aircraft types, which in turn are classified into aircraft
classes. At the same time, aircraft types are also classified into (grouped by) wake turbulence category (WTC),
propulsion type and number of engines. This example (also shown in Figure I-C-1) illustrates the possible existence of
multiple aggregation hierarchies.

4.3.5 Use of fact and dimension tables in a data warehouse

4.3.5.1 Figure I-C-1 illustrates how fact and dimension tables are used together in a data warehouse design. The
fields (columns) named Movements, Flight duration and Distance flown are the metrics. The other fields represent the
fact dimensions. These refer (point at, relate, link) to entities listed in the dimension tables called Time, Airports and
Airframes. These tables, in turn, point to other dimension tables in accordance with the aggregation hierarchies. This is
the basis for summarizing the values contained in the fact table, to produce for example statistics on number of
movements, flight duration and distance flown by year, country pair and wake turbulence category (WTC).

4.3.5.2 In a data warehouse, descriptive (performance) data about dimension objects is never stored as extra
fields/columns inside the dimension tables. Such data is to be placed in separate fact tables. For example, to store
range and speed of aircraft models, one would need to create an additional fact table which links to the table models in
Figure I-C-1.

4.3.5.3 The result of the data warehousing approach is a database design containing many fact tables which link
to a limited number of dimension tables. Each dimension table can be referenced by many fact tables. Fact tables are
never directly linked to each other. The dimension tables are the “glue” which ensures that the categorization of different
indicators (stored in different fact tables) is based on common definitions.

4.3.5.4 Given that in real life, the suppliers of performance data do not always know all the dimensions of the
reported fact data, there is a need to define a value “unknown/any” for each dimension. Correspondingly, each
dimension table needs to contain a record representing this “unknown/any” value.

4.3.5.5 The contents of dimension tables do change over time: new objects (e.g. airports, airframes, cities,
countries, regions, etc.) are introduced, cease to exist or are otherwise changed. Also, the aggregation hierarchies are
subject to change. By consequence, temporality must be designed into the dimension tables in such a way that the full
history of their contents is still available for use with older fact data.

4.3.5.6 The design approach shown in Figure I-C-1 is called a “snowflake” schema: the fact table is the centre of
the snowflake, the tables containing most granular dimension objects are the inner ring, and extra “rings” of dimension
tables are connected to these to reflect the various ways in which the most granular dimensions can be aggregated. This
solution is optimum for maintaining the contents of the data warehouse, but less suitable for querying and analysing the
data.

4.3.5.7 The data warehousing community speaks of a “star” schema when the contents of the dimension tables
representing the higher levels of the aggregation hierarchies (i.e. the outer rings of the snowflake schema) are placed
into the basic dimension tables in the form of extra fields/columns. In a star schema, dimension tables are never
connected to each other. This design solution is better for querying and analysing the fact data, but less ideal for
maintaining the higher level aggregation dimensions. For example, if a country dimension table does not exist anymore,
then each change to the list of countries would have to be applied in all dimension tables referring to countries.

4.3.5.8 In an alternative form of star schema design, all dimension tables present in the snowflake schema
solution are still maintained, but all direct connections between the dimension tables are removed. Rather, these extra
dimensions are directly included in the fact tables. In the example of Figure I-C-1, this would imply the addition of fact
dimensions like departure city, arrival city, departure country, arrival country, etc. This solution is particularly useful if no
clean hierarchical relationships exist, and the relationships are really built into and reported via the fact data.
Part I — Global Performance
Appendix C. Data and information management I-App C-7

Propulsion Number of
Years Regions Classes WTC Type Engines
Dimension tables

Months Countries Types

Days Cities Models

Time Airports Airframes

Fact table: traffic

Departure Arrival Departure Arrival Flight Distance

time time airport airport Airframe Movements duration flown

Figure I-C-1. Fact and dimension tables: snowflake schema

4.3.6 Core data warehouse

For each planning region, it is advised that a complete set of performance data and dimensions are stored in a core data
warehouse. The function of the core data warehouse is to facilitate data loading, integration and quality assurance. This
core data warehouse is best implemented using the snowflake schema approach. For performance data analysis and
dissemination, separate data marts (see 4.3.7.1 below) should be built by copying (usually aggregated) subsets of the
core data warehouse contents into the data marts, after appropriate transformation and using their own publication
schedule, which is independent from the update schedule applied to the core data warehouse.

4.3.7 Data marts

4.3.7.1 A data mart is a specialized version of a data warehouse. The key difference is that the creation of a data
mart is predicated on a specific, predefined need for a certain grouping and configuration of select data. Since the
purpose of a data mart is easy access to relevant information, the star schema is a fairly popular design choice, as it
enables a relational database to emulate the analytical functionality of a multidimensional database.

4.3.7.2 A dependent data mart is a logical subset (view) or a physical subset (extract) of a larger (core) data
warehouse, isolated for one of the following reasons:

— To cater for the specific needs (and limitations) of the data analysis software which will be used to
access the data.
I-App C-8 Manual on Global Performance of the Air Navigation System

— Performance: to offload the data mart to a separate computer for greater efficiency.

— Security: to separate selectively an authorized data subset.

4.3.8 Managing dimensions and aggregation hierarchies

4.3.8.1 Even when all parties use the same performance indicators, it will be impossible to compare their values if
the different sources have incompatible views on (i.e. incompatible definitions of) the meaning of the dimensions. For
example, if two sources publish data for a region they each call “South-East Asia” or “Europe”, but each includes a
different set of States in their definition of that region, then there is no point comparing the values, or combining them to
calculate a derived performance indicator for “South-East Asia” or “Europe”.

4.3.8.2 Despite the existence of standardization organizations, the various organizations involved in air navigation
system performance planning use many variations on dimensions and aggregation hierarchies which are essentially the
same. This problem of “diverging dimension definitions” is perhaps the single most important reason for performance
data incompatibility. For successful data integration, it is essential that a common set of dimensions and aggregation
hierarchies is defined and used within the ATM performance management community.

4.3.8.3 A second issue is the codes and names used to refer to the dimensions in the reported fact data. When
incoming data are processed, these codes and names have to be decoded to reveal the true identity of the dimension
objects stored in the dimension tables.

— For technical, historical, organizational or political reasons, multiple codes and names can exist to
refer to the same dimension object. Rather than attempting to cross-reference codes (a typical
example applies to airports: ICAO 4-letter codes versus IATA 3-letter codes), the approach should be
to maintain a common set of dimension objects and to maintain the association of codes with these
objects separately.

— Implementers of data warehouses should be aware that the coverage of coding systems is not always
identical:

• Incomplete coverage: for example, not every aerodrome has an ICAO 4-letter code or an IATA 3-
letter code.

• Different object type scope: for example, not all ICAO 4-letter codes and IATA 3-letter codes refer
to aerodromes. Some ICAO codes refer to weather stations, ATC facilities, etc. Some IATA codes
refer to train stations, bus stations, cities, etc.

— While names and codes are relatively stable, occasionally objects get renamed and/or receive a
different code; it is recommended to implement an explicit translation from code to real dimension
object ID to ensure continuity of statistics through time (and to handle objects simultaneously referred
to by different codes and names). What should not be done is to create a new dimension object every
time an object’s code is changed. One should also not design an ATM performance data warehouse
on the assumption that certain sets of codes will never change.

— Sometimes, organizations modify someone else’s code system for their own use by adding extra
codes, etc. These should always be considered as a different code system (for example, data
suppliers using a modified set of ICAO 4-letter codes should not claim they are using ICAO codes).

— Codes (and names) are sometimes (in some instances frequently) re-used. Therefore, their
association with a dimension object is always to be considered temporary. For successful
Part I — Global Performance
Appendix C. Data and information management I-App C-9

interpretation of fact data, data warehouses need to keep track of the temporality of this association.
The period during which a code is associated with a dimension object, is called a code lease.
Accurately determining the start and end dates of individual code leases is one of the most difficult
things to achieve in dimension management, because that information is often not explicitly
propagated through the data supply chain. Moreover, the users of codes and names (those who
supply the fact data) often do not rigorously stick to the official validity periods: it is not uncommon to
receive fact data containing expired codes.

— Codes are administered by a code issuing authority which, in principle, ensures that, for a given
moment in time, each code provides a unique identification of a dimension object. However, this is not
the case for names that are often ambiguous (same name refers to different objects depending on
context, e.g. Paris in France and Paris in Texas; multiple name variants refer to the same object, e.g.
United States and United States of America; or different names are used in different languages for the
same object, e.g. Germany and Deutschland).

4.3.8.4 For the above reasons, it is recommended to keep the lists of code leases and name/object associations in
tables which are separate from the dimension tables. Each set of code leases for a particular code type is maintained in
a code registry table, and each set of name/object associations in a name directory table. Such a design allows tracking
of name/code temporality and the many relationships between names/codes and dimension objects. As part of the
loading of fact data, a “decode” lookup into these code registries and name directories will ensure correct translation of
codes and names into “true” (common) dimension IDs.

4.3.8.5 Two solutions are possible for maintaining temporality of dimension data:

— Working with standard dimension update cycles (e.g. AIRAC cycle). Advantages: additive loading of
dimension data, easy roll-back in case of error, easy lookup. Disadvantages: higher volume of data
(each cycle stores a complete image of all dimensions), many sources follow different update cycles
which may not be compatible with the one chosen.

— Working with validity periods (keeping track of start and expiration date of each individual object, code
and name). Advantages: independence from update cycles, lower volume of data (only changes need
to be recorded). Disadvantages: loading not additive (no easy roll-back possible after validity periods
have been changed), difficult lookup.

4.3.8.6 Maintaining a common framework of dimensions, with associated sets of codes, names, aggregation
hierarchies and correct temporality, is an essential enabler for data integration. The performance management
community should be aware that maintaining the dimensions is far more time consuming and labour intensive than the
loading of the fact data. What should definitely be avoided is to simply rely on the code values supplied by the providers
of fact data, without maintaining a quality controlled data warehouse-owned copy of the corresponding dimension data.

4.3.8.7 In the IT world, the function of maintaining a common framework of dimensions throughout disparate IT
systems and among a variety of stakeholders is called master data management (MDM).

4.3.8.8 It is recommended to perform the function of MDM in support of performance management centrally at the
global level where possible (already done for certain data such as location indicators (see Location Indicators
(Doc 7910)), aircraft type designators (see Aircraft Type Designators (Doc 8643)), aircraft operating agencies,
aeronautical authorities and services (see Designators for Aircraft Operating Agencies, Aeronautical Authorities and
Services (Doc 8585)), and otherwise centrally within each planning region. The challenge is to establish a process which
continuously ensures the integration of codes, names and registration data maintained by a large number of
independent organizations: ICAO, ISO, industry standardization bodies, States (Civil Aviation Authorities),
intergovernmental organizations, user associations (e.g. IATA), etc.
I-App C-10 Manual on Global Performance of the Air Navigation System

4.3.9 Data sets

4.3.9.1 A special type of fact data dimension is the data set. A data set is a group of records in a fact table which
logically belong together. By including a data set dimension, performance data managers can load data for different
versions, realities, assumptions or scenarios in the same table. Likewise, data sets can be used to distinguish between
measured values (indicators) and targets. If the fact data would be plotted on a graph, each data set would correspond
to a separate line. For example:

— A traffic forecast (edition 2000 for example) could consist of four data sets: the historic data, the low
growth scenario forecast edition 2000, the medium forecast edition 2000 and the high forecast edition
2000;

— Periodically, new traffic forecasts are issued. As a result, multiple data sets are present: high forecast
edition 2000, high forecast edition 2002, high forecast edition 2004, etc.

4.3.9.2 Data cannot and should not be aggregated over data sets, but the data sets can be compared against
each other: for example, different forecast scenarios, or different forecast editions, or measured values against targets.
The data set dimension is an important instrument for performance evaluation.

4.3.10 Cleaning and loading data

4.3.10.1 Performance management requires the following types of data feeds into the core data warehouse:

— Input/output metrics and performance indicators. This type of data is loaded into fact tables. Each data
load results in the simple addition of extra records/rows, followed by a process of classification (i.e.
decoding of codes and names, to associate them with dimension table objects).

— Dimension data. This type of data is usually provided as separate data feeds. The data are loaded into
dimension tables and code/name tables.

4.3.10.2 Global air navigation system performance monitoring implies that these two types of data flows are
supplied by many different sources (systems, people and organizations) within the ATM community. The data flows are
inevitably characterized by large differences in format, quality, update frequency, method of delivery, etc. The data have
to be extracted from the source media, analysed to determine the level of quality and completeness, cleaned,
transformed and finally loaded into the core data warehouse before it can be used for performance analysis.

4.3.10.3 The loading of data into a data warehouse is usually referred to by the term ETL process (extract,
transform and load). This is a process in data warehousing that involves:

— extracting data from outside sources;

— transforming data to fit performance monitoring needs; and ultimately

— loading the data into the data warehouse.

ETL is important, as it is how data actually gets loaded into the warehouse.

4.3.10.4 Extract

4.3.10.4.1 The first phase of an ETL process is to extract the data from the source systems. Most data warehousing
projects consolidate data from different source systems. Each separate system may also use a different data
organization/format. Common data source formats are relational databases, and flat files, but other source formats exist.
Part I — Global Performance
Appendix C. Data and information management I-App C-11

4.3.10.4.2 The extraction phase places the raw source data in a part of the data warehouse called the staging area.

4.3.10.4.3 Once the extraction is complete we can refer to each specific data column (sometimes also referred to as a
field) and process each row of data.

4.3.10.5 Transform

The transform phase applies a series of rules or functions to the extracted data to define the data to be loaded. Some
data sources will require very little manipulation of data. However in other cases, any combination of the following
transformation types may be required:

— Select only certain columns to load (or if you prefer, select columns not to load).

— Translate coded values (e.g. if the source system contains IATA codes but the warehouse stores
ICAO codes).

— Derive a new calculated value (e.g. avg_speed = distance_flown/time_flown).

— Join together data from multiple sources (e.g. lookup, merge, etc.).

— Summarize multiple rows of data (e.g. annual number of flights at an airport).

4.3.10.6 Load

The load phase is when the data is entered into the data warehouse. This means: adding records (rows) to the fact
tables and updating dimension tables.

4.3.11 Data quality checking

4.3.11.1 To avoid corruption of the core data warehouse contents, data should never be loaded without checking its
quality.

4.3.11.2 In practice, data quality cannot be judged from the supplier’s documentation of the data format (table
definition) or other descriptive material. For example, just because a certain data field is included in the data feed does
not mean that it will (always) contain (meaningful or useful) data. One must examine the data. Due to the sheer size of
the incoming data volume, this cannot be done manually by checking individual records or spot-checking selected data
samples. A systematic, automated process called data profiling is required. Data profiling creates a “profile” or picture of
the composition of the whole set of data from different perspectives. Its role is to identify anomalies in large quantities of
incoming performance data while it is still being held in the staging area.

4.3.11.3 Data profiling is a process whereby one examines the data available in an existing database and collects
statistics and information about that data. Typical types of metadata sought are:

— Availability: if on average 80 per cent of the flight records have an aircraft registration mark filled in,
then there must be a problem somewhere if a batch of data is processed in which none of the aircraft
registration marks is filled in.

— Domain: whether or not the data in the column conforms to the defined values or range of values it is
expected to take. For example: requested flight levels are expected to be between 0 and 700. A value
I-App C-12 Manual on Global Performance of the Air Navigation System

of 1 000 would be considered out-of-domain; a code within the fact data (e.g. aircraft type code) would
need to be present in the dimension (or code) table. If not present, the received code would be
considered out-of-domain.

— Type: alphabetic or numeric.

— Pattern: for example, a Mexican aircraft registration mark should start with XA, XB or XC.

— Frequency counts: traffic patterns have weekly and seasonal cycles. Traffic volumes reported for a
given day should be within the expected range for the given weekday and month.

— Statistics: minimum value, maximum value, mean value (average), median value, modal value,
standard deviation.

— Interdependency within a table: the airport code field always depends on the country code.

— Interdependency between tables: the aircraft type code in a flight should always appear in the aircraft
type dimension table.

4.3.11.4 It is recognized that perfect quality is not achievable or necessary in a data warehouse application (in this
case for performance management); however, the most significant data errors should be identified and rectified before
incorporating the data into the core data warehouse. While this is applicable to the fact data, it should be noted that
every attempt should be made to ensure the highest possible quality levels for the dimension data. The reason is that —
even if the basic fact data is correct — faulty dimension data can lead to significant errors in aggregated performance
indicators.

4.3.11.5 For those faced with the need to choose a suitable, efficient method for processing the received
performance data and loading it into their core data warehouse, it is advised to buy a commercially available ETL tool
with workflow management, data profiling, data quality and metadata capabilities, rather than to hand-code the whole
ETL process.

4.4 Exchanging interoperable performance data

4.4.1 It has been stated before (see 4.3.5 and 4.3.8) that the existence of a common framework of dimensions is
the glue which makes the values of performance indicators compatible and comparable. Common dimensions make up
half of the definitions and standards which ensure global harmonization of performance data management. The other
half of these definitions and standards consists of a common definition of metrics and performance indicators.

4.4.2 First, a capability is needed to exchange metadata: the definitions of dimension object types, their
relationships (generic representation of aggregation hierarchies), code registries and name directories, the definitions of
metrics; and the definitions of fact tables. It is recommended that this metadata be also available in machine readable
form and not just in manuals or other textual specification documents.

4.4.3 Next there is a need to exchange the dimension data, including their associated code registries and name
directories. As mentioned in 4.3.8, some dimension data are already kept at a global level (see Location Indicators
(Doc 7910) and Aircraft Type Designators (Doc 8643)). Dimension data should be obtainable from a known location or
locations with mechanisms in place for ensuring consistent updates. To each fact data processing party, the availability
of current dimension data is a prerequisite for being able to process the fact data and calculate correct performance
indicators.
Part I — Global Performance
Appendix C. Data and information management I-App C-13

4.4.4 The exchange of performance data (metrics, performance indicators, performance targets) occurs via the
transmission of fact data, which are often exchanged in aggregated form. A suitable fact table should be defined for the
exchange, containing all metrics and dimensions relevant to the desired aggregation. If a dimension is not included, it is
to be assumed that the data are summarized over all possible values in that dimension. If this is not the case, a
corresponding dimension field should be included, even if it contains the same value for all fact records. This is
necessary to unambiguously identify the context of the exchanged performance data. If for example a set of traffic data
consists entirely of IFR flights, it would be bad practice to not include a “flight rule” dimension in the fact table and let the
receiver implicitly assume that the context is “IFR”.

4.4.5 As mentioned in 4.3.6 and 4.3.7, the role of data marts is to facilitate data dissemination and analysis. This
implies that data marts can also be used as a way to “package” data which needs to be exchanged. In such an
application, a data mart would be transmitted in its entirety to its recipient, rather than it being hosted on a database
server with the users consulting it.

5. PERFORMANCE EVALUATION

5.1 Performance evaluation (or review) consists of four separate activities:

— data access and publication;

— data analysis;

— formulation of conclusions; and

— formulation of recommendations.

5.2 Data access and publication

5.2.1 Once the required quality controlled data are available in the core data warehouse, they can be used for
performance evaluation, the first step of which is data access and publication. While doing this, care must be taken to
respect the data use/dissemination policies that were agreed upon with the original data suppliers.

5.2.2 In practice, ATM performance will be evaluated by two groups:

— performance specialists (e.g. analysts from designated ATM performance review organizations); and

— people with a general, high-level interest in ATM performance.

5.2.3 Each group has its own specific need for access to ATM performance data, which should be satisfied by
suitable data access and publication means.

5.2.4 People with a general interest in ATM performance will wish to see executive level, quality controlled data
and draw their own conclusions. From there comes the need to make certain performance data publicly available, in the
interest of transparency. A capability is therefore required which enables them to monitor the current situation against
the performance targets, and to provide them with the general trends, the big picture and their own performance in
comparison with others. This need can be satisfied by publishing high-level performance indicator dashboards. These
dashboards are periodically updated and generally allow limited or no interactivity (customization by the user).
I-App C-14 Manual on Global Performance of the Air Navigation System

5.2.5 In addition, analysts from designated ATM performance review organizations are tasked with gaining an
indepth understanding of ATM performance, and finding causes and effects. Their work is an integral part of the
performance management process described in Chapter 3. Their data needs can be satisfied by publishing selected
data from the core data warehouse in performance analysis data marts which are designed to suit the analysts’ needs.
These data marts should allow high levels of interactivity (querying and analysis).

5.3 Data analysis

5.3.1 This is a different kind of data analysis than the one needed for data quality checking (see 4.3.11). At this
stage, rather than struggling with data quality issues, analysts should be able to focus on their main task: performance
evaluation.

5.3.2 Analysts will need to develop a factual understanding of the reasons for (good/bad) performance, and
explain these to decision-makers. This includes gaining a better insight in past, current and future ATM performance.

5.3.3 To that effect, analysts will compare performance indicators against targets, identify performance evolution
trends, analyse historical evolution of performance, and find relationships (correlations) between performance indicators,
supporting metrics, etc. They will look at the “big picture” (annual totals and averages, performance indicators
summarized during the planning cycle) and break down the data into very detailed levels to find the causes of
performance gaps and the reasons for trade-offs. Analysts will also make use of various modelling techniques to
increase their understanding of system performance. See Appendix D, 2.

5.3.4 As a side-effect of data analysis, analysts will be able to propose performance objectives, define new
performance indicators and identify data needs.

5.4 Formulation of conclusions

After completing the data analysis, analysts are expected to document the gained insight by formulating conclusions for
each KPA. Normally, these conclusions contain an assessment as to the sufficiency of current and expected future
performance, for each performance objective. Alternatively, a conclusion could be that the available data are insufficient
for meaningful performance evaluation. Typically, the conclusions are published in a performance review report.

5.5 Formulation of recommendations

5.5.1 An integral part of the performance review process is the formulation of recommendations. These should
be derived from the conclusions, and also be included in the performance review report.

5.5.2 Recommendations should focus on how to meet ATM community expectations through agreed
performance objectives, performance indicators and performance targets. When evaluation indicates inconsistency
between ATM community expectations and performance objectives, and performance indicators and performance
targets, recommendations may include:

— the need to set or change performance objectives;

— the need to (re-)define performance indicators; and

— the need to set or change performance targets.

Part I — Global Performance
Appendix C. Data and information management I-App C-15

5.5.3 Recommendations will also fall more typically into the following categories (non-exhaustive list):

— Related to the need to improve performance data collection.

— Suggested initiatives aimed at closing identified performance gaps.

— Suggestions to accelerate or delay performance improvements, based on anticipated evolution of

traffic demand and predicted performance indicator trends.

— Recommendations of an organizational nature (set up a task force, define an action plan, etc.) with a
view of actually starting the implementation of the above recommendations.

_____________________
 Appendix D

PERFORMANCE ANALYSIS AND MODELLING

1. INTRODUCTION

The aim of this appendix is not to present a compact tutorial in data analysis and modelling, rather, the objective is to
provide some insight in the role of performance analysis and modelling within the performance-based approach. In
addition, this appendix briefly touches upon a number of subjects in an attempt to provide inexperienced readers with a
checklist of items to consider prior to launching a modelling exercise.

2. ROLE WITHIN THE PERFORMANCE MANAGEMENT PROCESS

2.1 Performance analysis

The aim of performance analysis is to gain a better understanding of the performance behaviour of the system being
examined. Two kinds of analysis can be distinguished:

— Quantitative analysis (see also Appendix C, 5.3). Analysts use (sometimes large quantities of)
available data to compare performance indicators against targets, identify performance evolution
trends, analyse historical evolution of performance, and find relationships (correlations) between
metrics, etc. with the aim of gaining better insight into past, current and future ATM performance. They
will look at the “big picture” (annual totals and averages, performance indicators summarized during
the planning cycle) and break down the data into very detailed levels to find the causes of
performance gaps and the reasons for trade-offs. A side effect of data analysis is that they can use the
results to formulate performance objectives, define new performance indicators and identify data
needs. Quantitative analysis plays a role in most performance management process steps:

• Step 1: Define/review scope, context and general ambitions/expectations (quantitative analysis

used to a limited extent).

• Step 2: Identify opportunities, issues and set (new) objectives.

• Step 3: Quantify objectives.

• Step 4: select solutions to exploit opportunities and resolve issues.

• Step 5: Implement solutions.

• Step 6: Assess achievement of objectives.

I-App D-1
I-App D-2 Manual on Global Performance of the Air Navigation System

— Qualitative analysis. When insufficient data are available, the initial steps of the performance
management process can be executed in a qualitative way based on stakeholder consultation, expert
knowledge and a good understanding of the system under consideration. This covers:

• Step 1: Define/review scope, context and general ambitions/expectations.

• Step 2: Identify opportunities, issues and set (new) objectives.

2.2 Performance modelling

Performance modelling is used for different purposes:

— As a communication tool to explain the performance behaviour to a variety of audiences using

diagrams (e.g. influence diagrams), graphs, charts, animations, etc.

— To increase understanding of and validate assumptions on performance behaviour (e.g. through

analytical models, fast-time simulations, real-time simulations).

— To produce forward-looking performance data (e.g. via the use of forecasting scenarios and models).

It plays a particular role in:

— Step 2: Identify opportunities, issues and set (new) objectives.

— Step 3: Quantify objectives.

— Step 4: Select solutions to exploit opportunities and resolve issues.

— Step 6: Assess achievement of objectives.

3. PRACTICAL GUIDANCE FOR PERFORMANCE MODELLING

3.1 General considerations

3.1.1 From the above, it follows that within the context of the performance-based approach, one should have a
particular interest in those models and modelling techniques which support an improved understanding of ATM
performance, and support the choice of ATM changes aimed at improving performance.

3.1.2 In that sense, the objective of modelling is not to explain how the air navigation system works in terms of
data flows, messages, etc., but to construct ATM performance models which help to — qualitatively and/or quantitatively
— understand the cause-effect relationships between performance variables, showing how individual performance
objectives can be achieved and how they interact (enhance or interfere) with each other. ATM performance modelling
may also help to understand complex interactions between stakeholders with differing objectives and strategies.

3.1.3 Because performance is measured in terms of metrics and indicators, it follows that the variables in
performance models should include all metrics from which the indicators are derived, as well as the indicators defined in
the performance framework.
Part I — Global Performance
Appendix D. Performance analysis and modelling I-App D-3

3.1.4 To illustrate this with an example from the environment area: models for performance planning should be
expressed in terms of traffic volume, fleet composition (noise characteristics, engine technology), fuel burn,
environmental rules and constraints (input metrics); noise and gaseous emissions in absolute terms (output metrics);
environmental efficiency (average noise and gaseous emission per flight, per km flown, per passenger-km etc.), level of
compliance with environmental rules and constraints (outcome metrics); local air quality, impact of aviation greenhouse
gas emissions on the global climate, number of people affected by noise (impact metrics).

3.1.5 Performance modelling also requires that the context of the metrics and indicators is clearly defined: this
comprises all relevant objects and events involved in air transport operations, as well as their interactions. This includes,
for example, physical objects such as geographical entities, airports, airspace, aircraft, but also the air transport
operation itself. The latter is described in terms of flights, trajectories and events such as departures, arrivals, airspace
crossings, incidents, accidents, etc. A formal definition of this context is a prerequisite to the formal definition of metrics
and indicators.

3.1.6 The modelling of the operational environment also defines the dimensions and aggregation hierarchies for
the metrics and indicators. This is necessary to discuss performance at a high level (e.g. aggregated to the level of a
planning region) based on performance models at a sufficiently granular level (which might need to be as detailed as the
individual flight or sector level). The subject of performance data dimensions is explained in Appendix C, 4.3.8.

3.1.7 It is not advisable to attempt to construct a single all-encompassing model for performance management
and planning. This would most likely not be workable. Rather, it is considered important (even essential) to base
individual models on common sub-models or input models, with the one providing the traffic forecast being the most
important. For example, if everyone uses their own (different) traffic forecasting models and scenarios to provide input
assumptions to their KPA-specific performance evaluation models, the resulting conclusions in the different KPAs are
guaranteed to be inconsistent.

3.1.8 This leads to the notion of a toolbox of models, of which a selection should be combined (modular coupling:
output metrics of one model are the inputs for another) to study a particular issue (benefit mechanisms contributing to a
given performance objective).

3.2 Suggested checklist for developing and using a performance model

3.2.1 Select audience and purpose of the model

First, decide which audience and which needs the model will serve, e.g.:

— Developed for or by a specialized (e.g. research and development) audience

• Seeking to validate assumptions on performance behaviour.

• Seeking to produce forward-looking performance data (e.g. production of traffic forecasts, or

assessing the estimated performance improvement resulting from an operational change).

— Developed for or by a general, high-level audience

• Seeking to understand validation results.

• Seeking to develop or acquire a general, high-level understanding of the performance behaviour

of the system.
I-App D-4 Manual on Global Performance of the Air Navigation System

3.2.2 Select the required scope and precision of the model

Clearly define and communicate the scope and precision of the model:

— Select the performance scope.

• Select which KPA (e.g. safety, security, capacity, etc.) the model is addressing. Within a KPA, you
may decide to restrict the scope of the model to a particular focus area or even a particular
performance objective.

• Select the measurement scope. Based on the scope in terms of performance objectives, select
the indicators to be modelled, then identify the supporting metrics which need to be included.
Supporting metrics include events to be counted and quantities to be measured.

— Select the system scope to be modelled: which geographic scope, which entity types (e.g. airports and
aircraft), and which interaction types (e.g. departures and arrivals).

— Select the process scope

• Are you aiming to model the performance of the planning process (model how strategic, pre-
tactical or tactical plans are produced and what their quality is) or to model the performance of the
execution of the plans (looking at the finally achieved performance)?

• Within the planning or execution process, are you seeking to model end-to-end performance or
specific capabilities such as problem detection, mitigation, or recovery (see Appendix A,
Table I-A-1 for a complete list of process capability areas)? Examples of specific capabilities
within the Safety KPA are: conflict detection, conflict resolution, and airborne and ground-based
safety nets.

• Are you intending to produce a “black box” model (which just relates input factors to output
performance), or to expose (to model) internal and background mechanisms in order to identify
performance issues and study options for improvement? When going into such detail, ensure that
your choice of scope covers all relevant issues (e.g. blocking factors) and (performance
improvement) opportunities. For example, when studying the possibilities for improving ATM
productivity, one may decide to not only model controller productivity, but also extend the scope
to include rostering and human resource management.

— Select the validity requirements: decide in advance for which metric ranges the model is expected to
be valid. There are “wide-range” models (suitable for representing system performance behaviour
under a wide range of input parameter values) and “narrow range” models (these are simplified
models which are only valid under specific, limited operating conditions). In particular, decide whether
the model is supposed to explain performance in unconstrained conditions only, constrained
conditions only, or needs to be valid for both unconstrained and constrained conditions. For example:
to what extent does your traffic forecasting model need to be designed to include airport capacity
constraints, and how accurately does the effect of these constraints need to be modelled. Examples of
metric ranges to be considered under this heading include:

• time horizon: short-/medium-/long-term performance modelling;

• season: summer/winter traffic conditions;

• day of week: weekday/weekend traffic conditions;

Part I — Global Performance
Appendix D. Performance analysis and modelling I-App D-5

• time of day: daytime/night-time traffic conditions;

• weather: which range of wind, visibility, temperature, precipitation conditions, etc.;

• aircraft mix: which traffic mix in terms of wake turbulence category, speed, climb capabilities,
navigation capabilities, etc.; and

• demand/capacity ratio: congested/uncongested conditions.

— Select the precision/accuracy requirements: decide whether the model needs to:

• explain performance in a qualitative way only (show factors playing a role in performance, and
show the existence of relationships between metrics);

• show quantitative results, based on a simplified model (not taking secondary effects into account)
(e.g. for strategic planning);

• sow quantitative results with high precision (e.g. for performance optimization);

• work with a single input scenario, or show results for several sets of input assumptions (e.g.
high/medium/low forecast);

• work with uncertainty or not. In the absence of uncertainty, models can work with single numerical
values of input and output metrics. On the other hand, with high levels of uncertainty on the input
metrics or with unpredictability of process performance, it may be necessary to use modelling
techniques which can work with probability distributions of variables.

3.2.3 Select the type of model

A suitable type of model should be selected to cater for the needs of the selected audience, scope, precision and
purpose (see 3.2.1 and 3.2.2). The following are suggested:

— Influence diagrams are used to depict a chain of (assumed or validated) cause-effect relationships.
They can be used in a simple way to depict a high-level, qualitative understanding of performance.
When such diagrams are used in a more complex way (supported by influence modelling tools), they
follow precise conventions which serve to document relationships between metrics (via processes and
mechanisms), and to link these to well-defined performance indicators. Influence diagrams are useful
as a tool to develop and document an initial understanding of the performance behaviour of the
system. They can also serve as a communication tool to explain performance behaviour to a non-
technical audience.

— Analytical models are used when influence diagrams need to be populated with numerical data. For
simple models they can be implemented using spreadsheets (not recommended for complex models).
Some influence modelling tools provide integrated support for numerical analysis (they allow influence
diagrams to be populated with data and to be “executed”, thereby delivering numerical results). These
models are typically used for “what-if “analyses (evaluation of different scenarios).

— Simulation models are used to model and evaluate complex system behaviour that cannot be
modelled using analytical models. A main characteristic of a simulation model is that it processes an
input scenario expressed as a sequence of events in chronological order. It produces large quantities
of data that must be summarized (using statistical techniques) to draw meaningful conclusions. A
I-App D-6 Manual on Global Performance of the Air Navigation System

distinction is made between fast-time and real-time simulations. The use of simulation models is
appropriate when there is a need to validate complex cause-effect relationships in influence diagrams.
Real-time simulations are often used for human participation in the simulations; this is useful when the
system performance is strongly affected by man-machine interactions.

3.2.4 Building the model

3.2.4.1 It is important to build the model in collaboration with the concerned stakeholders. While they may not be
modelling experts, they can provide an important contribution because they have the best knowledge of the mechanisms
and metrics which affect them. In addition, they need to understand the inner workings of the model so that there is a
shared belief that the modelling leads to credible results.

3.2.4.2 For these reasons, it is useful to complement numerical models (analytical models and simulation models)
with corresponding influence diagrams. These can then be used to communicate in simple terms what is modelled in the
numerical models, and how these work.

3.2.4.3 When building influence models, it is perfectly possible to start with pencil and paper, or any office software
which can produce graphs. However, when the size of the model(s) grows, and/or more people and perhaps different
teams are involved, it is advisable to start using a commercial influence modelling tool. This has quite a number of
advantages: it avoids problems which arise when participants have a different understanding or interpretation of the
methodology; the collection of models (and their interfaces) remains consistent and is easier to maintain; such tools
have built-in support for version management and collaborative model development; more can be achieved in less time
(modelling productivity is higher); and the models are more than just graphs (most tools can generate analytical models
from the graphs).

3.2.5 Select input data for the model

3.2.5.1 Whenever the intention is to use an analytical or simulation model, there will be the task of selecting input
data. These models need both historical input data (baseline values and in some cases historical time series data) as
well as forward-looking input data (forecasting scenarios consisting of projected time series data for a variety of input
metrics). A collection of input data which is sufficient to run the model is called a scenario. Care must be taken to ensure
that each scenario is consistent, i.e. that the values of the various metrics do not contradict each other. In forecasting,
this is called “defining a consistent future”. If you need to work with different sets of assumptions, you can define
different input scenarios. Part II of this document provides guidance for the development and application of these
scenarios at various stages of performance evaluation and analysis.

3.2.5.2 Selecting appropriate values can be quite challenging. The quality of the modelling results does not only
depend on the quality of the model, but also very much on the quality of the input data. Otherwise, it is “garbage in,
garbage out”.

3.2.5.3 Many of the modelling assumptions will have to be transformed from initial qualitative statements into
quantitative input data. During this task, many decisions have to be made. Therefore, it is important to do this in a
collaborative manner, to achieve consensus among the stakeholders.

3.2.5.4 Typical modelling activities where strong emphasis is placed on collaborative development of input
scenario data include traffic forecasting and cost-benefit modelling.
Part I — Global Performance
Appendix D. Performance analysis and modelling I-App D-7

3.2.6 Calibrate the model

Prior to using a data-oriented model, it should be calibrated to ensure that it will produce meaningful output. Many
models are parameterized (meaning that they include internal parameters such as sensitivity coefficients), and these
parameters need to be tuned. This is done by running the model with historical data, and observing whether the outputs
correspond to historical performance data measured in the real world. Parameter values are then adjusted until the
model delivers realistic results.

3.2.7 Using the model

3.2.7.1 This section briefly discusses how the various model types (influence diagrams, analytical models,
simulation models) can be used in the performance-based approach.

3.2.7.2 Part of the value of using influence diagrams comes from the experience of developing them
collaboratively with the stakeholders.

3.2.7.3 Once they are completed, they can be a powerful tool to facilitate discussions about where strengths,
weaknesses, opportunities and threats are and to explore where the focus of future performance management initiatives
should or could be (see performance management process Step 2: Identify opportunities, issues and set (new)
objectives).

3.2.7.4 When developed to a sufficient level of detail, they can also be used to show where the various candidate
solutions would have a performance effect, and how this effect propagates to produce an overall performance
improvement. Alternatively, such effects may also be negative, and then the influence diagrams can facilitate qualitative
reasoning about the need for trade-offs.

3.2.7.5 Because influence diagrams show how performance effects propagate, they can also be used to resolve or
avoid discussions about “double counting” performance benefits.

3.2.7.6 The other models (analytical models, simulation models) produce data which can be used to:

— feed other models (output of one model becomes (part of) the input scenario of another model). For
example, the output of traffic forecasting models is used as input to many other performance
assessment models;

— validate and quantify particular cause-effect relationships in influence diagrams;

— assess the anticipated performance effects of proposed ATM changes, prior to implementing them. In
essence, this means that the model outputs are expressed in terms of the metrics and indicators of the
performance framework, with subsequent comparison against targets, to see whether the performance
objective could be reached. When conducting this exercise for several alternative solutions, modelling
will provide tangible evidence to support the decision on what the preferred solution should be.

3.2.7.7 All of the above modelling applications play a role in the production of the “performance case” for a
proposed ATM change.

_____________________
 Appendix E

METRICS AND INDICATORS

This appendix was written in light of the desire to globally harmonize ATM performance indicators. However, it does not
propose such a globally standardized or harmonized set of metrics and indicators. It illustrates the challenges associated
with standardization by providing a number of examples from various organizations.

1. COMMONALITY IN ATM PERFORMANCE INDICATORS

1.1 A comparison of ATM performance indicators for two organizations was undertaken. Indicators were
investigated in the 11 KPAs identified in the Global Air Traffic Management Operational Concept (Doc 9854). The
objective was to determine if a set of indicators could be derived that were identical to provide an example set to be
used as an appendix in this manual.

1.2 The result of this investigation yielded no identical indicators. However, some commonality was identified
in certain indicators. For these, the steps required to reach consensus on a common set of indicators were identified. In
general, discrepancies stem from:

— definition of terms — the same terms are used but precise descriptions of what is being measured and
how are not explicitly described. Some indicators would likely require modelling or expert judgement
which would have to be described;

— filtering criteria — certain measurements are applied at specific locations (e.g. airports), times (e.g.
only during the day), under certain circumstances (e.g. only instrument meteorological conditions
(IMC)), or for specific events (e.g. ATM causes);

— normalization — indicators are normalized in a variety of manners (e.g. per flight, hour, passengers);

— statistical derivation — indicators can be averaged over various time horizons, or the 95th percentile
taken, etc.

1.3 The above discrepancies must be specifically described in order to reach comparable indicators.

1.4 For those KPAs where no commonality was found, a few examples are presented from yet a third source.

2. KPA 01 — ACCESS AND EQUITY

2.1 No common indicators were obtained.

2.2 Some stakeholders are attempting to use the following indicator: unsatisfied demand versus overall
demand (measured in volume of airspace time).

I-App E-1
I-App E-2 Manual on Global Performance of the Air Navigation System

3. KPA 02 — CAPACITY

3.1 Direct capacity measures fall into three types: system-wide, airspace and airport capacity.

3.2 System-wide

3.2.1 At the system-wide level, common indicators could not be found. For one organization, indicators focused
on the definition of capacity as the number of flights, flight hours and flight kilometres that can be accommodated. This
either requires a modelling approach, or subjective expert judgement. Consensus would require agreement on a
modelling approach. The expert judgement approach would require methods to increase objectivity.

3.2.2 Another approach to system-wide capacity indicators focused not on the number of flights, flight hours and
flight distance that can be accommodated, but instead measured the actual number that were produced. This approach
looks at the number of flights, available plane miles, etc. While these measures eliminate the need for models and
subjective assessments, they do not directly measure capacity, but rather supply. This approach combines the effect of
the marketplace with the available capacity.

3.3 Airspace

3.3.1 At the airspace level, some commonality is found between indicators. One organization seeks to determine
the number of IFR flights able to enter an airspace volume. Another is considering an indicator aggregating the agreed-
upon airspace capacity rates. These rates are the number of IFR flights able to be present in sectors at any one time.

3.3.2 The airspace measures are conceptually similar, with the caveat that disparity can exist on the methods for
determining the number of flights. In particular, the agreed-upon rates are a subjective evaluation used for traffic flow
management. As for the system-wide case, the ability to objectively determine the number of flights able to enter an
airspace volume is not a settled matter.

3.4 Airport

3.4.1 Methods for determining airport capacity are more accepted than airspace capacity. For this reason, basic
indicators for airport capacity are more likely to be easily harmonized.

3.4.2 Indicators are based upon the number of movements per unit of time that can be accepted during different
meteorological conditions. Differences occur in terms of how these are aggregated into indicators. Examples of
indicators are:

— hourly number of IFR movements (departures plus arrivals) possible during low visibility conditions
(IMC);

— daily number of IFR movements (departures plus arrivals) possible during a 15-hour day between
07 00 and 22 00 hours local time during low visibility (IMC) conditions;

— average daily airport capacity for a group of 35 airports measured as a five-year moving average; and

— average daily airport capacity for a group of seven major metropolitan areas.
Part I — Global Performance
Appendix E. Metrics and indicators I-App E-3

3.4.3 Achieving common airport capacity indicators must consider the following:

— the basic measure of airport capacity under specific conditions would need to be defined and methods
for its calculation agreed upon. Conditions can include those that actually occurred (fleet mix,
configuration, meteorological) and a forecast scenario or a standard scenario (e.g. typical IMC).
Methods for calculation can include the “called rate” when looking at actual occurrences, up to a
simulated IMC capacity when looking at standard scenarios;

— averaging methods — annual, hourly or daily — aggregation can easily be achieved if the fundamental
data is available; and

— aggregation into a single airport or groups of airports — these will always be specific to a region.

4. KPA 03 — COST EFFECTIVENESS

4.1 Common cost-effectiveness measures focus on the cost of ATM normalized per flight:

— average cost per flight at a system-wide annual level;

— total operating cost plus cost of capital divided by IFR flights; and

— total labour obligations to deliver one forecast IFR flight in the system, measured monthly and year-to-
date.

4.2 Even with these commonalities, agreement on a common metric would require standardization of the
following:

— components of cost to be included in the cost calculation — this should be based on ICAO’s policies
on cost recovery for air navigation services. ICAO’s Policies on Charges for Airports and Air
Navigation Services (Doc 9082), paragraph 44 i) states: “The cost to be shared is the full cost of
providing the air navigation services, including appropriate amounts for cost of capital and
depreciation of assets, as well as the costs of maintenance, operation, management and
administration”; and paragraph 44 v) states: “Air navigation services may produce sufficient revenues
to exceed all direct and indirect operating costs and so provide for a reasonable return on assets
(before tax and cost of capital) to contribute towards necessary capital improvements”.

— normalization — agreement on whether normalization should be conducted per flight hour or operation
is necessary. Organizations with longer duration flights, all other factors being equal, would likely have
higher costs per flight. Furthermore, the types of operations (e.g. IFR only) being considered for
normalization should be agreed upon;

— derivation — a minimum level of reporting time (monthly, annual) will help in comparison, although
comparison would likely be straightforward.

4.3 Individual organizations further break down normalized costs into finer and finer detail to understand
internal cost effectiveness.
I-App E-4 Manual on Global Performance of the Air Navigation System

5. KPA 04 — EFFICIENCY

5.1 The KPA Efficiency should comprise both focus areas “Temporal Efficiency” (i.e. delay) and “Flight
Efficiency” (trajectory oriented).

5.2 Common metrics of efficiency focus on delay performance, with some discrepancies in the definitions and
filtering criteria. For one organization, indicators include:

— per cent of flights departing on-time;

— average departure delay of delayed flights;

— per cent of flights with normal flight duration; and

— average flight duration extension of flights with an extended flight duration.

5.3 For another organization, indicators include:

— per cent of flights with on-time arrival at a predetermined set of airports; and

— total number of minutes to actual gate arrival time exceeding planned arrival time on a per flight basis
at the predetermined set of airports.

5.4 While the above seems to provide an easily reconciled approach to delay measurement, a significant
number of details and caveats are required to understand discrepancies.

— Certain types of delays are excluded based upon cause; this can be a subjective assessment.
Typically, only ATM-related delays are considered with exclusions due to weather. Other delay factors
that may be factored out include: system failures, carrier action, security delays, diverted flights and
arrival delay repercussion on departures.

— Delays are measured from an agreed-upon baseline time (of departure, flight, or arrival). While
scheduled times could be used, reported and proposed indicators focus on a baseline using times
obtained from a filed plan or a “shared business trajectory”. These seek to obtain the operator desired
times prior to the imposition of collaborative transparent forecasting method (TFM) initiatives.
However, as these are based on time forecasts, a TFM is required. This includes considerations of
such effects as winds aloft and taxi times (nominal or unimpeded).

— The threshold for a delayed flight varies by organization. A threshold of three minutes is applied by
one organization compared to a threshold of 15 minutes for another.

6. KPA 05 — ENVIRONMENT

6.1 Not all organizations report on both noise and emissions; therefore, common environmental metrics cannot
always be identified. Example indicators include:

— amount of emissions (CO2, NOx, H2O and particulate) which are attributable to inefficiencies in ATM
service provision;

— number of people exposed to significant noise as measured by a three-year moving average; and

— fuel efficiency per revenue plane-mile as measured by a three-year moving average.

Part I — Global Performance
Appendix E. Metrics and indicators I-App E-5

6.2 Agreement on the above metrics would necessitate common definitions and methods for:

— attribution of emissions to ATM service provision inefficiency, and methods to compile the emissions
inventory;

— definition of “significant” noise and methods for determining people exposed; and

— computation or measurement of fuel efficiency and agreement on normalization approach.

7. KPA 06 — FLEXIBILITY

7.1 No commonality in flexibility metrics could be identified.

7.2 Some stakeholders are attempting to use the following indicators:

— Number of rejected changes to the number of proposed changes (during any and all phases of flight)
to the number of flight plans initially filed each year.

— Proportion of rejected changes for which an alternative was offered and taken.

8. KPA 07 — GLOBAL INTEROPERABILITY

8.1 Commonality in the global interoperability KPA is focused on the area of level of compliance with
international Standards. One organization quantifies this as:

— the number of filed differences with ICAO Standards and Recommended Practices.

A second organization describes the following indicator:

— level of compliance of ATM operations with ICAO CNS/ATM plans and global interoperability
requirements.

8.2 Agreement on an indicator would require more specificity on the measurement technique for the level of
compliance and the specific definition of a global interoperability requirement.

9. KPA 08 — PARTICIPATION BY THE ATM COMMUNITY

9.1 No common indicators were obtained.

9.2 Some stakeholders are attempting to use the following indicators:

— number of yearly meetings covering planning, implementation and operation, and covering a
significant estimated proportion (e.g. 90 per cent) of the whole of the regional aviation activity;

— number of yearly meetings for planning;

— number of yearly meetings for implementation; and

— number of yearly meetings for operations.

I-App E-6 Manual on Global Performance of the Air Navigation System

10. KPA 09 — PREDICTABILITY

Some delay measures included in the efficiency KPA are considered to be measures of predictability by certain
organizations. For predictability indicators expressed through delay, the issues are the same as for those in the
efficiency KPA. The distinction between efficiency and predictability indicators stems from which baseline times are used.
In the case of efficiency, delay indicators include all ATM sources of delay. For predictability, only the components of
delay that are unknown by a certain event (e.g. at OUT time) are considered. In this latter case, agreement on the point
at which predictability is measured is important. No other commonalities in predictability measures could be identified.

11. KPA 10 — SAFETY

11.1 Within the safety KPA, common metrics focus on the number of accidents normalized through either the
number of operations or the total flight hours. Differences arise in the definition of the terms and the filtering criteria used
to include the data in the count. Examples of different filtering criteria are:

— Accidents are only counted for aircraft with a maximum take-off weight exceeding 2.25 tonnes,
operating under IFR. The term accident is narrowed to refer to “ATM-induced accidents”, and the
definition of accident is:

An occurrence associated with the operation of an aircraft which takes place between the time a
person boards the aircraft with the intention of flight until such time as all such persons have
disembarked, in which:

a) a person is fatally or seriously injured as a result of:

— being in the aircraft, or

— direct contact with any part of the aircraft, including parts which have become detached from
the aircraft, or

— direct exposure to jet blast;

except when the injuries are from natural causes, self-inflicted or inflicted by other persons, or when
the injuries are to stowaways hiding outside the areas normally available to the passengers and crew;
or

b) the aircraft sustains damage or structural failure which:

— adversely affects the structural strength, performance or flight characteristics of the aircraft,
and

— would normally require major repair or replacement of the affected component;

except for engine failure or damage, when the damage is limited to the engine, its cowlings or
accessories; or for damages limited to propellers, wing tips, antennas, tires, brakes, fairings, small
dents or puncture holes in the aircraft skin; or

c) the aircraft is missing or is completely inaccessible:

Note 1.— For statistical uniformity only, an injury resulting in death within thirty days of the date of
the accident is classified as a fatal injury by ICAO.
Part I — Global Performance
Appendix E. Metrics and indicators I-App E-7

Note 2.— An aircraft is considered to be missing when the official search has been terminated
and the wreckage has not been located.

— Only fatal accidents are considered in certain metrics.

— Specific measures consider fatal accidents for general aviation and “Part 135” operations. (This last
designation refers to operators governed by a specific part of the United States Code of Federal
Regulations (CFR).)

— Geographical filtering also occurs. Accident rates are monitored and targeted individually for specific
areas within the purview of an air navigation service provider.

11.2 In addition to differences in the filtering criteria, derived measures are obtained beyond an annual rate by
considering multi-year moving averages.

11.3 Achieving commonality in safety indicators requires agreement on definition of terms, filtering criteria,
statistical derivations and normalization. Cross-comparison can be achieved if indicators are provided at the finest level
of granularity without normalization. This allows the filtering, statistical derivation and normalization to be conducted
independently.

12. KPA 11 — SECURITY

No commonality in indicators identified. Some stakeholders are attempting to use the following indicators:

— number of acts of unlawful interference reported against air traffic service provider fixed infrastructure;

— number of incidents involving direct unlawful interference to aircraft (bomb threat, hijack, or imitative
deception) that required air traffic service provider response; and

— number of incidents due to unintentional factors, such as human error, natural disasters, etc. that have
led to an unacceptable reduction in air navigation system capacity.

______________________
 PART II

PERFORMANCE-BASED TRANSITION GUIDELINES

 FOREWORD

The main objectives of this part are threefold:

1— To raise awareness for the need to change how the evolution of air traffic management (ATM) is
planned at local, regional and global levels. In the past, planning was very much technology- and/or
solution-driven, with insufficient advance knowledge of resulting performance improvement. Today
and in the future, planning choices increasingly need to be justified in advance by a thorough
analysis of anticipated performance needs and achievements. Such an explicit management and
planning of ATM performance is needed to ensure that throughout the transition process, the
expectations of the ATM community are met.

2— To provide “getting started” guidance on how to adopt a performance-based approach in the

transition from today’s ATM system, towards the future ATM system, as described in the Global Air
Traffic Management Operational Concept (Doc 9854).

3— To promote a globally harmonized and agreed upon approach to transition planning in order for
regions and States to work collaboratively in developing their future transition arrangements towards
the ATM system envisioned in the global ATM operational concept.

The scope of this part is purposely limited to providing a broad overview of the tasks that need to be undertaken to adopt
a performance-based transition approach.

For complementary and more detailed information, the reader is referred to the following documents:

— Global Air Traffic Management Operational Concept (Doc 9854) — provides the overall vision;

— Manual on Air Traffic Management System Requirements (Doc 9882) — elaborates the overall vision
into material specifying the functional evolution of ATM; and

— Global Air Navigation Plan (Doc 9750) — focuses on implementation planning guidance.

_____________________

II-(i)
 TABLE OF CONTENTS

Page

Chapter 1. Introduction and overview ............................................................................................................. II-1-1

1.1 Scope and related documents ............................................................................................................. II-1-1

1.2 Context and applicability...................................................................................................................... II-1-1
1.3 Overview of the transition planning process ........................................................................................ II-1-2
1.4 Overview of the performance-based transition approach .................................................................... II-1-3
1.5 Glossary of terms ................................................................................................................................ II-1-7

Chapter 2. Measuring and assessing performance ........................................................................................ II-2-1

2.1 Introduction .......................................................................................................................................... II-2-1

2.2 Translate ATM community expectations into quantified performance targets ..................................... II-2-2
2.3 Measuring and assessing performance ............................................................................................... II-2-6
2.4 Identify and diagnose performance gaps............................................................................................. II-2-9

Chapter 3. Addressing performance gaps ...................................................................................................... II-3-1

3.1 Operational improvements .................................................................................................................. II-3-1

3.2 Building/updating your transition roadmap .......................................................................................... II-3-4
3.3 Choosing solutions to mitigate performance gaps ............................................................................... II-3-4

Chapter 4. Ensuring alignment throughout the planning process ................................................................ II-4-1

4.1 Planning at global, regional and local levels ........................................................................................ II-4-1

4.2 Need for negotiated alignment ............................................................................................................ II-4-1
4.3 Role of guidance material .................................................................................................................... II-4-3
4.4 Aligning performance measurement and assessment ......................................................................... II-4-3
4.5 Aligning the planning processes and their outputs .............................................................................. II-4-4
4.6 Improving the planning environment.................................................................................................... II-4-5

Appendix. Illustrations of operational improvements .................................................................................... II-App-1

1. Airborne spacing ................................................................................................................................. II-App-2

2. Airport and flight information sharing ................................................................................................... II-App-3
3. Decision support for sector operations ................................................................................................ II-App-4
4. Situational awareness for aerodrome ground operations .................................................................... II-App-5
5. Flexible airspace structures ................................................................................................................. II-App-6
6. Data link services ................................................................................................................................ II-App-7

Figures

Figure II-1-1 Distinction between implementation plan and transition roadmap ............................................. II-1-2
Figure II-1-2 Performance-based transition approach .................................................................................... II-1-3

II-(iii)
II-(iv) Manual on Global Performance of the Air Navigation System

Figure II-2-1 Mapping of ATM community expectations to performance targets ............................................ II-2-2
Figure II-2-2 What is a performance gap? ...................................................................................................... II-2-10
Figure II-2-3 Effect of more challenging performance targets......................................................................... II-2-11
Figure II-2-4 Effect of changing traffic evolution ............................................................................................. II-2-11
Figure II-2-5 Effect of delayed implementation ............................................................................................... II-2-12
Figure II-2-6 Effect of revised benefit estimates ............................................................................................. II-2-13
Figure II-3-1 Role of short, medium and long terms in a transition roadmap (example) ................................. II-3-8
Figure II-4-1 Role of global, regional and local levels ..................................................................................... II-4-2

_____________________
 Chapter 1

INTRODUCTION AND OVERVIEW

1.1 SCOPE AND RELATED DOCUMENTS

1.1.1 This document is dealing with a process and techniques for strategic (time horizon until twenty years
ahead) performance management and transition roadmap development, in the context of regional ATM planning. The
process produces important inputs to regional and local research and implementation planning (time horizon typically
five years).

1.2 CONTEXT AND APPLICABILITY

1.2.1 The Global ATM Operational Concept was developed in order to achieve an interoperable, global air traffic
management system for all users during all phases of flight that meets agreed levels of safety, provides for optimum
economic operations, is environmentally sustainable and meets national security requirements. The operational concept
describes an ATM system in 2025 based on the provision of services and driven by the need to meet the expectations of
the ATM community. The transition to the operational concept is to occur in a focused way via a set of coordinated
planning processes which operate at local, regional and global level.

1.2.2 In terms of the level of detail, these planning processes produce three kinds of outputs which will be
regularly updated according to the need (see 1.4):

— Transition roadmaps are a high-level representation of the selection of operational improvements

and their deployment interdependencies (in terms of prerequisites), adapted to the needs of a
particular planning area (at regional or local level).

— Implementation plans are intended to be derived from the short-term part of the transition roadmaps.
They lay out a detailed set of development and deployment actions — including their timing — for all
involved members of the ATM community.

— Research plans lay out the research needed today to develop the medium- and long-term parts of
transition roadmaps to a level of maturity suitable for turning them into implementation plans.

1.2.3 Transition roadmaps are considered to be at a more strategic level than implementation plans, not only
because they contain less detail, more uncertainty and provide guidance for the development of implementation plans,
but also because they usually cover a longer time horizon. This is illustrated in Figure II-1-1.

1.2.4 The performance-based approach applies to the development of transition roadmaps as well as
implementation plans. However, in this document the focus is on showing how to include the performance-based
approach in the development of transition roadmaps.

II-1-1
II-1-2 Manual on Global Performance of the Air Navigation System

2005 2010 2015 2020

IMPLEMENTATION PLAN
EDITION 2005

Development De
Deployment cre
TR as
AN ing
SIT lev
I el
RESEARCH PLAN Short term ON of
de
RO tai
EDITION 2005 AD la
MA nd
PE ma
DIT tur
Research Development Deployment IO ity
N2
00
5
Medium term

Research Research Development Deployment

Long term

Figure II-1-1. Distinction between implementation plan and transition roadmap

1.3 OVERVIEW OF THE TRANSITION PLANNING PROCESS

1.3.1 Transition planning is a cyclic process executed at global, regional and local levels. It is typically repeated
at five-year intervals to take into account changing forecasts, up-to-date information on implementation progress, new
performance assessments, changed performance expectations and policies (resulting in revised performance targets),
and any other relevant change. (Exceptional changes may lead the ATM community to decide to start a new transition
planning cycle before the regular five-year interval has elapsed.)

1.3.2 Each iteration results in updated versions of transition roadmaps, research plans and implementation plans
(the latter two may be updated on an annual basis). Considering that transition roadmaps typically cover a time horizon
of twenty years, a given five-year period will be updated several times in the transition roadmap before it is incorporated
into implementation plans and finally materializes as a set of real changes to the ATM system.

1.3.3 The sequence of steps outlined in 1.4.3 takes the results (transition roadmap, implementation plans and
performance assessments) of the previous iteration into account, and applies all the latest changes to see whether the
transition roadmaps and implementation plans are still valid from a performance perspective. If this is not the case,
transition roadmaps and implementation plans are modified to mitigate all identified performance issues (performance
gaps) and ensure alignment at global, regional and local levels (see Chapter 4), and the process is ready to start again
five years later.
Part II — Performance-based Transition Guidelines
Chapter 1. Introduction and overview II-1-3

1.4 OVERVIEW OF THE PERFORMANCE-BASED TRANSITION APPROACH

1.4.1 Figure II-1-2 provides an overview of the performance-based transition approach. It contains five steps with
questions, which must be answered as part of applying the approach:

— Step 1 (questions 1 – 5): translate ATM community expectations into quantified performance targets

— Step 2 (questions 6 – 8): conduct performance assessment and use performance targets to identify
current and anticipated performance gaps

— Step 3 (questions 9 – 12): update transition roadmaps and plans to mitigate identified performance
gaps

— Step 4 (not shown in the figure): analyse Steps 1–3 and generate lessons learned

— Step 5 (not shown in the figure): maintain the guidance material and the overall planning process

STEP 1
1 2 3 5

What are the What are the

How to measure What are the performance
ATM community performance
performance? targets?
expectations? objectives?
44

What is the current traffic

and the expected traffic
evolution?

STEP 2
6 7 8

What is the current and What are the current and

What are the current
planned anticipated performance gaps
roadmaps and plans?
ATM performance? and their reasons?

10
STEP 3
9 11 12

What is the Global ATM What selection and sequence

What are the available What are the resulting
Operational Concept and of operational improvements
options for operational updates to current
associated system address current and
improvement? roadmaps and plans?
requirements? anticipated performance gaps?

Figure II-1-2. Performance-based transition approach

II-1-4 Manual on Global Performance of the Air Navigation System

1.4.2 The remainder of this section gives a brief overview of the role of each step in the approach. Subsequent
chapters provide more detail:

— Chapter 2 (Measuring and Assessing Performance) addresses Steps 1 and 2;

— Chapter 3 (Addressing Performance Gaps) deals with Step 3;

— Chapter 4 (Ensuring Alignment Throughout the Planning Process) address Steps 4 and 5, as well as
collaboration issues which arise due to the distributed nature of planning and the layered organization
of the process (at global, regional and local levels. The underlying philosophy can be described as
“Think global, act local”.

1.4.3 What are the ATM community expectations?

The ATM community expectations are a set of general, high level expectations listed in the Global Air Traffic
Management Operational Concept (Doc 9854). These are (in alphabetical order): Access and equity, Capacity, Cost
effectiveness, Efficiency, Environment, Flexibility, Global interoperability, Participation by the ATM community,
Predictability, Safety, and Security. These expectations are used in performance management as the framework for key
performance areas (KPAs). (See Chapter 2, 2.2.2.)

1.4.4 What are the performance objectives?

1.4.4.1 The ATM community expectation embodied by each KPA will be met by pursuing more specific
performance objectives. These are defined to assist the ATM community in producing relevant and timely enhancements
(operational improvements) to a given region’s ATM system, in order to satisfy the ATM community expectations.

1.4.4.2 Performance objectives are expressed in qualitative terms and may include a desired or required trend for
a performance indicator (e.g. reduce the cost per kilometre flown) while not yet expressing the performance objective in
numeric terms (this is done as part of a performance target setting). Care must be taken to ensure that the agreed
performance objectives are “SMART” — (specific, measurable, achievable, relevant and timely). (See Chapter 2, 2.2.3.)

1.4.5 How to measure performance?

1.4.5.1 In order to be able to measure performance, a number of definitions, methods and facilities must be
implemented:

— For each performance objective, performance indicators need to be defined to measure the
achievement of the performance objective. For example, the performance objective of “reducing cost
per kilometre flown” requires the cost per kilometre flown performance indicator. Performance
indicators should be chosen to convey meaningful information about ATM performance for a given
performance objective, and be suitable to generate improvement and change.

— In addition, a set of supporting metrics must be defined. In the prior example, total cost and total
distance flown are required metrics to compute the performance indicator. Supporting metrics
determine which data need to be collected to calculate values for the performance indicators.

— Common definitions need to be agreed upon for geographical areas, time periods and other
categories for which data are collected and published. This is essential for the compatibility of data
Part II — Performance-based Transition Guidelines
Chapter 1. Introduction and overview II-1-5

and determines how performance data can be aggregated (e.g. geographically from local to regional
and – as required – to the global level).

— Harmonized methods and facilities need to be established for collecting, cleaning, storing, analysing
and disseminating performance data.

1.4.5.2 Existing performance objectives and performance indicators in the area of environment can be found in
Assembly Resolutions in Force (as of 8 October 2004) (Doc 9848), Resolution A35-5, Appendix A. Specific goals are set
to limit or reduce:

— the number of people affected by significant aircraft noise;

— the impact of aviation emissions on local air quality; and

— the impact of aviation greenhouse gas emissions on the global climate. (See Chapter 2, 2.2.4.)

1.4.6 What is the current traffic and the

expected traffic evolution?

Traffic (demand) patterns, volumes, performance envelopes and equipage change over time. Demand evolution
forecasts need to be produced on the basis of a number of forecasting scenarios (i.e. sets of assumptions about the
future), to quantify how the need for performance evolves through time and varies by geographical area and other
categorization criteria. As part of this task, data on current and past traffic patterns and volumes need to be collected.
(See Chapter 2, 2.2.5.)

1.4.7 What are the performance targets?

1.4.7.1 The above-mentioned performance indicators are the agreed upon way for quantifying how well
performance objectives have been achieved.

1.4.7.2 Performance targets are closely associated with performance indicators: they represent the values of
performance indicators that need to be reached or exceeded to consider a performance objective as being fully achieved.
Note that performance targets can be set as a function of time (e.g. to plan year-on-year improvement); they can also
vary by geographic area.

1.4.7.3 A decision-making/policy-making process needs to be in place to collaboratively agree on performance

objectives, performance indicators and the values of performance targets at the local, regional and — where required —
global levels. (See Chapter 2, 2.2.6.)

1.4.8 What are the current transition roadmaps and plans?

Current transition roadmaps and implementation plans need to be known and taken into account in the performance-
based transition approach. They are the basis for determining the currently envisaged performance improvement. (See
Chapter 2, 2.3.3.)
II-1-6 Manual on Global Performance of the Air Navigation System

1.4.9 What is the current and anticipated ATM performance?

1.4.9.1 Knowledge about the current ATM system and the current traffic levels is used to determine current
performance levels, which are to be expressed in terms of the performance indicators associated with the various
performance objectives.

1.4.9.2 In addition, existing knowledge — obtained from past validation activities — needs to be combined with the
traffic forecast to assess future (anticipated) ATM performance. (See Chapter 2, 2.3.2; in particular 2.3.5.)

1.4.10 What are the current and anticipated performance gaps and their reasons?

1.4.10.1 Paragraphs 1.4.8 and 1.4.9 provide current and anticipated values for each of the performance indicators,
under certain assumptions (forecasting scenarios and existing transition roadmaps and plans). These are compared
against the latest version of the agreed upon performance targets, resulting in the identification of current and
anticipated performance gaps.

1.4.10.2 The underlying reasons for the gaps should be identified:

— Have the performance targets been changed?

• Has there been a change of policy (decision to set more challenging performance targets)?

• Is the performance gap caused by a change in the expected traffic evolution (more challenging)?

— During the previous planning cycle, were current and anticipated performance levels below the target
levels?

• Has there been a problem with implementation since the previous planning cycle (e.g.
implementation delays)?

• Have the performance improvement estimates associated with transition roadmaps and plans
been revised downwards since the previous planning cycle?

(See Chapter 2, 2.4.)

1.4.11 What is the global ATM operational concept and associated system requirements?

1.4.11.1 The ICAO vision of an integrated, harmonized and globally interoperable ATM system for a planning
horizon up to and beyond 2025 can be found in the Global Air Traffic Management Operational Concept (Doc 9854).
The purpose of the operational concept document is to set a common goal.

1.4.11.2 This vision, and its envisaged performance characteristics, is further detailed in a set of system
requirements which can be found in the document entitled Manual on Air Traffic Management (ATM) System
Requirements (Doc 9882) (awaiting publication).

1.4.11.3 Together, these documents form the guidance material for determining the available options for operational
improvement (see 1.4.12.1 and 1.4.12.2). (See also Chapter 3, 3.1.5.)
Part II — Performance-based Transition Guidelines
Chapter 1. Introduction and overview II-1-7

1.4.12 What are the available options for operational improvement?

1.4.12.1 An operational improvement is a transition step in a transition roadmap. Operational improvements are
changes to the ATM system that are on the transition path towards the global ATM operational concept and result in a
direct performance enhancement. Because of their need to deliver performance enhancement, the elementary changes
that make up an operational improvement are intended to be implemented simultaneously.

1.4.12.2 A common list of possible operational improvements needs to be compiled to deliver the list of options from
which to develop transition roadmaps adapted to the specific needs of each region. The Global Air Navigation Plan
(Doc 9750) is one of the sources for developing the list of candidate operational improvements for a region. Use can
also be made of work already undertaken in other planning regions. A few illustrations are given in the Appendix to this
Part. (See also Chapter 3, 3.1.)

1.4.13 What selection and sequence of operational improvements addresses

current and anticipated performance gaps?

1.4.13.1 The answer to this question will be provided by a new version of the transition roadmap for a particular
region.

1.4.13.2 During this stage, the ATM community needs to modify the old version of the transition roadmap and
reassess the performance impact of the changes. The resulting new transition roadmap is acceptable if the ATM
community believes it has a realistic potential for mitigating the performance gaps. Such confidence will be underpinned
by a performance case, which is to document all the reasoning and arguments used to demonstrate that the
performance objectives (and performance targets) will be met. (See Chapter 3, 3.2.)

1.4.14 What are the resulting updates to current transition roadmaps and plans?

Any change to a transition roadmap may have implications for other transition roadmaps (neighbouring regions or at a
different planning level), and/or require a (partial) revision of existing research and implementation plans. This needs to
be addressed at this stage in the performance-based transition approach. (See Chapter 3, 3.3.3.)

1.5 GLOSSARY OF TERMS

The following terms are used in this document with a specific meaning, which is explained below:

Air traffic management (ATM). The dynamic, integrated management of air traffic and airspace — safely, economically
and efficiently — through the provision of facilities and seamless services in collaboration with all parties
(Doc 9854, 1.1 and Appendix B refer).

ATM community. The aggregate of organizations, agencies or entities that may participate, collaborate and cooperate
in the planning, development, use, regulation, operation and maintenance of the ATM system. (Doc 9854,
Appendix A, lists and describes the various members comprising the ATM community, e.g. States, ATM service
providers, airspace users and the ATM support industry.)

ATM community expectation. What members of the ATM community expect from ATM, in high-level qualitative
performance terms. These expectations are listed in Doc 9854, Appendix D and have been assigned to Key
Performance Areas (KPAs). (See 1.4.3 and Chapter 2, 2.2.2.)
II-1-8 Manual on Global Performance of the Air Navigation System

Air traffic management system. A system that provides ATM through the collaborative integration of people,
information, technology, facilities and services, supported by air- and ground- and/or space-based
communications, navigation and surveillance (Doc 9854, Appendix B refers).

Baseline performance. The performance of the baseline system during the baseline period. (See 1.4.9 and Chapter 2,
2.3.)

Baseline period. An initial (one year) period, for which performance data are available, which can be used as a
reference.

Baseline system. The ATM system (including a given traffic pattern and volume) as it was during the baseline period.
The baseline system is used as the reference against which to compare operational improvements and
performance enhancements. (See Chapter 2, 2.3.)

Deployment. The phase of an operational improvement or enabler during which it enters into service (potentially in a
progressive way) and starts delivering benefits (Figures II-1-1 and Chapter 3, Figure II-3-1 refer).

Development. The life cycle phase of an operational improvement or enabler during which it is transformed from
research results into ATM changes which are ready for deployment (Figures II-1-1 and Chapter 3, Figure II-3-1
refer).

Enablers. Initiatives, such as (new) technologies, systems, operational procedures, and operational or socio-economic
developments, which facilitate the implementation of operational improvements or of other enablers (Chapter 3,
3.1.4 and Doc 9854, Appendix B refer).

Global ATM operational concept. The global air traffic management (ATM) operational concept presents the ICAO
vision of an integrated, harmonized and globally interoperable ATM system. The planning horizon is up to and
beyond 2025 (Doc 9854, 1.1 refers).

Global level. The highest of the three planning levels. The global level is responsible for looking after the network effect
of regional planning activities (Chapter 4, 4.1 refers).

Implementation. The combination of development and deployment of an operational improvement or enabler

(Figure II-1-1 and Chapter 3, Figure II-3-1 refer).

Implementation plan. An implementation plan has a typical time horizon of five years. It is derived from the early (short-
term) parts of a transition roadmap. It lays out a detailed set of implementation focused actions — including
their timing — for all involved members of the ATM community (1.2, Figure II-1-1, and Chapter 3, 3.2.2 refer).

Implementation planning. The process of developing and updating implementation plans. The process is usually
repeated on an annual basis (1.3 refers).

Key performance area (KPA). The ATM community expectations fall into eleven categories, called key performance
areas (KPAs). These are (in alphabetical order): access and equity, capacity, cost effectiveness, efficiency,
environment, flexibility, global interoperability, participation by the ATM community, predictability, safety, and
security (1.4.3 and Chapter 2, 2.2.2 refer).

Life cycle phase. The life cycle of an operational improvement includes phases such as research, development, and
deployment (see also the term “Implementation” as well as Figure II-1-1 and Chapter 3, Figure II-3-1).

Local level. The lowest of the three planning levels. The local level corresponds to planning activities of individual
members of the ATM community (Chapter 4, 4.1 refers).
Part II — Performance-based Transition Guidelines
Chapter 1. Introduction and overview II-1-9

Long term. The third transition phase of a transition roadmap. While not precisely delineated along the time axis, it
typically covers a period from ten to twenty years into the future (Chapter 3, 3.3.4.2 refers).

Medium term. The second transition phase of a transition roadmap. While not precisely delineated along the time axis,
it typically covers a period from five to ten years into the future (Chapter 3, 3.3.4.2 refers).

Operational improvement (OI). Operational improvements are changes to the ATM system that are on the transition
path towards the global ATM operational concept and result in a direct performance enhancement. An
operational improvement is a set of elementary changes which are intended to be implemented together to
deliver performance. An operational improvement is a transition step in a transition roadmap (1.4.12 and
Chapter 3, 3.1 refer).

Performance. ATM performance is a measure of how well the ATM system satisfies the ATM community expectations.
In each of the KPAs, performance is measured at the level of individual performance objectives, using
performance indicators (Chapter 2 refers).

Performance assessment. The assessment of past, current and/or planned performance. The process of assessing
past and current performance is called performance review. Planned performance is assessed during the
research and development phases of the life cycle, using validation techniques (1.4.9, Chapter 2, 2.3 and 2.3.2
refer).

Performance-based transition approach. A method for transition planning in which planning choices are justified in
advance by a thorough analysis of anticipated performance needs and achievements.

Performance case. The documentation that contains all the reasoning and arguments used to demonstrate that the
performance objectives (and performance targets) will be met. A performance case can be seen as the
combination of the various cases that together address and balance all areas in which the ATM community has
expectations, e.g. the safety case, together with the business case, together with the environment case (1.4.13,
Chapter 2, 2.1 and 2.3.4 refer).

Performance gap. The shortfall between a performance indicator value and its performance target is referred to as a
performance gap for a particular performance objective. The existence of (anticipated) performance gaps is the
trigger for introducing additional operational improvements via the modification of current transition roadmaps
and plans (1.4.10 and Chapter 2, 2.4 refer).

Performance indicator. Performance indicators are defined in order to quantify the degree to which performance
objectives are being, and should be, met. When describing performance indicators, one must define what and
how measurements will be obtained (through supporting metrics) and combined to produce the indicator (1.4.5
and Chapter 2, 2.2.4 refer).

Performance management. The process of defining performance objectives, performance indicators and performance
targets. In addition, it includes performance monitoring and the identification of performance gaps (1.1 and
Chapter 2 refer).

Performance monitoring. The process of collecting performance data, as required, for calculating the values of
performance indicators. The aim is to monitor how well performance objectives are met (Chapter 2, 2.2.3
refers).

Performance objective. The ATM community expectation embodied by each KPA will be met by pursuing more specific
performance objectives. Performance objectives are expressed in qualitative terms, and include a desired or
II-1-10 Manual on Global Performance of the Air Navigation System

required trend for a performance indicator (e.g. reduce the cost per kilometre flown) while not yet expressing
the performance objective in numeric terms (this is done as part of performance target setting). Care must be
taken to ensure that the agreed upon performance objectives are “SMART” — (specific, measurable,
achievable, relevant and timely) (1.4.4 and Chapter 2, 2.2.3 refer).

Performance review. The assessment of past and current performance using measured data obtained via performance
monitoring (Chapter 2, 2.4.3 refers).

Performance target. Performance targets are the values set on performance indicators that need to be reached or
exceeded to consider a performance objective as being fully achieved. Note that performance targets can be
set as a function of time (e.g. to plan year-on-year improvement); they can also vary by geographic area (1.4.6
and Chapter 2, 2.2.6 refer).

Planned performance. The future performance associated with a transition roadmap. Planned performance is
assessed during the research and development phases of the life cycle, using validation techniques (see also
“Performance assessment”, as well as 1.4.9 and Chapter 2, 2.3.2).

Planning cycle. The interval at which the transition planning process is repeated to take into account changing
forecasts, up-to-date information on implementation progress, new performance assessment, changed
performance expectations and policies (resulting in revised performance targets), and any other relevant
change. Typically, five years for transition roadmaps and one year for research and implementation plans (1.3
and Chapter 4, 4.5 refer).

Planning level. Transition planning takes place at different levels called planning levels. These are the global level,
regional level and local level (Chapter 4, 4.1 refers).

Regional level. The intermediate of the three planning levels. At the regional level, ATM community members have
agreed to develop the ATM system in accordance with a common transition plan. Operating environments and
priorities may be different. Regional performance targets will be defined. The regional level is defined by a
division of the world into homogeneous regions and major traffic flows with similar characteristics, and a
common ATM interest in terms of performance and transition planning. The regional level is responsible for the
network effect of local planning activities (Chapter 4, 4.1 refers).

Research. The life cycle phase of an operational improvement during which it is progressively transformed from a
concept into a refined and validated ATM change which is ready for development (and can be included in the
implementation planning) (Figure II-1-1 and Chapter 3, Figure II-3-1 refer).

Research plan. A research plan has a time horizon of typically five years. It is derived from the medium-term and long-
term parts of a transition roadmap. It lays out a detailed set of research focused actions — including their timing
— for all involved members of the ATM community (1.2, Figure II-1-1 and Chapter 3, Figure II-3-1 refer).

Research planning. The process of developing and updating research plans. The process is usually repeated on an
annual basis.

Short term. The first transition phase of a transition roadmap. It covers the period for which implementation plans have
been established, which typically corresponds to a five-year time horizon (Chapter 3, 3.3.4 refers).

Supporting metric. Supporting metrics determine which data need to be collected to calculate values for the
performance indicators (1.4.5 and Chapter 2, 2.2.4 refer).
Part II — Performance-based Transition Guidelines
Chapter 1. Introduction and overview II-1-11

Traffic pattern. The distribution of the total annual traffic volume in geographic terms (airport-pair flows and route-
dependent overflights), time distribution (seasonal, weekly, daily, hourly fluctuations) and aircraft types (1.4.6
and Chapter 2, 2.2.5 refer).

Traffic volume. The amount of air traffic, usually expressed in terms of number of flights or movements, but sometimes
also in terms of distance flown or controlled, or flight hours flown or controlled (1.4.6 and Chapter 2, 2.2.5 refer).

Transition. The sequence of deployments that constitute the transition from the current ATM system to the future ATM
system envisaged in the global ATM operational concept. The transition is documented in the transition
roadmap (1.2 refers).

Transition phase. The transition roadmap is typically divided into three transition phases: short term, medium term and
long term (Chapter 3, 3.3.4 refers).

Transition plan. A set of aligned, consistent plans and roadmaps, consisting of: a transition roadmap, the research plan
and the implementation plan (1.3 and Chapter 4, 4.5 refer).

Transition planning. The process of developing and updating the transition plan. It consists of transition roadmap
development, research planning and implementation planning (1.4 and Chapter 4, 4.5 refer).

Transition planning step. The performance-based transition approach defines five steps which must be executed
during a single iteration of the transition planning process. These steps are: (1) translate ATM community
expectations into quantified performance targets; (2) conduct performance assessment and use performance
targets to identified current and anticipated performance gaps; (3) update transition roadmaps and plans to
mitigate identified performance gaps; (4) analyse Steps 1-3 to generate lessons learned; and (5) maintain the
guidance material and the overall planning process itself (1.4 and Chapter 4, 4.5 refer).

Transition roadmap. The transition roadmap covers a twenty-year rolling time period. It contains a sequence of
operational improvement deployments, which is suitable for transitioning the current ATM system to the future
ATM system envisaged in the global ATM operational concept, while meeting the performance requirements as
documented by the performance objectives and performance targets (1.2, 1.4.13 and Chapter 3, 3.2.1 refer).

Validation. The process of determining that an operational improvement or enabler (and by extension, the entire
transition roadmap) functions as intended, and of developing the performance case which provides sufficient
confidence that the operational improvement/enabler/transition roadmap will be able to meet the performance
requirements as documented by the performance objectives and performance targets (1.4.9, Chapter 2, 2.3.2,
2.4.3, Chapter 3, 3.1.3 and 3.3.2 refer).

_____________________
 Chapter 2

MEASURING AND ASSESSING PERFORMANCE

2.1 INTRODUCTION

2.1.1 The performance-based approach rests on the basic principle that the ATM community expectations can
best be satisfied by quantifying these expectations into a set of agreed performance targets (which are periodically
adjusted), then using these performance targets as the benchmark to introduce ATM performance improvements in a
controlled way, and finally, using the performance targets to justify these improvements (including them in system-wide
safety, business and environment cases, or — more generally — a performance case).

2.1.2 For this approach to succeed, the performance-based ATM system requires performance measurement
and assessment. While the process is iterative, several steps can be isolated (Steps 1 and 2, as defined in Chapter 1,
1.4; numbers in parenthesis refer to boxes in Chapter 1, Figure II-1-2).

— Step 1: Translate ATM community expectations into quantified performance targets

— (1) Identification of ATM community expectations and corresponding key performance

areas (KPA) — Starting with the set of general expectations listed in Doc 9854, key areas of
performance have been identified which serve as the general framework for classifying
performance needs and improvements. All planners are expected to use this standardized set of
KPAs.

— (2) Reaching agreement on performance objectives — The ATM community expectation

embodied by each KPA is to be met by pursuing a choice of more specific performance objectives
in each planning region, which are adapted to the challenges facing the region. The selected
performance objectives will serve as the regional and local drivers for performance improvement.

— (3) Reaching agreement on performance measurement methods — Quantitative

performance indicators must be defined for each performance objective, together with a
description of the supporting metrics, guidance on how to collect the data, and the computation
required for obtaining the performance indicators.

— (4) Reaching agreement on the expected traffic evolution — For certain performance
objectives (e.g. in the capacity area), the performance targets are dependent on the
traffic/demand forecast.

— (5) Setting performance targets — A performance-based ATM system seeks to attain

performance objectives by setting specific performance targets on the performance indicators.
These performance targets must be selected such that reaching a performance target
corresponds to meeting a performance objective. Performance targets must also be validated to
ensure that they are feasible and not arbitrary.

II-2-1
II-2-2 Manual on Global Performance of the Air Navigation System

A decision-making/policy-making process needs to be in place to collaboratively agree on performance

objectives, performance indicators and the values of performance targets at the local, regional and —
where required — the global levels.

— Step 2: Conduct performance assessments and use performance targets to identify current and
anticipated performance gaps

— (7) Measurement of performance — Measurement of the actual performance of the ATM

system must be undertaken. This measurement establishes the level of performance of the
system as measured by the performance indicators previously established.

— (6–7) Prediction of planned ATM system performance — Anticipated changes in traffic levels,
fleet equipment, procedures, and infrastructure modernization can all lead to variation in
performance over time. Estimates of the future evolution of performance indicators must be
undertaken to pre-emptively determine if additional ATM system improvements are required to
meet the established performance targets.

— (8) Identification of current and anticipated performance gaps — The shortfall between a
performance indicator and its performance target is referred to as a performance gap for a
particular performance objective. The existence of (anticipated) performance gaps is the trigger
for introducing additional operational improvements via the modification of current transition
roadmaps and plans.

2.2 TRANSLATE ATM COMMUNITY EXPECTATIONS INTO

QUANTIFIED PERFORMANCE TARGETS

2.2.1 The process of translating ATM community expectations into quantified performance targets is illustrated in
Figure II-2-1. Key performance areas (2.2.2 refers) are mapped into performance objectives; the expression of those
performance objectives is mapped into quantitative performance indicators which have set performance targets.

Expectations

KPA

Objectives

Indicators Quantitative

Targets ...
Objective is met when indicators meet or exceed targets

Figure II-2-1. Mapping of ATM community expectations to performance targets

Part II — Performance-based Transition Guidelines
Chapter 2. Measuring and assessing performance II-2-3

2.2.2 What are the ATM community expectations and the key performance areas (KPAs)?
[see Chapter 1, Figure II-1-2, Step 1, (1)]

The ATM system seeks to meet diverse expectations in terms of service delivery. These expectations are detailed in
Appendix D of Doc 9854 and constitute the starting point for ATM performance objectives. For performance
management purposes, it is considered that each of these expectations corresponds to a single KPA as shown below:

— KPA 01 – Access and equity

— KPA 02 – Capacity

— KPA 03 – Cost effectiveness

— KPA 04 – Efficiency

— KPA 05 – Environment

— KPA 06 – Flexibility

— KPA 07 – Global interoperability

— KPA 08 – Participation by the ATM community

— KPA 09 – Predictability

— KPA 10 – Safety

— KPA 11 – Security

These KPAs serve as the general framework for classifying performance needs and improvements. All planners are
expected to use this standardized set of KPAs.

2.2.3 How to determine performance objectives?

[See Chapter 1, Figure II-1-2, Step 1, (2)]

2.2.3.1 The ATM community expectation embodied by each KPA is to be met by pursuing a choice of more
specific performance objectives in each planning region, which are adapted to the challenges facing the region. The
selected performance objectives will serve as the regional and local drivers for performance improvement.

2.2.3.2 Performance objectives are precisely scoped (i.e. which part of the ATM system is the performance
objective for), express performance in terms of specific aviation objects, events and quantities, and include a desired or
required trend for a performance indicator (e.g. reduce the ATM cost per kilometre flown) while not yet expressing the
achievement of the performance objective in numeric terms (this is done as part of performance target setting). For
example, improve on-time arrival of flights throughout a particular regional or local planning area could be one of the
performance objectives of the efficiency KPA.

2.2.3.3 The agreed performance objectives must be “SMART”:

— Specific — The performance objective must be expressed in terms of the objects and events that
represent air traffic and its operational environment.
II-2-4 Manual on Global Performance of the Air Navigation System

— Measurable — It must be associated with one or more clearly defined performance indicators, and it
must be possible and affordable to establish the required data collection processes and to solve
information disclosure issues.

— Achievable — Performance objectives can be challenging, but must realistically consider the public
environment, timing and available resources.

— Relevant — Performance objectives should only be defined where there are anticipated performance
problems and opportunities related to ATM community expectations.

— Timely — The performance objective must be achievable in a timely fashion so as to contribute to the
ATM community expectations.

2.2.4 How to measure progress towards a performance objective?

[See Chapter 1, Figure II-1-2, Step 1, (3)]

2.2.4.1 Management of performance will be done at the level of specific performance objectives. Performance
indicators are defined in order to quantify the degree to which performance objectives are being met. When describing
performance indicators, one must define what and how measurements will be obtained and combined to produce the
performance indicator. Performance indicators should be quantitative, robust and outward looking, convey meaningful
information about process performance for a given performance objective and be suitable to trigger improvement and
change.

2.2.4.2 For the example “improve on-time arrival…” performance objective of 2.2.3, the performance indicator
could be “average arrival delay per flight in planning area X”. To calculate this performance indicator, one must obtain
data on the scheduled and actual arrival times of all flights in planning area X. From this, one can determine the total
arrival delay (a supporting metric), and divide that value by the number of arrivals (another supporting metric) to
calculate the desired performance indicator.

2.2.4.3 Performance indicators intended for use at regional and global levels should be standardized so as to
facilitate consistency of data collection at the local level.

2.2.5 Traffic forecasting

[See Chapter 1, Figure II-1-2, Step 1, (4)]

2.2.5.1 Why forecast?

2.2.5.1.1 A shared and consistent understanding of the future is one of the pillars for setting performance targets as
well as assessing the performance impact of existing plans and transition roadmaps.

2.2.5.1.2 Forecasts are used as input for some of the performance targets. For example, the capacity target for an
ATM operational environment (e.g. en-route centre, volume of airspace) is dependent on the forecasted traffic.

2.2.5.2 What needs to be in a forecast?

2.2.5.2.1 Forecasting should produce the information necessary to gain a better understanding of the characteristics
of the traffic — that is, of the demand. This qualitative view is important input for planning ATM systems.
Part II — Performance-based Transition Guidelines
Chapter 2. Measuring and assessing performance II-2-5

2.2.5.2.2 For example, forecasts containing just traffic and aircraft size based on the number of seats are not
sufficient to investigate the performance impact of improvements relying on airborne equipage. Analysis of future
performance may also require information on fleet mix, specific aircraft equipage levels, engine types, and demand (at
the passenger/cargo level). For setting performance targets on the environment it could be important to have a forecast
of kilometres (km) or nautical miles (NM) flown per engine type in addition to a forecast of the total km/NM flown.

2.2.5.2.3 Existing forecasts probably need to be enhanced to incorporate such additional information.

2.2.5.3 Do we need different types of forecast?

2.2.5.3.1 Forecasts can have different time horizons (e.g. long term (20+ years), short term (one year)) and
scope/forecasted information (e.g. number of movements, passenger kilometres, and fleet).

2.2.5.3.2 The scope and time horizon of the traffic forecast should be driven by the requirements for setting
performance targets.

2.2.5.3.3 For example, planning a year’s capacity for an en-route centre requires a more detailed and accurate
forecast than is required for developing a region-wide strategic transition roadmap for the next twenty years. Dependent
on their time horizons, forecasts use different forecasting methods. A forecast for one year is typically made using
statistical approaches whereas a long-term forecast uses econometric approaches and forecast scenarios.

2.2.5.4 Does performance affect the forecast?

The ATM system will react to changes in performance; thus, performance changes may alter the forecast. For example,
airport capacity will act as a constraint on traffic demand and may result in migration to larger aircraft and/or increased
demand at nearby uncongested airports. The provision of improved performance for early-adopters of an airborne
technology may increase the equipage rate for that technology. This illustrates that the performance assessment and the
forecast scenarios should be verified for consistency and may require iteration.

2.2.5.5 Need for iterative forecasts

Consistency between regions and localities may require an iterative process. Certain decisions by one region or locality
may be reached only after performance has been assessed. However, these decisions may substantially impact the
forecasts required by adjacent areas. For example, mandatory requirements for equipage or other changes that
significantly alter the cost structure in one region or locality might have a significant impact on neighbours. At a minimum,
a cyclical and synchronized performance planning process will allow these effects to be captured.

2.2.5.6 Cooperation in developing forecasts

2.2.5.6.1 Developing a forecast is also a process to achieve a shared understanding of the future with ATM
community members. This consensus is essential since it is the basis for identifying needs and agreeing on plans.

2.2.5.6.2 Not only should the forecast itself be developed in a collaborative manner, to enhance credibility of the
forecast, but the forecasting approach and method also need to be collaborated between community members.

2.2.5.6.3 The development of a long-term forecast typically starts with identifying and agreeing on the forecast
scenarios. Each scenario represents a consistent possible future specified in terms of values for the input parameters of
the forecast (e.g. economic growth, oil price, development of tourism, etc.). During this phase, it is important to capture
all future scenarios believed to be possible.
II-2-6 Manual on Global Performance of the Air Navigation System

2.2.5.6.4 A forecast will normally include three or four different input scenarios (sets of input assumptions about
external factors), resulting in a corresponding number of traffic forecasts. As part of the cooperation, it is important that
ATM community members agree to use the same forecasting scenario as the basis for setting performance targets and
estimating future performance.

2.2.6 Performance targets — how one knows performance objectives are attained
[See Chapter 1, Figure II-1-2, Step 1, (5)]

2.2.6.1 Performance targets are closely associated with performance indicators: they represent the values of
performance indicators that need to be reached or exceeded to consider a performance objective as being fully achieved.
Note that performance targets can be set as a function of time (e.g. to plan year-on-year improvement); they can also
vary by geographic area.

2.2.6.2 A decision-making/policymaking process needs to be in place to collaboratively agree on performance

objectives, performance indicators and the values of performance targets at the local, regional and — where required —
global levels.

2.2.7 How to set valid performance targets?

Performance targets are not set in an arbitrary manner. For certain performance objectives (e.g. in the capacity area),
the performance targets are dependent on the traffic/demand forecast. Knowledge gained through the baseline activities
and modelling of future scenarios should provide guidance as to both current and achievable values for performance
indicators. In particular, for short-term performance improvements, performance targets should reflect what is possible in
the time period. Over the long run, performance targets can push the envelope of what is currently possible.

2.2.8 Learning from your neighbours

Through benchmarking and the analysis of best practices, localities and regions may be able to learn from other areas
what achievable performance indicator values are, and what the combination of practices is which is required to obtain
those levels of performance. The benchmarking activity allows feasible performance targets to be set, based upon
experience in other areas. While these practices may not be directly transferable, understanding the relationship
between performance and practices provides guidance that is useful as a supplement to performance modelling or when
modelling is not feasible.

2.3 MEASURING AND ASSESSING PERFORMANCE

2.3.1 Assess current performance

[See Chapter 1, Figure II-1-2, Step 2, (7)]

2.3.1.1 At regular intervals, the performance of the actual system must be established through measurement of
operational data and calculation of performance indicators. Measurement of current performance is used for the
following purposes:

— to establish an initial performance level (i.e. a baseline); and

— To track actual progress towards meeting performance objectives by comparing the performance
indicators with set performance targets.
Part II — Performance-based Transition Guidelines
Chapter 2. Measuring and assessing performance II-2-7

2.3.1.2 When and how to assess current performance?

Prior to focusing on the future performance of the ATM system, one must first determine the performance of the current
ATM system. This process is referred to as assessing the baseline performance and allows one to establish the level of
performance of the current system in each KPA and for each performance objective through the relevant performance
indicators. Once performance indicators have been defined and mapped to each performance objective and KPA, the
assessment of the baseline performance requires the collection of required data during the baseline period, and the
calculation of each performance indicator.

2.3.1.3 Data quality and availability

2.3.1.3.1 Many ATM community members can be involved in the performance data reporting process with each
member having to justify the effort. The data reporting process should be harmonized between these organizations with
common definitions, and reporting requirements. Furthermore, legal, organizational and managerial impediments may
also exist.

2.3.1.3.2 Mechanisms should be in place to ensure that confidentiality is preserved such that sensitive data are not
used inappropriately.

2.3.1.3.3 Data quality will impact those performance indicators which can be confidently obtained and which may
negatively affect the performance-based approach. Consistency in data quality is also required across multiple
dimensions (e.g. time, space, type of flights). Just as data quality should be consistent across multiple dimensions, data
should also be available across these dimensions. This can be difficult to achieve.

2.3.1.3.4 Performance monitoring is an ongoing evolving process. Comparison of forecasts to actual performance
and tracking tasks all require consistency and stability in data. Data that are collected today may be required for a
different purpose in the future. It is therefore critical that data collection efforts be forward-looking and precisely
described.

2.3.1.3.5 Initial performance indicators may be constrained by the available data. While these may provide
reasonable starting points for transition to a performance-based ATM system, future data requirements should be
planned to ensure that the requisite data will be available.

2.3.1.4 Using the baseline to understand performance

Once the baseline performance has been established, one can begin to quantify the understanding of performance. For
example, each KPA may have performance objectives that the ATM community agrees are not being met. Through the
application of quantitative performance indicators, one begins to understand the actual performance indicator levels at
which performance objectives are or are not being met, and consequently, the appropriate performance target levels.
For example, if the ATM community agrees that ATM-induced delays are a problem, only after ATM-induced delays are
measured can one begin to understand the level of delays that is unacceptable.

2.3.1.5 Understanding interrelationships

Determination of the baseline performance will involve multiple performance objectives within KPAs and across multiple
KPAs. Prior to embarking on performance improvement, it is important to develop an understanding of the
interrelationships between different performance objectives within a KPA, and between different KPAs. These
interdependencies can allow performance improvement in one area through a trade-off in performance within another
area. Alternatively, interdependencies can lead to adverse consequences when trying to improve one area. Some
examples are listed below:
II-2-8 Manual on Global Performance of the Air Navigation System

— Within the efficiency KPA, trades can be made between fuel and time costs. Improvements in only one
measure may not reflect improvements in overall efficiency.

— Within the environment KPA, certain noise abatement procedures may decrease noise at the expense
of increasing emissions.

— Between the environment KPA and the capacity KPA, continuous descent approach procedures may
provide improvements in both noise and emissions at the expense of capacity.

— The ability of airspace users to modify flight trajectories or arrival and departure times will favourably
impact the flexibility KPA, but may adversely impact the capacity KPA.

— While a positive correlation exists between capacity and efficiency, as measured by delay,
consideration of cost efficiency is required.

2.3.2 Validating planned performance

[See Chapter 1, Figure II-1-2, Step 2, (7)]

Planned performance is the performance achieved if a given set of plans and transition roadmaps is implemented.
Planned performance needs to be assessed through validation activities. The main objective of validation is to reduce
uncertainties in critical decision areas such as performance. It is an essential part of the performance planning process,
as it allows the evaluation of future performance under circumstances such as:

— no ATM system improvements (do nothing scenario);

— cancellation of planned improvements;

— implementation of already planned ATM system improvements; and

— implementation of additional ATM system improvements beyond those currently planned (assessment
of the performance impact of proposed updates to plans and transition roadmaps).

2.3.3 What assumptions to use for future performance?

[See Chapter 1, Figure II-1-2, Step 2, (6)]

Estimating the future performance of the ATM system is an essential subject of ongoing research and development and
should be organized in such a way that it provides input to the planning process. Two possible sets of assumptions
commonly used are as follows:

— The “do nothing” case — this case assumes that the ATM system will remain as it is today. However,
externally-driven changes (e.g. growth in traffic, changes in fleet composition) will force changes in
performance.

— The “planned improvements” case — this case assumes that the ATM system will change according
to a specified plan already in place. Changes in performance will be affected by the prior external
changes, in concert with all ATM system changes that are already planned. Note that this case may
still not meet performance targets if situations have changed since the improvements were planned.
Part II — Performance-based Transition Guidelines
Chapter 2. Measuring and assessing performance II-2-9

2.3.4 What are commonly used validation methods?

Methods for estimating the performance of the future ATM system are varied and include analytical/statistical methods,
system dynamics, influence analysis, fast-time and real-time simulation, and prototyping. Modelling of interactions
between local and regional effects may be required for certain performance cases as one may not always be able to
isolate ATM performance to one locality.

2.3.5 How to deal with uncertainty?

[See Chapter 1, Figure II-1-2, Step 2, (7)]

One aspect that must be considered is uncertainty. Uncertainty is present not only in the traffic forecast, but also in the
validation results. Therefore, the estimated future values of performance indicators are subject to uncertainty. Methods
for dealing with this uncertainty include:

— Probabilistic estimation of the performance outcome of future improvements. This can be used in
models which describe how uncertainty propagates through the chain of related performance
indicators, to allow sensitivity analysis.

— Real measurement data on past and current performance, as improvements are deployed. This
provides a control mechanism to track progress and re-evaluate whether improvements are sufficient
or excessive to meet performance targets.

2.4 IDENTIFY AND DIAGNOSE PERFORMANCE GAPS

2.4.1 What is a performance gap?

[See Chapter 1, Figure II-1-2, Step 2, (8)]

2.4.1.1 The term performance gap is used to denote a current or anticipated mismatch between the current or
planned performance and the performance target. This is illustrated in Figure II-2-2.

2.4.1.2 In this example, the capacity target increases gradually — in line with the traffic forecast — and the
planned capacity increases in a stepwise fashion. As can be seen in the figure, a performance gap exists for those years
where available capacity is lower than required capacity.

2.4.1.3 The (anticipated) existence of a performance gap indicates that a performance objective is not or will not
be fully achieved, and by consequence that the ATM community expectations in one or more KPAs will not be met to the
extent desired.

2.4.1.4 The opposite is also possible: performance (temporarily) exceeding the performance target. This is rarely
seen as a problem, as long as other performance targets (e.g. cost effectiveness) are still met. In many cases, there are
good reasons for having “spare” performance in the system: e.g. to be ready for future more challenging performance
targets. A typical example is the presence of spare capacity in a traffic growth scenario.

2.4.1.5 Gaps are calculated at the level of performance indicators, but need to be analysed at a higher level
(performance objectives and KPAs to determine where and when improvements are necessary.
II-2-10 Manual on Global Performance of the Air Navigation System

Performance Target

Capacity (movements/hr)

Performance gap Performance gap

Time (years)

Figure II-2-2. What is a performance gap?

2.4.2 What are the possible reasons for performance gaps?

2.4.2.1 Normally, transition roadmaps and plans have been adapted during the previous planning cycle to mitigate
all performance gaps. If, as part of the current planning iteration, new gaps are discovered, an analysis should be
conducted to understand the underlying reasons for these new gaps. Such an understanding helps to choose the most
appropriate approach for closing the gaps. Possible causes include the following:

2.4.2.2 Performance targets have been changed

When more challenging performance targets are applied or traffic demand is higher than previously forecasted, current
performance levels and/or the previously agreed performance enhancement profile may no longer be sufficient. An
example of this is shown in Figure II-2-3.

2.4.2.3 Revised traffic forecast negatively affects performance levels

2.4.2.3.1 The opposite is also possible: performance targets which remain the same over time, but performance is
expected to be less favourable due to a revised (more challenging) traffic forecast, again leading to performance gaps.

2.4.2.3.2 In the example of Figure II-2-4, safety performance is expressed in terms of the number of ATM related
accidents per year. Lower values represent better performance. The performance objective stipulates that the annual
number of accidents should not increase, even in the face of traffic growth (the performance target remains constant
through time). The presently foreseen operational improvements are sufficient to keep the performance at a constant
level, well below the performance target. In the example, a revised (increased) forecast without adapted operational
improvements is expected to lead to an increased number of ATM related accidents, eventually resulting in a
performance gap.
Part II — Performance-based Transition Guidelines
Chapter 2. Measuring and assessing performance II-2-11

Performance New target

Capacity (movements/hr)

Old target

Time (years)

Figure II-2-3. Effect of more challenging performance targets

New (expected)
performance
Target
Safety (accidents/year)

Old expected
performance

Time (years)

Figure II-2-4. Effect of changing traffic evolution

II-2-12 Manual on Global Performance of the Air Navigation System

2.4.2.4 Implementation not according to plan

2.4.2.4.1 When plans do not get implemented as foreseen (implementation is delayed or plans are only partially
being implemented), performance enhancements will be shifted into the future. As shown in Figure II-2-5, this may result
in performance falling behind performance targets.

2.4.2.4.2 The first time the process is applied, there may not yet be a plan. In that case, the performance gaps
indicate what would happen in a “do nothing” planning scenario.

2.4.2.4.3 Comparison against a “do nothing” scenario may also be useful to demonstrate the added value of a given
transition roadmap or plan against a baseline situation.

2.4.2.5 Operational improvements deliver less than expected performance benefits

When performance improvement estimates associated with transition roadmaps and plans are revised downwards (see
Figure II-2-6), the resulting benefits may not be sufficient to cover the performance needs.

New (expected)
performance
Capacity (movements/hr)

Old expected
performance

Target

Time (years)

Figure II-2-5. Effect of delayed implementation

Part II — Performance-based Transition Guidelines
Chapter 2. Measuring and assessing performance II-2-13

New expected
performance

Capacity (movements/hr)
Old expected
performance

Target

Time (years)

Figure II-2-6. Effect of revised benefit estimates

2.4.3 Performance assessment/review in practice

[See Chapter 1, Figure II-1-2, Step 2, (8)]

2.4.3.1 The task of identifying and diagnosing performance gaps corresponds to answering question 8 (What are
the current and anticipated performance gaps and their reasons?) that was introduced in Chapter 1, 1.4.10.

2.4.3.2 This type of activity is called performance assessment or performance review (latter term used when
analysing past and current performances). It is a data-oriented analysis, which according to Chapter 1, Figure II-1-2 is
based on the following input:

— output from box 7: current and anticipated values for each of the performance indicators, under certain
assumptions (forecasting scenarios and existing transition roadmaps and plans);

— output from box 5: the latest version of the agreed performance targets.

2.4.3.3 In principle, the output of this activity is just a list of performance gaps and their causes. In practice, the
scope of the activity is often interpreted as being much wider and includes the offering of recommendations to mitigate
the gaps.

2.4.3.4 For the purpose of organizing performance assessment/review, the task can be broken down into four
separate activities:

— data publication;

— data analysis;

— formulation of conclusions; and

— formulation of recommendations.
II-2-14 Manual on Global Performance of the Air Navigation System

2.4.3.5 Data publication

2.4.3.5.1 Performance assessment/review can start once the required data (performance targets and
current/anticipated values for performance indicators) are available. The first activity in this process is data publication.

2.4.3.5.2 With proper ATM community participation in place, ATM performance will be evaluated by two different
groups:

— members of the ATM community at large; and

— analysts from designated ATM performance review organizations.

2.4.3.5.3 Each group has its own specific need for access to ATM performance data, which should be satisfied by
appropriate data publication means.

2.4.3.5.4 The ATM community at large has a general interest in ATM performance. Even those not directly involved
in planning activities need to see executive level, quality controlled data and draw their own conclusions, at which point,
the need arises to make certain performance data publicly available in the interest of transparency. A capability is
therefore required which enables members of the ATM community to monitor the current situation against the
performance targets, and to provide them with the trends and the “big picture”. This need is generally satisfied by
publishing high-level performance indicator dashboards. These dashboards are periodically updated, and generally
allow limited or no interactivity by the user.

2.4.3.5.5 In addition, analysts from designated ATM performance review organizations are tasked with gaining an in-
depth understanding of ATM performance, and finding causes and effects. Their work is an integral part of the
performance management process described in Chapter 2. Their data needs are best satisfied by publishing selected
data in performance assessment databases which are designed to suit the analysts’ needs. These databases should
allow high levels of interactivity (querying and analysis).

2.4.3.6 Data analysis

2.4.3.6.1 At the data analysis stage, the performance assessment/review organization should ensure that the data
are already quality checked. Rather than struggling with data quality issues, analysts should be able to focus on their
main task: performance assessment/review.

2.4.3.6.2 Analysts will need to compare performance indicators against performance targets, identify performance
evolution trends, analyse historical evolution of performance, and find relationships (correlations) between performance
indicators, supporting metrics, etc.

2.4.3.6.3 This is achieved with the aim of gaining better insight into past, current and future ATM performances.
They will look at the “big picture” (annual totals and averages, performance indicators summarized during the planning
cycle) and break down the data into very detailed levels to find the causes of performance gaps and the reasons for
trade-offs. As a side-effect of data analysis, they should be able to formulate performance objectives, define new
performance indicators and identify data needs.

2.4.3.6.4 Analysts, as well as decision-makers, will benefit from using well-established cause and effect analysis
methodologies/models that facilitate the identification of the main drivers impacting the performance of the system.
Part II — Performance-based Transition Guidelines
Chapter 2. Measuring and assessing performance II-2-15

2.4.3.7 Formulation of conclusions

After completing the data analysis, analysts are expected to document the insight they have gained by formulating
conclusions for each KPA. Normally, these conclusions contain an assessment of the sufficiency of current and
expected future performance, for each performance objective. Alternatively, a conclusion could be that the available data
are insufficient for meaningful performance assessment/review. Typically, the conclusions are published in a
performance review report.

2.4.3.8 Formulation of recommendations

2.4.3.8.1 An integral part of the performance assessment/review process is the formulation of recommendations.
These should be derived from the conclusions, and be also included in the performance review report.

2.4.3.8.2 Recommendations should focus on how to meet ATM community expectations through agreed
performance objectives, performance indicators and performance targets. When evaluation indicates inconsistency
between ATM community expectations and performance objectives, performance indicators and performance targets,
recommendations may include:

— the need to set or change performance objectives;

— the need to (re-)define performance indicators; and

— the need to set or change performance targets.

2.4.3.8.3 Recommendations will also fall more typically into the following categories (non-exhaustive list):

— related to the need to improve performance data collection;

— suggested operational improvements related to identified performance gaps; and

— recommendations of an organizational nature (a task force, define an action plan, etc.) with a view to
actually starting the implementation of the above recommendations.

2.4.3.9 Positioning of performance assessment/review within the overall process

2.4.3.9.1 This document recommends a sufficient integration of the performance assessment/review activity into the
overall performance planning process, to ensure that the conclusions and recommendations serve as a direct input to
box 11 of the performance-based transition approach outlined in Chapter 1, Figure II-1-2.

2.4.3.9.2 At the same time, performance assessment/review should maintain a certain independence from the other
parts of the process in order to ensure a sufficient level of objectivity and impartiality.

_____________________
 Chapter 3

ADDRESSING PERFORMANCE GAPS

Addressing performance gaps corresponds to Step 3 (update transition roadmaps and plans to mitigate identified
performance gaps) as listed in Chapter 1, 1.4.

3.1 OPERATIONAL IMPROVEMENTS

3.1.1 This section addresses box 10 (What are the available options for operational improvement?) in Chapter 1,
Figure II-1-2 and the guidance that is to be obtained from box 9 (What is the global ATM operational concept and
associated system requirements?).

3.1.2 What are operational improvements?

[See Chapter 1, Figure II-1-2, Step 3, (10)]

3.1.2.1 Operational improvements are changes to the ATM system that are on the transition path towards the
global ATM operational concept (3.1.5 refers) and result in a direct performance enhancement. Because of their need to
deliver performance, the elementary changes that make up an operational improvement are intended to be implemented
together. An operational improvement is a transition step in a transition roadmap.

3.1.2.2 By its very nature, an operational improvement:

— is associated with a particular “before” state of the ATM system (defines the environment in which the
change can be implemented);

— describes the “after” state of the transition; and

— includes an estimate of the corresponding performance improvement.

3.1.3 What is the role of operational improvements in the planning process?

3.1.3.1 Since an operational improvement is a transition step in a transition roadmap, it should be suitable for
being developed into a (major) implementation project or programme. This will need to be done when the early parts of a
transition roadmap are elaborated into an implementation plan.

3.1.3.2 The output of box 10 of the performance-based transition approach in Chapter 1, Figure II-1-2 should
ensure that a common list of possible operational improvements is offered to the planning community to deliver the list of
options from which to develop transition roadmaps adapted to the specific needs of each region.

II-3-1
II-3-2 Manual on Global Performance of the Air Navigation System

3.1.3.3 Since the various regions of the world develop at different speeds and also may find diverse solutions to
similar performance problems, this list of options will essentially consist of two groups of operational improvements:

— mature improvements which have already been implemented in parts of the world, and are considered
“best practice” for similar performance problems and/or for meeting specific performance objectives;
and

— improvements not implemented anywhere yet, but currently in the process of being researched,
developed, and/or validated, and considered to be viable candidate solutions for meeting future
performance needs.

3.1.3.4 The Appendix to this Part provides a number of illustrations of operational improvements.

3.1.4 Developing a list of available options

for operational improvement

3.1.4.1 Operational improvements are made possible by technical system, human factor, procedure and
institutional enablers. These need to be identified and analysed to understand the feasibility, timing, cost, or, in general,
the impact of an operational change. The operational improvements are developed from different complementary
perspectives:

— high-level strategy and policy orientations;

— operational concept/concept of operation describing the operational evolution through time;

— architecture in which the technical system enablers (e.g. flight data processing systems, CNS
systems) of the operational improvements should fit;

— baseline (“before” state) from which the operational improvements need to be deployed and deliver
benefits;

— feasibility and timing from complementary perspectives defining the feasibility/timing of developing/
deploying the enablers required for the operational improvement. Typically this will provide information
on earliest availability;

— the safety and human factors assessment is required to have sufficient confidence that the operational
improvement is feasible from a human factors and safety perspective, and that a first list of issues that
need to be addressed during the development life cycle is raised; and

— the expected performance enhancement contribution in each of the KPAs should be assessed. More
specifically, it should be explicitly specified to which performance objective(s) the improvement is
targeted, and which performance indicator(s) is intended to be influenced.

3.1.4.2 The Global Air Navigation Plan (Doc 9750) is one of the sources for developing the list of operational
improvements. Use can also be made of work undertaken in other planning regions.

3.1.4.3 In those cases where a list of operational improvements has already been developed during a previous
planning cycle, the task consists of updating the list taking the latest developments into account.
Part II — Performance-based Transition Guidelines
Chapter 3. Addressing performance gaps II-3-3

3.1.5 Using guidance from the Global ATM operational concept and associated system
requirements to develop operational improvements

This section explains the role of box 9 of the performance-based transition approach in Chapter 1, Figure II-1-2.

3.1.6 Role of concept and requirements documents

[See Chapter 1, Figure II-1-2, Step 3, (9)]

3.1.6.1 The operational improvements that are applied to achieve performance enhancements should be focused
on the achievement of a common goal — a common vision of ATM. The ICAO vision of an integrated, harmonized and
globally interoperable ATM system for a planning horizon up to and beyond 2025 can be found in the Global Air Traffic
Management Operational Concept (Doc 9854).

3.1.6.2 This vision and its envisaged performance characteristics is further detailed in a set of system
requirements which can be found in the Manual on Air Traffic Management (ATM) System Requirements (Doc 9882).

3.1.6.3 Together, these documents form the main guidance material for determining the available options for
operational improvement.

3.1.6.4 Categorization and sequencing of operational improvements

3.1.6.4.1 Chapter 2 of Doc 9854 defines seven operational concept components which can serve as the general
categorization framework for operational improvements:

— Airspace organization and management (AOM)

— Aerodrome operations (AO)

— Demand and capacity balancing (DCB)

— Traffic synchronization (TS)

— Airspace user operations (AUO)

— Conflict management (CM)

— ATM service delivery management (ATM SDM)

3.1.6.4.2 This same structure is also used to group requirements in section 2.4 of the Manual on Air Traffic
Management (ATM) System Requirements (Doc 9882).

3.1.6.4.3 In addition to the seven operational concept components, both documents describe the exchange and
management of information used by the different processes and services — information services.

3.1.6.4.4 The list of operational improvements should be classified around those eight categories, and should be
based on the material in the above-mentioned operational concept and requirements documents.

3.1.6.4.5 As mentioned in 3.1.4, operational improvements should also be categorized according to the performance
objectives to which they are intended to contribute (while it is recognized that operational improvements may also have
an adverse impact on certain other performance objectives, the main purpose of categorization is to point the ATM
community towards suitable solutions for given performance problems).
II-3-4 Manual on Global Performance of the Air Navigation System

3.1.6.4.6 Some operational improvements may cut across classification categories — irrespective of whether such
categories are derived from the operational concept components, or are the KPAs or are the performance objectives.

3.2 BUILDING/UPDATING YOUR TRANSITION ROADMAP

3.2.1 This section covers boxes 11 and 12 in Chapter 1, Figure II-1-2. The result of box 11 is a new version of
the transition roadmap for a particular region. The ATM community will need to modify the current version of the
transition roadmap (built during the previous planning cycle) and reassess the performance impact of the changes. The
resulting new transition roadmap is acceptable if it has a realistic potential for mitigating the performance gaps.

3.2.2 What is a transition roadmap,

and how is it different from a plan?

3.2.2.1 A transition roadmap is a high-level representation of the selection of operational improvements and their
implementation sequence, adapted to the needs of a particular planning area (at regional or local levels).

3.2.2.2 A transition roadmap has a long time horizon (typically twenty years) and a low update frequency (typically
once every five years).

3.2.2.3 It is more a driver for the initiation of research than for implementation. Therefore, in a transition roadmap,
specifying the sequence of improvements is more important than trying to pinpoint the exact timing of their deployment.
It is common practice to subdivide the time period covered by a transition roadmap into three phases: short term,
medium-term and long-term improvements.

3.2.2.4 The role of a transition roadmap is to show how the ATM system in a particular planning area will be able
to migrate in a coherent way from its present state, via a number of feasible intermediate steps (the envisaged situation
at the end of the short- and medium-terms), to a long-term “end state” in which the global ATM operational concept is
implemented. The speed of developments (introduction of operational improvements) is driven by the expected evolution
of performance needs, and constrained by earliest availability of operational improvements.

3.2.2.5 An implementation plan is complementary. It has a much shorter time horizon (typically five years) and is
usually updated on an annual basis. It is derived from the early (short-term) parts of a transition roadmap. It lays out a
detailed set of implementation focused actions — including their timing — for all involved members of the ATM
community.

3.2.2.6 Research plans are similar to implementation plans, except that they focus on the research actions needed
today to improve the maturity of operational improvements placed in the medium- and long-term portions of the transition
roadmap.

3.3 CHOOSING SOLUTIONS TO MITIGATE PERFORMANCE GAPS

3.3.1 Developing a new transition roadmap

[See Chapter 1, Figure II-1-2, Step 3, (11)]

3.3.1.1 In choosing how to adapt the current transition roadmap (or build a new one), the following general
guidelines can be used.
Part II — Performance-based Transition Guidelines
Chapter 3. Addressing performance gaps II-3-5

3.3.1.2 The analysis of performance gaps (box 8, Chapter 2, 2.4 refers) has not only identified (anticipated)
performance gaps in terms of performance indicators, but has also provided insight into the causes. Planners also know
where and when the performance gaps are expected to occur, and what particular performance objectives will no longer
be achieved.

3.3.1.3 The affected performance objectives indicate specific areas that need action to improve performance. This
is the point at which the categorization of operational improvements according to performance objectives is to be used
as a tool to develop an appropriate shortlist of candidate solutions. The shortlist is by definition compliant with the global
ATM operational concept, because it is derived from the full list which was developed using the operational concept
(3.1.5 refers).

3.3.1.4 Solutions (i.e. operational improvements) should be selected from the shortlist as follows:

— if (some of) these solutions are already included in the current transition roadmap, one should see
whether the performance gaps cannot be mitigated by accelerating their deployment (in time,
geographically, in terms of fleet equipage levels, etc.), so as to increase their performance
contribution;

— if this is not possible, solutions should be selected which are not yet included in the transition
roadmap. Consideration should be given to mature operational improvements with a proven track
record (deployment of current best practices); and

— finally, planners can select from operational improvements which have not been implemented
anywhere yet, but are currently in the process of being researched, developed, and/or validated.

3.3.1.5 Note that with each planning iteration, the transition roadmap will also have to be extended by a number of
years corresponding to the planning cycle (usually five years), because of the need to maintain a constant time horizon
(twenty years).

3.3.2 Validating the new transition roadmap

3.3.2.1 The new transition roadmap should be checked as a whole for consistency. Using available validation
results, (1) a “shared belief” needs to be developed that the transition roadmap has the potential to mitigate the
performance gaps, and (2) that it is aligned at global, regional and local levels (Chapter 4 refers). The process of
developing a new transition roadmap may need to be reiterated until these two criteria are met.

3.3.2.2 Paragraph 3.1.4 describes a set of validation perspectives within the context of developing individual
operational improvements for the generic list of options. The same perspectives should be applied again, but this time
for the transition roadmap as a whole, and adapted to the specific assumptions and circumstances of the transition
roadmap’s planning area.

3.3.2.3 Because of the strategic nature of the medium- and long-term portions of transition roadmaps (higher
uncertainty levels and longer time horizons), it is generally accepted that their validation — prior to release during the
current planning cycle — can be done at a less rigorous than required for the validation of the short-term portion of the
transition roadmap (which is to be incorporated in the implementation plans). However, the medium- and long-term
portions of the new transition roadmap become the basis for focused validation activities in the research plan which
should result in decreased uncertainty by the time the next planning cycle is started.
II-3-6 Manual on Global Performance of the Air Navigation System

3.3.3 Update plans in accordance with the new version of the transition roadmap
[See Chapter 1, Figure II-1-2, Step 3, (12)]

3.3.3.1 Any change to a transition roadmap may require a (partial) revision of existing implementation plans.

3.3.3.2 In particular, what used to be “medium term” in the previous (five year old) version of the transition
roadmap has now become short term; therefore, a corresponding implementation plan needs to be developed. For the
same reason, the research plan needs to be updated.

3.3.4 Transition roadmap development in practice

3.3.4.1 The following guidance applies to the development of transition roadmaps at local (State) and regional
levels.

3.3.4.2 Develop the proposed transition roadmap, in at least three transition phases, giving consideration to the:

— existing plans and transition roadmaps of the State/region (if not already incorporated into the
performance gap analysis);

— existing transition roadmaps and plans of other States/regions;

— existing global plan; and

— existing treaties, agreements and plans with other members of the ATM community.

3.3.4.3 It is acknowledged that States and regions may have prior treaties, agreements and plans that need to be
addressed.

3.3.4.4 These transition roadmaps should be outlined for the three transition phases below, to provide common
ground for comparisons (between local/State and regional transition roadmaps) and to provide relevant information to
different groups of ATM community members.

— Transition Phase 1: short term (a period typically from the present to five years into the future)

— Transition Phase 2: medium term (a period typically from five to ten years into the future)

— Transition Phase 3: the longer term (a period typically from ten to twenty years into the future)

3.3.4.5 Transition Phase 1 and to a certain extent Phase 2 are implementation-driven (i.e. oriented towards
development and deployment).

3.3.4.6 Transition Phase 3 is mainly concept-driven because the technical solutions are still in the research phase.
However, it is still important to plan for all of these phases:

— to ensure the integrity of the twenty-year transition roadmap as a whole; and

— to encourage and influence the development of technologies to meet the requirements planned for
Transition Phases 2 and 3. This implies that a research plan should exist, covering today’s (short
term) research activities needed for tomorrow’s (medium and long term) development and
deployment. It is recognized that the commitment of resources towards research, development and
deployment is achieved at a local level. The regional transition roadmaps are intended to reflect the
coordinated efforts of the regional members.
Part II — Performance-based Transition Guidelines
Chapter 3. Addressing performance gaps II-3-7

3.3.4.7 Transition Phase 1

3.3.4.7.1 In Transition Phase 1, changes to the ATM system infrastructure will be built almost completely on current
or already planned or purchased ATM technology (taking into account the long lead times that are common for ATM and
avionics systems changes).

3.3.4.7.2 Therefore, implementation issues will probably focus on:

— ATM procedures;

— processes;

— standards; and

— organization.

However, in Phase 1, operational changes will also be deployed, particularly those that:

— are currently in the planning stages (e.g. the global plan); or

— are about to be implemented as a result of existing research and development efforts.

3.3.4.7.3 The contents of Transition Phase 1 should already be elaborated to the level of (detailed) implementation
plans.

3.3.4.8 Transition Phase 2

3.3.4.8.1 The deployment of the operational improvements foreseen for Transition Phase 2 is usually more than five
years into the future, and is typically not yet included in the initial implementation plans (see Figure II-3-1). While
uncertainties are already more reduced in comparison to those in Transition Phase 3, Phase 2 is characterized by the
fact that research may not yet be completed: more work is needed to validate the operational improvements and develop
their performance cases to the level required for implementation planning. It should be noted that the development of
performance cases occurs in a progressive way: they are refined as the operational improvements are taken through
their different life cycle phases, i.e. from research through development to deployment.

3.3.4.8.2 To summarize, the focus of Phase 2 is on:

— building on the foundations laid down in Phase 1; and

— further developing the details of the list of operational improvements developed in box 10 of Chapter 1,
Figure II-1-2.

3.3.4.9 Transition Phase 3

3.3.4.9.1 The latter part of Phase 2 and all of Phase 3 are largely conceptual because the technical solutions are yet
to be developed. However, the transition roadmaps for these phases will start to influence the development of
technology solutions for the operational improvements allocated to these phases. This is one of the real benefits of
developing a roadmap that stretches twenty years into the future.
II-3-8 Manual on Global Performance of the Air Navigation System

2005 2010 2015 2020

IMPLEMENTATION PLAN
EDITION 2005

Development De
Deployment cre
TR as
AN ing
SIT lev
IO el
RESEARCH PLAN Short term NR of
de
OA tai
EDITION 2005 DM la
AP nd
ED ma
ITI tur
Research Development Deployment ON ity
20
05
Medium term

Research Research Development Deployment

Long term

Building upon an evolving institutional and regulatory framework

Figure II-3-1. Role of short, medium and long terms in a transition roadmap (example)

3.3.4.9.2 Planning for Phases 2 and 3 in advance, allows the research and development plans and efforts to be
focused on providing solutions that achieve:

— the desired performance;

— the expected benefits;

— respect for the long lead times needed for developing the operational concept into implementable
operational improvements.

_____________________
 Chapter 4

ENSURING ALIGNMENT THROUGHOUT THE

PLANNING PROCESS

4.1 PLANNING AT GLOBAL, REGIONAL AND LOCAL LEVELS

4.1.1 ATM planning takes place at different levels:

— At the global level, through ICAO involvement, regional differences that are obstacles to global
interoperability must be resolved.

— At the regional level, via Planning and Implementation Regional Groups (PIRGs), ATM community
members have agreed to evolve the ATM system in accordance with a common transition plan.
Operating environments and priorities may be different. Regional performance targets will be defined.
The regional level is defined by a division of the world into homogeneous regions with similar
characteristics, and a common ATM interest in terms of performance and transition planning. The
regional level is responsible for looking after the network effect of local planning activities.

— The local level corresponds to planning activities of individual members of the ATM community
(States, air navigation service providers, airspace users, equipment manufacturers, etc.). Subject to
regional and local consultation, local research and implementation plans should be aligned with their
regional transition plan such that regional performance would improve. Local performance targets will
be set as a function of regional performance targets. Local performance monitoring will take place with
the aim of aggregating this to regional performance indicators.

4.1.2 This approach ensures that planning takes place at the level of its effects — that is, involving all those
affected. In this manner, planning takes place in State, interstate, regional, interregional and all-regions groups,
depending on the extent of its inputs and effects.

4.1.3 The involvement of the different levels in the planning process is illustrated in Figure II-4-1.

4.1.4 More information on global, regional and local planning can be found in the Global Air Navigation Plan
(Doc 9750).

4.2 NEED FOR NEGOTIATED ALIGNMENT

4.2.1 One of the most critical challenges in transitioning to the ATM system envisioned in the global ATM
operational concept will be the alignment of global, regional and State/local planning and implementation activities. It is
obvious that there is only one global process — but there are several regional and local/State processes depending on
the division of planning and implementation activities.

II-4-1
II-4-2 Manual on Global Performance of the Air Navigation System

STEP 1
1 2 3 5

What are the What are the

How to measure What are the
ATM community performance
performance? performance targets?
expectations? objectives?
Global Regional Global Regional Global Regional Local Regional Local
4

What is the current traffic

and the expected traffic
evolution?
Global Regional Local

6 7 8 STEP 2
What is the current and What are the current and
What are the current
planned anticipated performance gaps
roadmaps and plans?
ATM performance? and their reasons?
Global Regional Local Regional Local Regional Local

STEP 3
9 10 11 12

What are the Global ATM What selection and sequence

What are the available What are the resulting
Operational Concept and of operational improvements
options for operational updates to current
associated system addresses current and
improvement? roadmaps and plans?
requirements? anticipated performance gaps?
Global Global Regional Regional Local Global Regional Local

Figure II-4-1. Role of global, regional and local levels

4.2.2 Ensuring effective alignment throughout this multiplayer and multilevel planning process requires:

— negotiation between planners on the same planning level (for adjacent States and regions) and
between levels to produce harmonized, coordinated transition roadmaps and plans;

— the common application of common processes — that is, achieving effective standardization of
transition planning and implementation processes in line with the guidance provided in this document;
and

— the introduction of an improved collaborative planning environment (common information exchange

platform), to facilitate the sharing of planning information, to improve the dialogue between planners
and to help guide the application of the common process. This subject is addressed in 4.6.
Part II — Performance-based Transition Guidelines
Chapter 4. Ensuring alignment throughout the planning process II-4-3

4.3 ROLE OF GUIDANCE MATERIAL

4.3.1 Harmonization of planning should be ensured right from the beginning, to minimize the need for alignment
of incompatible transition roadmaps and plans at the end of the process.

4.3.2 This is to be achieved by using global guidance material during regional planning, and global/regional
guidance in local planning processes.

4.3.3 Such guidance material includes (box numbers refer to Figure II-4-1):

— the Global Air Traffic Management Operational Concept (Doc 9854) and associated system
requirements document (box 9), for use at regional and local planning levels;

— the generic (global) list of available options for operational improvement (box 10), for use at the
regional planning level;

— region-specific lists of available options for operational improvement (box 10), for use at regional and
local planning levels;

— The global list of ATM community expectations (box 1), as specified in Doc 9854 (for use at the
regional planning level);

— ATM community expectations at the regional level and their priorities (box 1), for use at regional and
local planning levels;

— global and regional sets of performance objectives (box 2), for use at regional and local planning
levels;

— global and regional guidance on measuring performance (including common definitions for
performance indicators) (box 3), for use at regional and local planning levels; and

— global and regional guidance on developing long-term traffic forecasts (box 4), for use at regional and
local planning levels.

4.4 ALIGNING PERFORMANCE MEASUREMENT AND ASSESSMENT

4.4.1 A lack of alignment of performance measurement and assessment between regions and/or States will lead
to transition roadmaps and plans being based on different assumptions.

4.4.2 Therefore it is important that the data used for the performance-based transition approach are consistent
within and between regions. This data includes traffic forecasts, performance measurements and predictions (via
performance indicators), and performance targets.

4.4.3 To achieve consistency of the data, it is imperative that all members of the ATM community (in particular
regions and States) use a consistent set of assumptions about the future (i.e. compatible forecasting scenarios) and
consistent definitions of performance objectives and performance indicators.
II-4-4 Manual on Global Performance of the Air Navigation System

4.5 ALIGNING THE PLANNING PROCESSES AND THEIR OUTPUTS

4.5.1 Performance and transition planning will be conducted at regional and local levels, using a certain planning
cycle. For effective coordination at the global level, a standard cycle should be used with a common period (e.g. five
years), and the cycles of the different regions and States should be synchronized. This allows a global consolidation to
be built into the process at specific checkpoints.

4.5.2 The standard cycle will consider a planning horizon of twenty years. The objective of a planning cycle is to
adapt the evolution of the global ATM system to changes in the following areas:

— After each five-year cycle, the progress made in ATM implementation and planning should be
reviewed, resulting in improved knowledge about operational improvements and their (planned)
performance for the entire twenty-year period (see also Chapter 3, Figure II-3-1).

• The operational improvements planned in the old short-term (five year) plans should now be
operational.

• Research should have progressed the available knowledge about the transition roadmap, so that
the medium-term operational improvements are ready to be transformed into an implementation
plan, and improved understanding is available for the long-term operational improvements.

• Initial research should also have been completed for the first part of the long-term period, so that
more specific research can start for these operational improvements.

— Also, after each five-year cycle, the impact of performance-oriented planning parameters should be
reviewed, such as:

• changing constraints and drivers, i.e. traffic demand growth;

• changes to the expectations of society and the ATM community; and

• changing performance targets.

4.5.3 The process starts with a first step: the development of an updated set of performance targets through
time. This includes the production of a new long-term traffic forecast. At this point, a first global consolidation takes
place to ensure commonality in the underlying forecasting assumptions that are embedded in the socio-political,
economic and technical forecasting scenarios. The consolidation also ensures that all regions use the same set of
interregional traffic flow forecasts. The result of the first step consists of a consistent global long-term traffic forecast and
agreed regional performance targets.

4.5.4 This is followed by a second step: performance assessment and identification of current and anticipated
performance gaps.

4.5.5 Based on this information, each region reviews its list of candidate operational improvements and updates
its transition roadmaps, research plans and implementation plans (third step). After this exercise, a second global
consolidation activity will take place, which will result in the updated set of regional transition roadmaps, which together
make up the global transition plan.

4.5.6 A fourth step in the process generates lessons learned (at the global level), which become input to the
fifth: maintaining the guidance material and the overall planning process itself.
Part II — Performance-based Transition Guidelines
Chapter 4. Ensuring alignment throughout the planning process II-4-5

4.5.7 In case of timing constraints, the fourth and fifth steps could possibly overlap with the first step of the next
planning cycle. The process could take up to seven years, while still maintaining a schedule in which a new cycle starts
every five years.

4.6 IMPROVING THE PLANNING ENVIRONMENT

4.6.1 This collaborative planning process will be aided substantially by development of a virtual (rather than the
current paper-based) planning environment, with the transfer of information conducted in real time within a distributed
information network (e.g. via the Internet). This will enable — in fact oblige — compliance with more rigid decision-
making processes that require consideration of the concept of global harmonization and a “systems approach” to
planning.

4.6.2 It will also make the material required for informed decision-making easily accessible, including the
processes used by other States and regions, the information which those States and regions used to make their
decisions, and importantly, lessons learned. This will reduce the risk that States and regions will need to start over and
will allow the costs of the improved aircraft and other infrastructure capabilities to be spread over a much larger group of
ATM community members.

4.6.3 By its very nature, such a virtual planning environment should be global. It is expected that ICAO would
develop, operate and maintain this planning environment within the context of global ATM planning support.

_____________________
 Appendix

ILLUSTRATIONS OF OPERATIONAL IMPROVEMENTS

Operational improvements (OIs) are changes to the ATM system that are on the transition path towards the global ATM
operational concept and result in a direct performance enhancement. An operational improvement is a set of elementary
changes that are intended to be implemented together to deliver performance. An operational improvement is a
transition step in a transition roadmap (Chapter 1, 1.4.12 refers).

To illustrate the above definition, this appendix contains a list of example operational improvements:

1. Airborne spacing

2. Airport and flight information sharing

3. Decision support for sector operations

4. Situational awareness for aerodrome ground operations

5. Flexible airspace structures

6 Data link services

Each operational improvement is described as follows:

1. Reference ATM system (Starting point)

The reference ATM system is the baseline situation that will be improved by the operational improvement.

2. Operational improvement (OI) description

In this field the OI is described briefly and the description is focused on the essential changes.

3. Primary performance purpose

This field lists the KPAs that are most affected by the OI and describes its effect for each area.

4. Links to the operational concept components of the global ATM operational concept

Reference links are made to the operational concept components of the global ATM operational concept. The links
could be used to look up information about the conceptual background.

5. Links to ATM system requirements

Reference links are made to the ATM system requirements (requirement reference numbers as specified in the
Manual on Air Traffic Management (ATM) System Requirements (Doc 9882)). This gives an overview of the ATM
system requirements that need to be taken into account when implementing this operational improvement. The links
to the ATM system requirements form an illustrative list.

II-App-1
II-App-2 Manual on Global Performance of the Air Navigation System

1. AIRBORNE SPACING

Reference In this state, air traffic situational awareness information is provided on the surrounding
ATM system traffic to the airspace users without change of the roles and responsibilities between
(Starting point) pilots and controllers.

OI description Controllers will be able, under defined conditions, to delegate the spacing tasks to the
flight crew of suitably equipped aircraft. The flight crews will perform these new tasks
using new aircraft capabilities.

The core roles of controllers and flight crews remain unchanged. The flight deck is
responsible for spacing according to ground instructions and responsibility for
separation remains with the ground.

Agreement between the controller and the pilot is a prerequisite. Instructions are limited
in time or distance. This requires that a sufficient number of aircraft are equipped to
achieve benefits.

Three examples of ATM operations are considered:

1) Sequencing and merging operations

The controllers will be provided with a new set of instructions directing, for example,
the flight crews to establish and to maintain a given time or distance from a
designated aircraft. This application could be used to facilitate traffic
synchronization to the runways.

2) In-trail procedure in oceanic airspace

Under the appropriate circumstances, the controller will clear the flight crew to climb
or descend to a given flight level. It will allow the aircraft to fly at a more optimum
level.

3) Enhanced crossing and passing operations

To solve conflicts between two aircraft (or allow aircraft to avoid hazards) the
controller will use new ATC instructions and procedures that will use new
capabilities of aircraft to fly relatively to one another or avoid the hazard.

Primary performance — Capacity: The delegation of tasks to the pilots is expected to allow the handling of
purpose more flights per airspace volume per time period.

— Efficiency: There are fuel benefits from allowing aircraft to climb while in oceanic
airspace (e.g. in-trail climb procedures).

— Environment: Reduction of fuel consumption will reduce gaseous emissions and

less dense traffic in the vicinity of airports could reduce noise impact.

— Safety: The involvement of more participants in the separation provision is expected

to have a positive effect on safety, e.g. through additional redundancy in conflict
detection and resolution.
Part II — Performance-based Transition Guidelines
Appendix. Illustrations of operational improvements II-App-3

Links to operational — Conflict management (CM).

concept components
— Traffic synchronization (TS).

— Airspace user operations (AUO).

Links to ATM system R48, R53, R61, R66, R85, R115, R211, R213
requirements

2. AIRPORT AND FLIGHT INFORMATION SHARING

Reference Collaborative decision-making (CDM) process between the service provider and the
ATM system major airport users with limited real time access to status information and operations
(Starting point) factors. Current collaborative decision-making process is locally adapted and may not
be globally interoperable.

OI description Airport operators will participate in airport information sharing and improve the planning
of their resources by using real time flight information accessible via CDM. Aircraft
operators participate in sharing of relevant real time flight plan information via CDM and
improve their planning of schedules by using the shared information from other
stakeholders. Where feasible they can indicate their priorities in managing their own
flights within the arrival schedules. Service providers will be aware of the requirements
of other users and service providers and the agreed collaborative rules to resolve
competing requests for ATM resources.

Local CDM processes at aerodromes will build on sharing of key flight scheduling
related data that will enable all participants to improve their awareness of the aircraft
status.

Primary performance — Cost effectiveness: CDM allows the trade-off be made across the ATM community.
purpose
— Efficiency: Collaboration allows ATM system users to operate in a manner that is
consistent with their individualized business cases that may not be known to other
ATM system participants.

— Environment: Gains in efficiency often result in decreased fuel consumption that

can lead to performance gains in the environment KPA.

— Participation by the ATM community: Increased participation by the airspace user

and better information sharing.

Links to operational — Traffic synchronization (TS)

concept components
— Demand and capacity balancing (DCB)

— Aerodrome operations (AO)

II-App-4 Manual on Global Performance of the Air Navigation System

— Airspace user operations (AUO)

— ATM service delivery management (ATM SDM)

— [Information management, collaborative decision-making]

Links to ATM system R21, R23, R24, R25, R26, R27, R29, R30, R32, R33, R34, R35, R36,
requirements
R53, R55, R56, R67, R71, R80, R83, R84, R92, R98, R100, R112,

R113, R114, R115, R125, R143, R145, R146, R153, R154,

R155, R159, R160, R161, R169, R178, R179, R211

3. DECISION SUPPORT FOR SECTOR OPERATIONS

Reference Control of en-route and TMA sectors is supported by safety nets. The early detection
ATM system of potential conflicts is limited and not very accurate. There is a lack of decision
(Starting point) support.

OI description The tasks of controllers will become supported by more automation.

The detection up to approximately 20-30 minutes before the event of potential

conflicts between flights, between flights and hazards or between flights and
restricted airspace will be supported by capabilities, which will facilitate earlier
handling of such events.

The monitoring for conformity of the traffic situation to planning and ATC clearances
will be supported by flight path monitoring capabilities.

Improvements will be made to safety nets.

For early resolution of planning conflicts, basic levels of “what-if” probing functionality
will become available.

These changes will have an impact on the roles and tasks of both the executive and
planning controllers. All these changes should be part of an integrated update of the
controller working position and of operational procedures.

Primary performance — Capacity: The reduction of tactical action allows managing more traffic within the
purpose same acceptable workload limits. The workload will be better distributed over the
team.

— Cost effectiveness: The reduction of workload allows managing the same

volume of traffic with fewer resources.
Part II — Performance-based Transition Guidelines
Appendix. Illustrations of operational improvements II-App-5

— Efficiency: Early resolution of potential problems will generally lead to a lower

number of flight path modifications while reducing excess route lengths and
avoiding unnecessary speed control.

— Safety: Decision support systems allow better anticipation of potential conflicts.

Conflict resolution strategy will be optimized and the need for tactical action will
be reduced, thereby reducing risk.

Links to operational — Airspace user operations (AUO)

concept components
— Traffic synchronization (TS)

— Conflict management (CM)

— ATM service delivery management (ATM SDM)

Note.— There is a need for effective information management.

Links to ATM system R06, R11, R27, R31, R53, R61, R65, R66, R73, R83, R117, R183
requirements

4. SITUATIONAL AWARENESS FOR AERODROME GROUND OPERATIONS

Reference Weather can severely impede aerodrome ground operations. Decreased situational
ATM system awareness — particularly in bad weather — significantly reduces airport capacity in
(Starting point) order not to jeopardize safety.

OI description Capabilities for detecting the position and movements of all vehicles and aircraft on the
manoeuvring area and of all aircraft on aprons will be introduced, allowing for better
situational awareness both in the air and on the ground.

Conflict detection and resolution will be provided on all aerodrome movement areas,
including runways (e.g. to prevent runway incursions), taxiways and aprons. This will
contribute to better control of ground movements under low visibility conditions.

Primary performance — Access and equity: Through increased capacity and more efficient low visibility
purpose operations.

— Capacity, efficiency and environment: Through optimized aircraft taxi and handling.

— Flexibility: Through better use of existing ground resources (runways, taxiways,

gates, etc.)

— Safety: Through better situational awareness and conflict detection tools.

Links to operational — ATM service delivery management (ATM SDM)

concept components
II-App-6 Manual on Global Performance of the Air Navigation System

— Demand and capacity balancing (DCB)

— Airspace user operations (AUO)

— Conflict management (CM)

Links to ATM system R11, R23, R24, R25, R26, R29, R30, R32, R33, R35, R61, R73, R77, R80, R84, R92,
requirements R100, R101, R117, R127, R128, R143, R154, R167, R168, R177, R178, R197, R199,
R202, R211, R213, R214, R215, R216

5. FLEXIBLE AIRSPACE STRUCTURES

Reference Airspace structures are static and are generally constrained by national or facility
ATM system boundaries. The airspace management is not very flexible and is based on pre-planned
(Starting point) scenarios.

OI description Flexible airspace structures will support flights to be operated principally on airspace
user-preferred routes. Airspace management will have evolved to a very dynamic
system. The focus is to ensure that resources are available where and when necessary.
There will be a highly dynamic use of sectorization scenarios.

All airspace will be the concern of ATM and will be a useable resource; airspace
management will be dynamic and flexible.

Any restriction on the use of any particular volume of airspace will be considered
transitory and all airspace will be managed flexibly.

Airspace boundaries will be adjusted to particular traffic flows and should not be
constrained by national or facility boundaries.

Primary performance — Capacity: Airspace capacity will be better utilized and capacity will be increased by
purpose a sector design adjusted to the traffic flows.

— Cost effectiveness: This OI will provide an increase in annual network productivity

through dynamic allocation of resources.

— Efficiency: Higher flight efficiency due to maximized capability for enabling user
preferred routing.

Links to operational — Airspace organization and management (AOM)

concept components
— ATM service delivery management (ATM SDM)

— Demand and capacity balancing (DCB)

Links to ATM system R01-05, R6, R15-20, R32, R34-36g, R67-68, R71-73, R99, R105-109, R112-114,
requirements R121-125, R148-152, R159-161
Part II — Performance-based Transition Guidelines
Appendix. Illustrations of operational improvements II-App-7

6. DATA LINK SERVICES

Reference An ATM environment which uses voice communication as the primary means of
ATM system communication between ground and aircraft.
(Starting point)

OI description Data link capabilities will become available to replace some radiotelephony (R/T)
exchanges. The system automation and controllers will also be able to make use of
some aircraft derived data (ADD), giving a more accurate view of the situation.

Aircraft will have to provide ADD to the air traffic services. Also, data link capabilities will
need to be available on board the aircraft. From the pilot’s perspective, the change is
related to the introduction of data link capabilities to replace some R/T messages.

Primary performance — Capacity: The reduced R/T load per movement allows the safe handling of more
purpose traffic per unit of time within acceptable workload levels. Availability of ADD will
improve the quality of the support provided by automation.

— Cost effectiveness: The reduction of workload allows managing the same volume of
traffic with fewer resources.

— Safety: Transition to the use of data link communications reduces the probability of
misunderstandings and provides a backup for voice communication.

Links to operational — Airspace user operations (AUO)

concept components
— Traffic synchronization (TS)

— Conflict management (CM)

— ATM service delivery management (ATM SDM)

Links to ATM system R07, R27, R31, R98, R148

requirements

— END —