10 WAYS TO

BETTER AVIATION REGULATION

ASSURING SAFETY
OF DEFENCE AVIATION
 Employ Hazard-Based Regulation
1. Ensure regulatory obligations are only imposed to treat threats to aviation
safety.
Maximise Outcome-Based Regulation
2. Where possible, focus regulation on the outcomes needed to treat threats to
safety and not the means of achieving those outcomes.
Take a Purposive Approach
3. Express the purpose of the regulatory obligation simply and clearly and
interpret and apply regulation with its purpose at the foremost of mind.
Utilise Compliance Proofs
4. Define verification criteria against which to assess compliance.

Ensure Sufficient Prescription

5. Decompose outcomes into constituent parts so that obligations are
comprehensively specified and ambiguous principles are avoided.
Provide Comprehensive Explanation
6. Implement a comprehensive and ongoing education program on aviation
safety regulation.
Utilise Safety Indicators
7. Utilise a wide-range of indicators to understand safety performance and
drive continuous improvement.
Apply Risk-Based Oversight
8. Use a robust risk-based assessment process to allocate finite oversight
resources most effectively.
Take a Graduated Response
9. Escalate enforcement remedies to elicit acceptable and compliant behaviour
proportional to the observed behaviour and intent of the regulated entity.
Establish Genuine Engagement
10. Aim to develop mutual respect, appreciating the natural tension between
regulators and the regulated community.
 10 Ways to
Better Aviation Regulation

Assuring Safety of Defence Aviation

Air Marshal Geoff Brown, AO

Defence Aviation Authority

Air Force Headquarters

Canberra

January 2015
 Commonwealth of Australia 2014

This work is copyright. Apart from any use as permitted under the Copyright Act 1968 (Cwth), no
part may be reproduced by any process without prior written permission from the Department of
Defence. Requests and inquiries concerning reproduction and rights should be addressed to
Defence Publishing Service, Department of Defence.

Announcement statementmay be announced to the public.

Secondary releasemay be released to the public.

ISBN 978-1-925062-090

First edition 2014 (reprinted 2015)

Publisher

Defence Publishing Service

Department of Defence
CANBERRA ACT 2600

Author

Christopher De Luis

Acknowledgements

The many members of the Directorate-General Technical Airworthiness Australian Defence Force
(DGTA-ADF) and the Airworthiness Coordination and Policy Agency (ACPA) past and present, who
have contributed to this publication.
10 WAYS TO
BETTER AVIATION REGULATION

FOREWORD
Defence is responsible for self-regulation of its aviation practices and regulation of its industry
suppliers through implementation of the Defence Aviation Safety Program (DASP). The DASP is an
adaptation, with due consideration to the Defence environment and Commonwealth Legislation, of
the International Civil Aviation Organisation (ICAO) State Safety Programme (SSP) standard.

In implementing the DASP, and aligning with government expectations, aviation safety regulation
should aim to minimise unnecessary costs on Defence and industry while achieving the requisite
safety outcomes.

There are many facets to achieving this objective. One involves defining and articulating the
principles upon which our regulatory behaviour is based. Such is the aim of this publication.
But rather than dogma for automatic response, 10 Ways to Better Aviation Regulation aims to
promulgate guidance for intelligent application by regulatory staff. The goal for you the reader is to
guide you in the application of these principles in practice. In doing so you will be supporting
improved safety regulation of Defence aviation.

Air Marshal Geoff Brown, AO

Defence Aviation Authority

10 Ways to Better Aviation Regulation | Foreword iii

10 Ways to Better Aviation Regulation | Foreword iv
10 WAYS TO
BETTER AVIATION REGULATION

CONTENTS

FOREWORD ......................................................................................................................................... III

CONTENTS ........................................................................................................................................... V

UNDERSTANDING THIS PUBLICATION ..................................................................................................... VII

REGULATING AN INTRODUCTION ...........................................................................................................1

1. EMPLOY HAZARD-BASED REGULATION ............................................................................................15

2. MAXIMISE OUTCOME-BASED REGULATION........................................................................................33

3. TAKE A PURPOSIVE APPROACH.......................................................................................................45

4. UTILISE COMPLIANCE PROOFS........................................................................................................53

5. ENSURE SUFFICIENT PRESCRIPTION ...............................................................................................65

6. PROVIDE COMPREHENSIVE EXPLANATION ........................................................................................73

7. UTILISE SAFETY INDICATORS ..........................................................................................................81

8. APPLY RISK-BASED OVERSIGHT ......................................................................................................95

9. TAKE A GRADUATED RESPONSE ...................................................................................................105

10. ESTABLISH GENUINE ENGAGEMENT ..............................................................................................115

10 Ways to Better Aviation Regulation | Contents v

10 Ways to Better Aviation Regulation | Contents vi
10 WAYS TO
BETTER AVIATION REGULATION
UNDERSTANDING THIS PUBLICATION
1
Experience without theory is blind, but theory without experience is mere intellectual play.

RATIONALE OF THE WORK

Regulating is a discipline in its own right. Analysis of regulation has become a theoretical exercise
over the past three decades. Regulating for safety, environmental protection and tax administration
to name a few have much more in common than practitioners often realise. This means there is a
wide variety of sources available for us to benefit from, some theoretical, some more practical.

10 Ways to Better Aviation Regulation communicates our best understanding of the principles to
achieve more effective and efficient regulation of Defence aviation. The work was developed
following a significant period of investigation and research by Defence aviation regulators. This
publication aims to build on the existing body of knowledge including:

The Australian Government Guide to Regulation directs regulation-makers to always

question its need and address its impact early in the regulation-making process.2

The Organisation for Economic Co-operation and Development (OECD), of which Australia
is a member, provides seven guiding principles for regulatory quality and performance.3

The Australian National Audit Office (ANAO) describes a framework to assist regulators in
assessing the quality of their administrative practices and areas of improvement.4

This publication does not set out to replace, or be in conflict with, any of these publications. It aims
to complement them specifically for Defence aviation by:

(i) Providing more relevance achieved by interpretation within the context of safety-related
aviation regulation; and

1
Immanuel Kant (1724-1804).
2
Australian Government (2014) Australian Government Guide to Regulation.
3
Organisation for Economic Co-operation and Development (2005) OECD Guiding Principles for Regulation
Quality and Performance.
4
Australian National Audit Office (2007) Administering Regulation: Better Practice Guide.

10 Ways to Better Aviation Regulation | Understanding This Publication vii

 (ii) Ensuring practicality grounded by twenty years of real regulatory practice in Defence.

Throughout this publication you will find reference to many good sources of information including the
Productivity Commission and various Australian governments. A dash of academic theory is
included from sources including the Australian National Universitys Regulatory Institutions Network
(RegNet), Harvard Universitys Kennedy School of Government and others.

Aviation context is added through reference to authorities including, in no particular order, the United
States Federal Aviation Administration (FAA), European Aviation Safety Agency (EASA), the
Australian Civil Aviation Safety Authority (CASA), the European Defence Agency (EDA), the United
Kingdom Military Aviation Authority (MAA) and the International Civil Aviation Organisation (ICAO).

Lastly practical examples from Australian Defence were provided by regulatory staff.

TARGET AUDIENCE
10 Ways to Better Aviation Regulation is aimed at regulatory staff. It focuses on principles and
practice, and is more to do with advancing behaviours rather than changing regulation. The goal for
you the reader is to apply these principles in practice. In doing so you will be supporting improved
regulation of Defence aviation.

A FOCUS ON REGULATORY PRACTICE

10 Ways to Better Aviation Regulation supports regulatory reform. But the term regulatory reform
means different things to different people so it is worth explaining which aspects are addressed and
which are not. Regulatory reform can be neatly grouped into any of the following subject areas:5

Scope of regulation. Deregulation (removing) and re-regulation (replacing) of existing

regulation, and regulation of emergent issues (new regulation).

Nature of regulation. Implementing alternative forms of regulation, such as the use of self
regulation, voluntary compliance schemes and so on.

Locus of regulation. Altering the balance of centralisation versus decentralisation, such

as levels of regional or local autonomy and the relationship between different regulators.

Behaviour of regulators. The strategies, tactics, operational methods and culture of

regulatory agencies.

This publication focuses on the behaviour of regulators. Whatever improvement is made to the
scope, nature or locus of regulation, it falls to regulatory staff for implementation. The style and
nature of implementation, in other words the behaviour of regulators, can make or break reform.

5
Attributed to Cary Coglianese of the Kennedy School of Government in Sparrow, M (2000) The Regulatory
Craft: Controlling Risks, Solving Problems, and Managing Compliance, the Brookings Institution Press, pp 3.

10 Ways to Better Aviation Regulation | Understanding This Publication viii

STRUCTURE
This publication explains the ten most important behavioural principles to achieve better Defence
aviation safety regulation. Whilst the narrative of this publication is continuous each chapter is
largely self-contained and may be read in isolation. The structure is as follows:

Regulating an Introduction

Chapter 1 Employ Hazard-Based Regulation

Chapter 2 Maximise Outcome-Based Regulation

Chapter 3 Take a Purposive Approach

Chapter 4 Utilise Compliance Proofs

Chapter 5 Ensure Sufficient Prescription

Chapter 6 Provide Comprehensive Explanation

Chapter 7 Utilise Safety Indicators

Chapter 8 Apply Risk-Based Oversight

Chapter 9 Take a Graduated Response

Chapter 10 Establish Genuine Engagement

Those familiar with regulatory policy know it is important to bridge the gap between theory and
practice. So after commencing with a summary each chapter consists of three forms of content:

(i) In-Theory. Consisting of a summary of relevant regulatory theory. Most useful to readers
who prefer to develop a deeper understanding by focussing on the why.

(ii) In-Context. Here the regulatory theory is translated into language more readily applied to
Defence aviation safety regulation.

(iii) In-Practice. Examples of snapshots of time in the past, relating the principles with real
examples from Defence. In-practice examples appear as breakout boxes.

CHAPTER OUTLINE

REGULATING AN INTRODUCTION
The safety authority is a unique organisation, neither delivering products nor services in the
traditional sense. Rather the aim of a safety authority is to deliver safety assurance within complex
environments involving many stakeholders. This is the basis of the introductory chapter Regulating
an Introduction. Given the decision to regulate, which is but one means of a safety authority, we
introduce the three activities involved in regulation: to regulate, to conduct oversight and to
conduct enforcement.

10 Ways to Better Aviation Regulation | Understanding This Publication ix

Secondly, the importance of a regulator and its separation from those it regulates should be
recognised. The primary reason is to ensure that judgements can be made, and enforcement
actions taken, without pressure from interests that may conflict with the regulators primacy on
safety. This is referred to as regulatory independence. The credibility of the regulators authority,
particularly in non-legislatively backed regimes, depends in part upon whether the authority is
regarded as an independent decision maker.

Thirdly, a safety authority never delivers safety assurance alone. Consideration is given to the many
stakeholders in the regulatory process, in addition to those regulated, who may be relied upon by
the safety authority. Regulation never starts with a clean sheet after all; the space is often already
full of regulation. Regulators need to be aware of this and exploit it. This is referred to as
recognition.

Finally, the chapter concludes by describing a regulatory model to aid understanding of the ten
principles subject of this publication.

CHAPTER 1 EMPLOY HAZARD-BASED REGULATION

Hazard-based regulation means ensuring regulatory obligations are only imposed to treat threats to
aviation safety. The rationale behind hazard-based regulation is that it focuses the attention of all
those involved in the regulatory process on aviation safety and reduces regulatory burden by
avoiding non-safety related regulation. Hazard-based regulation is achieved by using a risk model to
analyse hazards that may materialise and prescribing regulations and standards to assure safety.

CHAPTER 2 MAXIMISE OUTCOME-BASED REGULATION

Maximising outcome-based regulation means, where possible, focussing regulation on the
outcomes needed to treat threats to safety and not the means of achieving those outcomes.
Outcome-based regulation is a style that allows for a range of acceptable solutions rather than
imposing or presuming single means of compliance. The result is flexibility for the regulated
community to develop and implement solutions while achieving the same level of safety assurance.

CHAPTER 3 TAKE A PURPOSIVE APPROACH

Purposive regulation means expressing the purpose of the regulatory obligation simply and clearly
and interpreting and applying regulation with its purpose at the foremost of mind. The benefit of
purposive approach is that it increases clarity as to the purpose of the regulation, thereby protecting
against tactical, black and white or nonsensical interpretation disputes between members of the
regulated community and the regulator.

CHAPTER 4 UTILISE COMPLIANCE PROOFS

Compliance proofs are verification criteria against which to assess compliance. Compliance proofs
define specific, measurable, demonstrable and repeatable verification requirements for each

10 Ways to Better Aviation Regulation | Understanding This Publication x

regulation to support unambiguous compliance determination. The benefit of compliance proofs is
certainty as to the compliance requirements, thereby minimising compliance disputes and
unnecessary gold plating.

CHAPTER 5 ENSURE SUFFICIENT PRESCRIPTION

Sufficient prescription means decomposing outcomes and compliance proofs into sub-outcomes
and sub-proofs so that obligations are comprehensively specified. Such prescription ensures clarity
and certainty of outcome-based regulation and avoids high-level and ambiguous principles.

CHAPTER 6 PROVIDE COMPREHENSIVE EXPLANATION

Implement a comprehensive and ongoing education program on aviation safety regulation.
Comprehensive explanation supports consistent interpretation and application of regulation by all
stakeholders in the regulatory process, protecting against conservative application inherent with
poor understanding of regulation.

CHAPTER 7 UTILISE SAFETY INDICATORS

For continuous monitoring and assessment of safety performance, the regulator should utilise a
wide-range of safety indicators to understand safety performance and drive continuous
improvement.

CHAPTER 8 APPLY RISK-BASED OVERSIGHT

To best utilise finite resources apply a robust risk-based assessment process to allocate oversight
resources most effectively. This means treating each member of the regulated community differently
based on management of the risks of noncompliance. The result is improved utilisation of scarce
resources by focussing resources according to risk.

CHAPTER 9 TAKE A GRADUATED RESPONSE

Ensure enforcement action is fair by escalating remedies to elicit acceptable and compliant
behaviour proportional to the observed behaviour and intent of the regulated entity. Graduated
response incentivises generative safety culture within members of the regulated community rather
than severely penalising minor infractions to the detriment of honest reporting.

CHAPTER 10 ESTABLISH GENUINE ENGAGEMENT

Appreciating the natural tension between regulators and the regulated community, aim to develop
mutual respect through genuine engagement using formal and informal approaches.

10 Ways to Better Aviation Regulation | Understanding This Publication xi

10 Ways to Better Aviation Regulation | Understanding This Publication xii
10 WAYS TO
BETTER AVIATION REGULATION
REGULATING AN INTRODUCTION
WHY REGULATE?
Social and informal mechanisms are basic modes of regulating behaviour. In a social context
behaviour is often regulated by personally asking someone to do (or not do) something. But when
these mechanisms prove inadequate, and the outcome results in negative effects on a community-
wide scale, someone or something is sought to act on behalf of individuals for the greater good.

When a government or some other empowered body makes a decision to regulate they seek to
bring about a social state better than without regulation. They do this by placing obligations on
individuals and organisations. Examples of where regulation is common include preventing market
failures, threatening of public confidence or harm to people, property or the environment.

WHAT IS REGULATION?
Regulation has no single definition. Some definitions are broad, capturing mechanisms of social
control or influence which may not explicitly be written. Some are much narrower, such as legal-
system definitions focused on legislation.

In this publication regulation refers to the diverse set of instruments used by government and other
authorities to influence or control the way people and organisations behave where there is at least a
reasonable expectation of compliance. It consists of 3 broad functions:

1. Developing, enacting and refining regulations promulgated via a regulatory instrument

placing obligations on the appropriate recipients to achieve the regulatory objective;

2. Verifying compliance with such regulations; and

3. In the event of departure from those obligations by those under regulatory oversight,
enforcing the established regulations by imposing appropriate corrective measures.

In short:
6
Regulation is any rule endorsed by government where there is an expectation of compliance.

6
Australian Government (2014) Australian Government Guide to Regulation, Canberra.

10 Ways to Better Aviation Regulation | Regulating an Introduction 1

INSTRUMENTS OF REGULATION
Regulation is enacted through a regulatory instrument. Instruments of regulation can be broadly
categorised along a continuum of government intervention, ranging from legislation to self regulation
as shown in Figure 1. Each form has advantages and disadvantages. The intention of this
publication is to inform the reader of the various forms of regulation, but not to compare them.7

Level of Government Intervention

Higher Lower

Legislation Quasi Regulation Co-Regulation Self-Regulation

Figure 1. The continuum of government control by regulatory instrument.

Primary legislation is law made by a State or Federal Parliament. An example is the Corporations
Act 2001 (Cth) which is an act of the Commonwealth of Australia that sets out the laws dealing with
business entities in Australia at federal and interstate level.

Delegated Legislation is law made by an authority to which Parliament has delegated part of its
legislative power in order to administer the requirements of the primary legislation. For example in
Australian civil aviation the federal Department of Infrastructure and Regional Development, the Civil
Aviation Safety Authority (CASA) and Airservices Australia administer delegated legislation under
powers given to them in the Civil Aviation Act 1988. An example instrument of delegated legislation
administered by CASA is the Civil Aviation Safety Regulations (CASR) 1998.

Quasi Regulation encompasses those rules, instruments and standards by which government
influences business to comply, but which do not form part of explicit government legislation.
Examples can include government endorsed industry codes of practice or standards, government
issued guidance notes, industry-government agreements and national accreditation schemes. An
example is the agreement between Telstra, Optus and Primus to voluntarily filter a list of websites
known to contain material relating to child abuse. The list is compiled and maintained by the
Australian Communications and Media Authority (ACMA).

Co-Regulation (or Enforced Self Regulation) is a hybrid in which industry develops and administers
particular codes, standards or rules, but the government provides formal legislative backing to
enable the arrangements to be enforced. Regulation of radio and television content is co-regulatory.
In this example industry groups develop codes under the Broadcasting Services Act 1992 (Cth), in
consultation with the ACMA. Most aspects of program content are governed by these codes, which

7
For further information on the various instruments see Australian Government (2007) Best Practice Regulation
Handbook, Canberra, appx A.

10 Ways to Better Aviation Regulation | Regulating an Introduction 2

include the Commercial Television Industry Code of Practice and the Commercial Radio Australia
Code of Practice and Guidelines. Once implemented, the ACMA monitors these codes and deals
with unresolved complaints made under them.

Self Regulation is an arrangement in which an organised group (such as an industry association)

regulates the behaviour of its members. For example, the content of advertising is subject to a self-
regulatory system created by the Australian Association of National Advertisers (AANA). The AANA
manages a Code of Ethics and the Advertising Standards Bureau (ASB) incorporates an
independent Advertising Standards Board to hear complaints regarding advertising content.

Box 1: In-practice instruments of Defence aviation safety regulation

The Defence aviation safety regulations are a system of self regulation and quasi
regulation with the aim of ensuring Defence aviation is conducted at acceptable level of
risk of harm to personnel or property. Whilst the regulations are all promulgated in the
same suite of documents the type of regulatory instrument according to the previous
definitions is determined by the two target groups comprising the regulated community.

DEFENCE UNITS AND PERSONNEL

Defence is self regulating under the authority of the Defence Aviation Authority (DAA).
The authority of the DAA to regulate Defence and its personnel stems from:

(i) Chief of Defence Force as a general order to Defence members under the
Defence Force Disciplinary Act 1982, and

(ii) Secretary of the Department of Defence as a lawful and reasonable direction to

Defence employees under the Public Service Act 1999.

Noncompliance with the aviation regulations could lead to disciplinary action of the
responsible individual under the applicable Act.

COMMERCIAL ORGANISATIONS AND STAFF

Defence is quasi regulating under the authority of the DAA. The authority of the DAA to
regulate commercial organisations stems from a contract between each commercial
organisation and the Commonwealth of Australia which includes scope to comply with the
Defence aviation safety regulations.

Noncompliance with the regulations could constitute a breach of contract and lead to
remedies under contract law. The regulations are not applicable to sub-contractors unless
a suitable contractual arrangement exists.

10 Ways to Better Aviation Regulation | Regulating an Introduction 3

SAFETY REGULATION
In any human endeavour involving a complex system such as aviation, a lack of safety comes about
because those involved fail in acting with the required diligence, or because they were unable to see
or influence existing or emerging threats. Furthermore organisational and environmental influences
contribute to latent conditions that can subvert or bypass existing controls.

Safety regulation aims to correct this to assure the most important controls are not reduced or
subverted. Too little safety is clearly a bad thing and is unacceptable. Improving safety by
constraining the unsafe activity may also be unacceptable as it may prevent the desired outcome
(which presumably is of benefit otherwise it would not have been attempted in the first instance).
Safety regulation is correctly targeted when it achieves the required level of safety, and no more.

ROLE OF AN AVIATION SAFETY AUTHORITY

In civil aviation, the promotion of safety is the underlying philosophy of all aviation regulation across
the world. The Convention on International Civil Aviation was signed in Chicago in 1944, and it
created the International Civil Aviation Organisation (ICAO), which in 1947 became a specialised
Organisation of the United Nations. The Convention focuses on international civil aviation.8

ICAO requires member states, Australia included, to establish civil airworthiness organisations who
should manage regulations, policy and guidance, conduct surveillance, investigations and
enforcement, staffing and training.9 In developing regulations the regulator has the option of
adopting provisions which will govern its role in the implementation of the regulations. This may
range from highly active to passive.

In the active role, a close day to day interest would be taken in the direction and control of all
airworthiness matters through an inspection organisation. This could be so rigorous as to dominate
and dictate conduct of all airworthiness activities leading to undermined personnel, lowered safety
and increased cost, and confusion regarding who is responsible for actions.

In the passive role the regulator would intervene only to institute action when a violation of the
regulations has occurred. The regulator could leave interpretation and implementation of the
regulations up to those regulated, relying upon their competence to interpret correctly and
encouraging compliance through threat of enforcement only.

In practice neither is solely compatible with equitable and effective division of responsibility between
regulators and regulated. The ICAO position is that considerable merit exists for a regulatory system
which has elements of both extremes, and will:

8
While focussing only on international civil aviation, ICAOs role overseeing the majority of the worlds aviation
regulators, its wealth of publically available information and lack of a military equivalent leads it as an obvious
reference.
9 rd
For further information see ICAO (2014) Airworthiness Manual Doc 9760 3 edn.

10 Ways to Better Aviation Regulation | Regulating an Introduction 4

 (i) Represent a well balanced allocation of responsibility between the State and those persons
10
or organisations conducting airworthiness-related activities;

(ii) Be capable of economic justification within the resources of the State;

(iii) Enable the State to maintain continuing regulation and supervision of the airworthiness
activities of the operator, manufacturer and maintenance facility without unduly inhibiting
their effective direction and control of their organisations; and

(iv) Result in the cultivation and maintenance of harmonious relationships between the State
and those persons/organisations applying regulations in practice.

These characteristics are just as applicable to military aviation.

Box 2: In-practice objective of Defence aviation safety regulation

Defence requires capability that delivers a required operational outcome in a nominated

environment, within a specified time, and the ability to sustain that effect for a designated
period. Central to this capability is the safe operation of airworthy aircraft.11

While Defence aviation is explicitly excluded from Australian civil aviation legislation
Defence chooses to regulate organisations and people involved in Defence aviation with
the aim of safety, acknowledging that the required operational outcomes must be
weighed against the potential for harm to people and/or property. Preventing market
failures or other public policies often the subject of regulation, such as ensuring
appropriate expenditure of Commonwealth funds, is not within the mandate of the
Defence aviation safety regulations.

Within the regulatory system, the role of Defence aviation safety regulators is to assure
safety, which means to achieve confidence and reassurance by monitoring and reporting
on those responsible for safety. The role of the regulator is not to ensure everything is
safe by either doing all the work themselves, or checking that it is done the way they
would have done it. The regulator is not 100% responsible for all actions taken by those
regulated.12

10
Here State refers to ICAO contracting states countries who are signatories to the Convention on International
Civil Aviation (the Chicago Convention).
11
Defence Instruction (General) OPS 02-2 Defence Aviation Safety Program.
12
To assure is to give confidence or to reassure, by monitoring. To ensure is to make happen.

10 Ways to Better Aviation Regulation | Regulating an Introduction 5

INDEPENDENCE AND RECOGNITION

INDEPENDENCE OF REGULATORY AUTHORITIES

There must be thorough independence throughout the regulatory regime, in particular in the
13
setting of safety and airworthiness policy, regulation, auditing and enforcement.

Human behaviour in any work system is shaped by objectives and bounded by financial, resource
and safety constraints (in addition to ethical and moral issues). Breach of any of these boundaries is
undesirable. Whether defending the country or earning a profit, organisations with an objective
(military or civilian, government or commercial) will tend to establish gradients away from financial
and resource boundaries. Organisations will drift towards the safety boundary as they seek
efficiencies with finite resources or for other reasons pertaining to organisational culture.

This is not for dispute. For example, the investigation into the loss of the Royal Air Force Nimrod
XV230 in 2006 resulting in the deaths of 14 service personnel found the Nimrod IPT [Integrated
Project Team] allowed operational pressures and workloads to detract or distract from safety
tasks.14

In a well designed and managed safety system organisations will employ numerous measures to
prevent drifting towards the safety boundary. But rather than rely on those measures alone, aviation
authorities prevent breaches of the safety boundary by placing obligations on those responsible.
They do this by regulating organisations and their personnel.

The independence of the regulatory decision making process distinguishes regulation from other
forms of influence. Independence of safety regulators ensures a primacy on safety at all times and
confers permanence of decisions.15 Whereas the regulated community achieves its objectives
through managing resources, remaining financially viable and doing it safely, the sole objective of
the regulator is safety.

The existence of a safety regulator does not mean those regulated view safety with any less
importance than the regulator. But when subjected to competing priorities organisations can and
sometimes do behave in a manner that is prejudicial to safety in ways that individuals within the
organisation would not condone. But the very existence of an objective by definition cannot allow a
single and permanent primacy on safety at all times. An independent regulator is not bound by this
restriction. They have no other reason for being.

13
Haddon-Cave, C. (2009) The Nimrod Review, an independent review into the broader issues surrounding the
loss of the RAF Nimrod aircraft MR2 XV230 in Afghanistan in 2006, The Stationary Office Limited, London,
para 20.18.
14
Ibid, para 11.257 (square brackets added).
15
Rasmussen, J (1997) Risk Management in a Dynamic Society: A Modelling Problem, Safety Science vol 27
no 2/3.

10 Ways to Better Aviation Regulation | Regulating an Introduction 6

Independence of a safety authority is a defining characteristic which distinguishes regulation from
governance or day-to-day line management. Notwithstanding the definition of regulation at the start
of this chapter, regulation cannot exist where independence from organisational influence cannot be
established. Only when sufficient independence is established can the authority be considered a
regulator.

Independence is not a binary characteristic, rather a scale that varies from no independence to full
independence. Figure 2 is an example five-level scale, but it could consist of any number of levels.

Degree of Independence
Lower Higher

Worker Manager Corporate Self/Quasi- External

Governance Regulator Regulator

LESS REGULATORY MORE REGULATORY

Figure 2. An example of the variation in regulatory independence.

A worker within an organisation has very little independence from organisational influence. This is
expected as workers should be under the direction of management to align with organisational
goals. Manager and corporate governance roles have increasing independence and authority as
their ability to resist organisational influences driving them towards the safety boundary increases.
But they are still part of the organisation meaning the level of independence required to focus solely
on safety at all times remains limited. None of these positions hold a level of independence
necessary to be a safety regulator.

Depending on the specific circumstances a self- or quasi-regulator role may achieve a level of
independence adequate to be a safety regulator. An external regulator is further independent from
organisational influence.

Independence of the regulator alone is not everything and does not necessarily equate to improved
safety. Rather a regulator with more independence is more likely to avoid organisational influence
by those it seeks to regulate. Strong independence but insufficient interaction with those it regulates
results in an uninformed and increasingly irrelevant regulator. This was surmised by Sir Charles
Haddon-Cave following Nimrod, in regards to the independence of a military aviation authority:

the military element adds an entirely different dimension to the picture. The MOD [UK Ministry
of Defence] has the responsibility for delivering a certain military capability and balancing risk
with task. A military organisation must be risk sensible but not too risk averse. The MAA
[Military Aviation Authority] must understand and appreciate operational relevance and,
importantly, be seen by military operators to understand and appreciate this, if it is to enjoy their

10 Ways to Better Aviation Regulation | Regulating an Introduction 7

 confidence. In my view, it would not be sensible or practicable to position the MAA legally and
16
physically outside the MOD.

The takeaway is that the degree of independence of the regulator is an important characteristic of a
regulatory regime that should always be consciously understood and managed appropriately.

RECOGNITION OF OTHER REGULATORY AUTHORITIES

A good regulator will do its utmost to ensure the obligations it places on the regulated community
are most likely to be followed. One of the best ways to achieve this is by harnessing existing
resources which support the objectives of the regulatory regime. In other words reuse the work of
other regulators as much as possible. Furthermore, it is not always practical or cost effective for a
regulator to undertake direct oversight over its own regulation. In such cases it may rely on another
body to undertake oversight on its behalf. In some cases it may not even be practicable to directly
regulate products or services obtained from a foreign nation.

All of these scenarios give rise to the concept of regulatory recognition. Recognition means being
aware of the existing regulatory space and relying on existing arrangements as much as possible.
The regulator should aim to steer, rather than row.

Recognition may be utilised to regulate for a particular set of hazards such as work, health and
safety (the safety regulator may choose not to regulate as it is already regulated by legislation); for a
particular activity such as oversight of quality management which relies on third party certification; or
recognising the authority of an entire regulatory system such as a certification of an aircraft by the
United States Federal Aviation Administration.

Recognition of other safety authorities can be achieved in three types of ways. Unilateral
recognition is where an authority uses services of another regulatory authority, either with or
without their knowledge, and without any specific tailoring. An example would be where an aviation
safety regulator availed themselves of aviation standards and information made publicly available by
another safety authority.

Bilateral recognition is where specific agreements are put into place between two authorities
covering services provided to, or between, authorities and the conditions under which those services
can be used. An example of a bilateral agreement would be where two authorities agreed to share
specific design and support data for a common aircraft type.

Multilateral recognition is where specific agreements are put into place between three or more
authorities covering services provided to, or between them, and the conditions under which that
service can be used. Most of the Air and Space Interoperability Council (ASIC) and North Atlantic
Treaty Organization (NATO) interoperability agreements are multilateral agreements.

16
Haddon-Cave, C. (2009) The Nimrod Review, an independent review into the broader issues surrounding the
loss of the RAF Nimrod aircraft MR2 XV230 in Afghanistan in 2006, The Stationary Office Limited, London,
para 21.16 (square brackets added).

10 Ways to Better Aviation Regulation | Regulating an Introduction 8

 Box 3: In-practice TAREG 3.2.6.a.1 Quality Management Systems

An organisation-wide quality management system is fundamental to any assignment of

engineering authority in Defence as it establishes a level of control and consistency over
an organisations activities. Rather than re-regulate for the specifics of a quality system,
Defence technical airworthiness regulation 3.2.6.a.1 requires engineering organisations
to reuse an independent certification such as AS/NZS ISO 9001 or equivalent. This is a
form of recognition. Defence recognises the existence of ISO9001 rather than regulating
directly. Oversight and enforcement is provided by a third party auditor.

Since ISO 9001 does not meet all of the technical airworthiness requirements itself
Defence specifically regulates for additional requirements and refers to it as an
engineering management system (EMS).

INDEPENDENCE AND RECOGNITION

The 3 broad functions to regulate, to oversight and to enforce are depicted in Figure 3.17 It is
important to realise each function need not be undertaken by the same organisation or personnel.
For example, in one case the regulator may regulate, conduct oversight and carry out enforcement.
In another circumstance the regulator may have limited resources and instead rely on another
organisation to conduct oversight and enforcement on their behalf. Each function therefore may
have a different level of independence in decision-making depending on the organisation
responsible and their circumstances.

While there may only be one overall regulatory authority, by combining the characteristic of
recognition it is quickly understood that regulation, oversight and enforcement of any safety domain
including aviation, whether military or civil, is a patchwork of multiple organisations performing
regulatory functions, each with a degree of independence as depicted in Figure 4. Recall that
independence allows a primacy on safety. The regulatory regime therefore is only as independent
as the least independent of the organisations involved in the regulatory functions.

17
In common language Oversight is not often used as a verb however its use here originates from safety
oversight, an aviation term meaning to ensure effective implementation of safety-related standards and
regulations.

10 Ways to Better Aviation Regulation | Regulating an Introduction 9

 DEGREE OF INDEPENDENCE

REGULATE

OVERSIGHT

ENFORCE
Worker Manager Corporate Self/Quasi- External
Governance Regulator Regulator

Figure 3. Each of the regulatory functions may have differing levels of decision-making independence.

DEGREE OF INDEPENDENCE
Regulatory Regulator Regulator
Authority B D Regulatory Authority

REGULATE Regulator B

Regulator
E Regulator C

OVERSIGHT Regulator D

Regulator E

ENFORCE Regulator F

Regulator Regulator
C F Corporate Self/Quasi- External
Worker Manager
Governance Regulator Regulator

Figure 4. Each organisation responsible for a regulatory function may have a unique level of independence. In
this example the regulatory authority relies on organisation B to prescribe a particular area of regulation and
conduct oversight of that regulation (hence they are referred to as regulator B). The regulatory authority must be
comfortable the level of independence of regulator B is satisfactory to ensure a primacy on safety.

10 Ways to Better Aviation Regulation | Regulating an Introduction 10

A MODEL OF REGULATION

PURPOSE
Models help to understand complicated structures of activities and stakeholders. The regulatory
model explained here expands the three regulatory functions introduced at the start of this chapter.
The model is referred to at the start of each subsequent chapter of this publication.

The characteristics of independence and recognition, whilst critical to understanding the role of a
regulator are not included in the model as they provide no further help in explaining any of the
subsequent chapters.

DESCRIPTION
It should already by clear that the regulatory process includes two groups of actors. The regulators
and the regulated community (individually referred to as a regulated entity).

A safety regulator is responsible for the following elements in the regulatory process:

(i) Regulate. Prescribing and amending regulation to place obligations on the appropriate
recipients to achieve the regulatory objective.

(ii) Educate. Providing explanatory material to support correct interpretation and effective
implementation to satisfy the regulation.

(iii) Oversight. Undertaking various activities to ensure that each regulated entity, and the
products or services they produce, is compliant with the regulation. Oversight
encompasses both the review that is done when issuing an approval for the first time and
the surveillance thereafter.

(iv) Enforce. Activities undertaken to ensure compliance when noncompliance is observed.

(v) Manage Relationship. The various activities involved in interacting with each member of
the regulated community. The manner in which a regulator conducts itself can be as
important to the outcome as the content of the regulation itself. This is why all of the other
elements should be thought of as flowing through the manage relationship element.

Each regulated entity is responsible for the following elements in the regulatory process:

(i) Interpret. Understanding the intent of the regulation within their business context.

(ii) Implement. The business activities undertaken to satisfy the regulation.

(iii) Adhere and Monitor. Continuing to adhere with the regulation and undertaking various
activities to remain regulatory compliant.

(iv) Manage Relationship. The regulated entity manages their relationship with the regulator
as they see fit, based on their level of trust, experience and other factors.

10 Ways to Better Aviation Regulation | Regulating an Introduction 11

Other than when establishing a new regulatory system, these elements and the activities they
comprise are undertaken concurrently and need not be sequential. Furthermore, the regulator would
implement feedback mechanisms to improve each element.

The regulatory model is not intended to be a work flow diagram. Rather, the aim of this model is to
assist in explaining the content of this publication, and will be referred to from time to time.
Regulatory staff need not understand which element they are working on at any particular time. The
model is merely a way of describing what the two key stakeholders in a regulatory process do and
how they interact.

Furthermore it is not the intention of this publication to explain all the activities within each element.
Rather this publication looks at some key areas of focus to achieve better regulation, commencing
with the next chapter on hazard-based regulation.

Regulator Regulated Entity

REGULATE

INTERPRET
MANAGE RELATIONSHIP

MANAGE RELATIONSHIP

EDUCATE

IMPLEMENT

OVERSIGHT

ADHERE AND MONITOR

ENFORCE

Figure 5. The regulatory model.

10 Ways to Better Aviation Regulation | Regulating an Introduction 12

10 Ways to Better Aviation Regulation | Regulating an Introduction 13
10 Ways to Better Aviation Regulation | Regulating an Introduction 14
1. EMPLOY HAZARD-BASED
REGULATION

SUMMARY

WHAT
Ensure regulatory obligations are only imposed to treat threats to aviation safety.

WHY
Without a proactive approach the regulator is responding to safety incidents in a reactive
manner. A better way is to combine past learning with a proactive stance. The effect is increased
focus on safety and reduction in cost of compliance by avoiding non-safety-related regulation.

HOW
1. Use a safety risk model to identify hazards which may eventuate into unsafe conditions.

2. Impose obligations through regulation and standard setting to enact defences to prevent the
realisation of those hazards or their escalation into unsafe conditions.

Hazard-based regulation is a characteristic of the REGULATE function in the regulatory model.

Regulator Regulated Entity

REGULATE

INTERPRET
MANAGE R ELATIONSHIP

MANAGE RELA TIONSHIP

E DUCATE

IMPLEMENT

OVERSIGHT

ADHERE

ENFORCE

10 Ways to Better Aviation Regulation | 1. Employ Hazard-Based Regulation 15

 EMPLOY HAZARD-BASED REGULATION IN THEORY

INTRODUCTION
Knowing the scope and manner of what to regulate must be at the heart of being a good regulator.
So how does a regulator ensure their regulations remain focussed on the objective?

Many factors will influence the regulator in its undertaking. First is a regulators core idea about the
nature of the world. If the regulator thinks the regulated community is immature or untrustworthy
then it will administer prescriptive and punitive regulation. Conversely belief that the community is
mature, competent and has the right attitude is likely to yield less prescriptive or punitive regulation.

Second is the ability to make decisions under conditions of uncertainty. When faced with an
uncertain level of risk of a particular hazard or hazards, should the regulator adopt a worse case
presumption regarding the harms of activities? Or adopt a more positive stance to avoid potentially
burdensome catchall regulation, leaving the decisions about levels of risk retainment to others.

Third is whether the regulator is confident enough to be proactive rather than reactive. Without a
proactive strategy to manage the stock of regulation over time the result is an ever increasing and
potentially burdensome regulatory stance.

Finally, what are other similar regulators doing?

Unfortunately neither regulatory theorists nor practitioners have categorised the different
approaches neatly. A review of approaches to safety regulation yields a bewildering array of
language and terminology like those below. Such is the field of safety regulation. This chapter
explores some of the above concepts and aims to explain a preferred, broad approach to aviation
safety regulation known as hazard-based regulation. The terminology shaded in grey is discussed in
later chapters.

Outcome-based Purposive approach Rules-based

Prescriptive
Hazard-based Principles-based

Literal approach Risk-based Performance-based

Management-based

KEY TERMINOLOGY
This publication intentionally avoids defining new terminology. However it is important to have a
consistent understanding of language particularly in safety and risk communication. For this reason
the following definitions are provided, according to ICAO.18

18 rd
ICAO (2013) Safety Management Manual Doc 9859 3 edn, s 2.13 and Definitions.

10 Ways to Better Aviation Regulation | 1. Employ Hazard-Based Regulation 16

A hazard is a source of potential harm. An aviation hazard is a condition with the potential to cause
or contribute to unsafe operation of aircraft or aviation-safety-related equipment, products or
services. A hazard should not be confused with a consequence. A consequence is an outcome of
an event triggered by a hazard, and may be multilayered, including such things as an intermediate
unsafe outcome before an ultimate consequence (accident). By first defining the hazard one can
then project the consequence. Safety risk is a different concept altogether, being the likelihood and
severity of the consequence of a hazard.

A safety threat represents a specific failure or loss of control mode through which a hazard can
materialise. Threats may be thought of as system or equipment failure modes identified through a
structured review process which act to defeat the protection of a hazard.

Hazards are an inevitable part of aviation. However their manifestation and possible consequences
can be addressed through various mitigating actions. A defence is a broad term meaning any
specific mitigating action, barrier, control or recovery measure put in place to prevent the realisation
of a hazard or its escalation into an undesirable consequence.

As an example, a strong wind blowing parallel to the runway is not necessarily a hazardous
condition and will improve take-off and landing performance. However the same wind blowing in a
direction perpendicular to the runway creates a crosswind condition that may be hazardous. Multiple
threats could be identified in the crosswind landing example. One example is the loss of control on
landing with a consequence of wing strike with the ground. The ultimate consequence could be a
catastrophic accident. A limitation on the maximum allowable crosswind landing is an example of a
defence to prevent loss of lateral control (the threat) in a strong crosswind (the hazard).

REGULATORY STANCE
Accidents or incidents are typically the malevolent coming together of a set of events and
circumstances. It is this coming together of all the elements that in itself, creates the critical unsafe
state of the system. For almost every aviation accident or incident the subsequent systemic
investigation has shown that:19

(i) The main contributing systemic factors were present before it happened.

(ii) In most cases they were relatively common knowledge, and had often been formally
documented.

(iii) In all cases, they could have, and should have, been identified and rectified before the
accident or incident.

19
Lee, R (2009) Maintaining and Enhancing Operational Safety in a Turbulent Aviation Environment: the Critical
Role of Integrated Safety Management Systems, in Proceedings of the Safeskies Conference, Canberra.

10 Ways to Better Aviation Regulation | 1. Employ Hazard-Based Regulation 17

An accident therefore can be viewed as a sequence of events that unfold over time. Contributing
20
factors can be traced back by unfolding the chronology of events as depicted in Figure 6. Each
stage in the chronology of an event is a potential intervention point. The stance of a regulator is
either reactive or proactive depending on when it intervenes.

The Event
Precursors Accident, incident
or near miss

Time

PROACTIVE REACTIVE
REGULATION REGULATION

Figure 6. A regulators stance is either reactive or proactive depending on when it intervenes.

REACTIVE REGULATION
The reactive approach to regulation is built on historical events. Each accident or incident is
reviewed and additional regulatory controls are put in place to prevent the same or similar hazards
from manifesting again, only limited by cost. One of the appeals of reactive regulation is the ease of
measuring regulatory success. Numbers of repeat incidents, accidents, or violations provide
unambiguous and objective measures.21

Over time the stock of regulation grows and continues to grow as there is always a reluctance to
remove existing regulatory interventions for fear of releasing a risk back into the regulated
community. The result is a stock of regulation at any point in time which is the end result of a
sequence of previous events. However, without a clear connection as to its purpose reactive
regulation over time provides little or no awareness of the hazards which it was designed to control.
The existence of reactive regulation can deaden the awareness of the hazards which the rules
intended to control.

20
Sparrow, M (2008) The Character of Harms: Operational Challenges in Control, Cambridge University Press,
pp 137.
21
Ibid, pp 136.

10 Ways to Better Aviation Regulation | 1. Employ Hazard-Based Regulation 18

PROACTIVE REGULATION
The proactive approach involves sliding up the chronology of any potential event before it occurs,
examining the full range of precursors, and the precursors to the precursors, searching for
opportunities to remove, restrain, or divert some essential ingredient or ingredients and thus
reducing the possibility of the hazard materialising. This systematic approach to regulation has
driven the need for safety management systems an example of proactive regulation.

Proactive regulation means regulating uncertainty. How does the regulator know what is going to
happen in the future? Conceptually this may be classified into three ideal type categories: 22

1. The harm the activity will cause is known and determinate. For example intentional
incorrect fitment of an aircraft wheel will certainly result in an unsafe condition.

2. The harm is probabilistic in character. In other words certainty is impossible to know but it
can be estimated based on probability theory. For example the safe life approach to design
of structurally critical aircraft components is probabilistic. Safe life components are
designed with an accepted, albeit extremely low, probability of failure through thorough
testing and analysis.

3. The probability of harm occurring and/or its magnitude is indeterminate. For example
maintenance performed by personnel with insufficient experience may contribute to harm at
some time in the future. However the likelihood or consequence of this hazard manifesting
is almost impossible to determine to any degree of accuracy.

Aviation lends itself mostly to type 3 cases due to the complex nature of technological, human and
organisational factors many of which are difficult to model. Regulation to avoid harm due to type 3
cases may be referred to as hazard-based regulation. Type 2 cases in aviation are less common,
usually in the design and certification domain where engineers are able to model failure modes to
some degree. Type 2 regulation may be referred to as risk-based regulation. Type 1 cases are the
realm of deliberate acts or gross omissions or neglect that are so obvious so as to cause harm they
are typically not directly regulated by Defence aviation safety regulation. Such instances are the
domain of broader legislation or the common law and therefore not discussed further.

Proactive regulation, whether risk or hazard based will have to rely on a constant flow of information
and data, which, following analysis, should provide an indication of the actual safety performance
including identification of emerging hazards before they cause harm. Safety indicators are
discussed in chapter 7.

What makes the discussion rather more complicated, however, is in regulatory theory the term risk-
based regulation is overused to mean almost anything while hazard-based regulation is used rarely.
A short description of each therefore follows.

22
Stewart, R (2002) Environmental Regulatory Decision Making Under Uncertainty, Research in Law and
Economics, vol 20.

10 Ways to Better Aviation Regulation | 1. Employ Hazard-Based Regulation 19

RISK-BASED REGULATION
Risk-based regulation is a term utilised widely but differs in meaning. In this publication we mean the
application of a systematic framework that prioritises and prescribes regulation to manage hazards
on an evidence-based assessment of safety risk.

The benefit of a risk-based approach is less burdensome regulation. However the ability to assess
risk accurately requires statistical distributions. Risk-based regulation is therefore the preferred
approach in fields where risk can be clearly understood such as in the use of chemical substances
where it is possible to determine probabilistic estimates of the safety risks associated with their use.

An example is the Therapeutic Goods Administration (TGA). The TGA is the Australian Government
entity responsible for regulation of medicines and medical devices. The use of any therapeutic good
carries health risks. One of the roles of the TGA is to regulate therapeutic products based on
scientific and clinical assessment of the evidence of those risks compared to their benefits. As an
example some medications to control high blood pressure may include side effects such as a
persistent cough. However, the risk of this irritation is balanced against the possibility of a life-
threatening heart attack if the medication is not used.

The amount of regulatory control needed to manage risks depends on the product and determines
how consumers gain access to the product. A low-risk product may be safely sold in a supermarket;
higher risk products may only be supplied by prescription after consultation with a health
professional. A medication with unacceptable risks will not gain approval to be sold at all.

Unfortunately risk-based regulation does not accommodate low occurrence, high consequence
events particularly well, as is prevalent in aviation and nuclear power generation industry. In such
scenarios the probabilities needed are often not available and cannot be estimated, resulting in a
default to hazard assessment (type 3).

In this publication risk-based regulation is not taken to include risk-based oversight. Risk-based
oversight, discussed in chapter 8, is an approach to the allocation of resources to the oversight role
based on an assessment of risk. They are different regulatory approaches as explained in Figure 7.

10 Ways to Better Aviation Regulation | 1. Employ Hazard-Based Regulation 20

 Regulatory Risk-Based Characteristic
Function Approach

RISK-BASED Regulation enacts controls

REGULATE REGULATION based on assessment of risk

Allocates oversight resources

RISK-BASED based on assessment of risk
OVERSIGHT OVERSIGHT (including risk of
noncompliance)

No equivalent risk-based
ENFORCE approach to enforcement
23

Figure 7. Risk-based regulation and risk-based oversight are not synonymous.

HAZARD-BASED REGULATION
Hazard-based regulation is the application of a systemic framework that prioritises regulation on an
assessment of hazards. Hazard-based regulation ensures regulatory obligations are imposed to
treat threats to safety where the safety risk of those threats cannot be estimated.

Hazard-based regulation starts with a risk model to analyse hazards which may materialise. The
regulations then place obligations on those regulated to create the necessary defences to prevent
the hazard from materialising. The benefit of hazard-based regulation is that it is proactive and
increases focus by avoiding non-safety related regulation that may creep into reactive regulatory
approaches.

Whilst the data needs of hazard-based regulation are less than risk based the major problem is the
recognition that the most serious scenarios cannot be identified from a general rule even where data
on past events exists. So the approach to regulating hazards can be either weak or strong form. The
weak form holds that the lack of evidence of a risk should not automatically initiate regulation or
measures to prohibit the activity (hazardous optimism). The strong form of the principle asserts that
regulation is required whenever there is a possible adverse risk, even if the supporting evidence is
speculative and even if the economic costs of regulation are high (prudent pessimism).

23
Nil enforcement action based on assessment of the risk of noncompliance is illogical and undermines the
regulation. Graduated response to enforcement is a better way and is described in chapter 9.

10 Ways to Better Aviation Regulation | 1. Employ Hazard-Based Regulation 21

Either way hazard-based regulation does not necessarily mean the hazard is removed entirely as it
often cannot be (recall the crosswind example). Hazard-based regulation mandates defences to
reduce the risk of the hazard materialising. A responsible person at some point will need to decide
whether the level of defence is adequate. This may be the regulator or the regulated entity
depending on whether the regulation is drafted prescriptively or in an outcome-based style
(discussed in chapter 2).

For example, a regulation requiring authorisation of design engineers every 12 months is the
regulator explicitly setting a standard. But how can the regulator know that 12 months is
satisfactory? An alternative is an outcome-based style which may require the regulated entity to
determine the minimum authorisation period themselves according to their own assessment of their
circumstances.

RISK TOLERABILITY
The fundamental question in any regulatory regime is how much risk is the regulator prepared to
tolerate. Regulators do not often articulate what their risk appetite is in public, or even private. But
who should apply the tests and make tolerability decisions? The answer is probably all stakeholders
(the risk creator, those at risk, and the regulator). As acceptance of risk depends on the potential
benefits, there are likely to be differences between stakeholders in perception and opinion.

In risk-based regulation the regulator is explicitly stating the level of risk tolerance. For example
stipulating a design standard for an aircraft system is a risk-based approach because inherent in
that decision is a level of risk. A move towards outcome-based regulation obligates the regulator to
consider who is assessing the risk. Hazard-based regulation does not necessarily require the
regulator to assess risk. It may place the risk assessment on the regulated communities. The
regulator, if involved, assesses the discharge of both the risk-creators and the risk assessors
responsibilities.

SIZE AND SHAPE OF HAZARDS

The size of a hazard drives the granularity of regulation. At the lowest level, attention might be
focussed on a problem so small and so particular that it really represents one incident or violation
and is really not a pattern at all. An incident may be part of a pattern, but is not a pattern by itself.
Focussing attention at the finest level of granularity moves practitioners below the level for effective
problem solving and back into a reactive case-by case processing mode.

The opposite extreme is where problems being addressed are not specific at all, but are bundles of
disparate objects, all lumped together. At this level one considers generalised classes of problems
and weighs the merits of generalised classes of response. At this macro level of regulatory attention
insufficient granularity means patterns of incidents or violations are missed entirely.

10 Ways to Better Aviation Regulation | 1. Employ Hazard-Based Regulation 22

In between the two extremes is where the most fruitful regulatory work lies.24 Whilst difficult to
define, the best advice is to think about individual incidents from the point of view of incident
response. If the organisation responsible already has processes and procedures established to
handle such an event (but they failed) then focus on enforcement of those processes and
procedures. Avoid regulating for more granularity when sufficient granularity already exists.

BOW-TIE ANALYSIS
This chapter has explained the difference between hazard and risk-based regulation and considered
the size of hazards to be controlled. One way of visualising this is using an adapted bow tie model.
The model provides a way of understanding how hazard-based regulation enacts defences against
safety hazards. But first is an explanation of the bow tie risk model.

INTRODUCTION TO BOW TIE

Bow tie (or cause and consequence) risk models are a diagrammatic representation of the
relationship between the management system and the hazards being managed, by linking hazards
and their consequences through event lines illustrating the routes to undesired events. The bow-tie
tool therefore facilitates an in-depth proactive assessment of hazard defences.

Bow-ties are most commonly used where there is a requirement to demonstrate that hazards are
being controlled, and particularly where there is a need to illustrate the direct link between the
controls and elements of the management system.

A generic bow tie risk model is depicted in Figure 8. Bow tie risk models are based on:

(i) Identifying a loss of control event;

(ii) Identifying possible causes for a loss of control event;

(iii) Identifying possible consequences following a loss of control event;

(iv) Identifying defences to prevent possible causes of a loss of control event; and

(v) Identifying defences to reduce or eliminate the consequence resulting from a loss of control
event.

BOW-TIE TERMINOLOGY
In addition to the earlier definitions, a further set of terminology is used in bow-tie methodology.25

The top event is the point at which control of the hazard is lost resulting in a change of state. Using
the crosswind example, the top event would be loss of lateral control of the aircraft on landing.

24
Sparrow, M (2008) The Character of Harms: Operational Challenges in Control, Cambridge University Press,
pp 79.
25
Department of Defence (2012) AAP6734.001 Defence Aviation Safety Manual, chap 7, annex E.

10 Ways to Better Aviation Regulation | 1. Employ Hazard-Based Regulation 23

The Consequences are the final results that could occur in the event of the entire accident
sequence mapped on the bow tie being realised. In other words, what will happen when a loss of
preventive control releases a hazard (a top event) and the recovery controls fail? In the crosswind
example a consequence could be a wing striking the ground.

Preventive controls are the mechanisms put in place to prevent the release of a hazard resulting in
the top event. A maximum crosswind limitation is an example to prevent against loss of lateral
control on landing.

Recovery controls are the mechanisms put in place to recover control following occurrence of the
top event. In the crosswind example a recovery control could be an established go-around
procedure.

Escalation factors are latent conditions/factors which act to weaken the effectiveness of controls.
An example applicable to the preventive control crosswind limitation would be an ambiguous
description of the maximum crosswind in the operating procedures resulting in misunderstanding
amongst flight crew as to its purpose (e.g. mandatory or recommended?).

Escalation controls are mechanisms put in place to prevent escalation factors from affecting the
performance of controls. An example is flight crew training with the go-around procedure in a
simulator prior to initiating it for real.

Figure 8. A generic example of a bow tie risk model.

10 Ways to Better Aviation Regulation | 1. Employ Hazard-Based Regulation 24

 EMPLOY HAZARD-BASED REGULATION IN CONTEXT
There is no such thing as an accident.
26
What we call by that name is the effect of some cause which we do not see.

A BETTER WAY TO REGULATION

Traditional Defence aviation safety regulation adopted a reactive approach, prescribing precise
measures based on previous experience. But in the past two decades we have seen a marked
change in the way safety regulation is framed by professional regulators across the world. Due to
the complex systemic organisational factors involved, aviation professionals realise the regulator
cannot always be in a position to assess every risk and mandate every defence. Furthermore,
regulation which is silent on the actual hazard it intends to protect against can become unwieldy as
the hazard is slowly lost in regulatory minutiae, which is itself a potential threat to safety in the
longer term.

The management of safety is, and has always remained, in the hands of operators, their equipment
maintainers and engineers. Acknowledging this, and to re-establish the awareness of the hazards
which the regulations intend to control, Defence aviation safety regulators should employ
hazard-based regulation.

APPROACH TO HAZARD-BASED REGULATION

The benefits of hazard-based regulation are to:

(i) Return the focus of regulation on safety hazards;

(ii) Remove regulation not related to defences against those hazards (cut red tape); and

(iii) Clearly place the management of risks associated with hazards on those in the correct
position to control them.

The three broad steps to hazard-based regulation are:

1. Identification of aviation safety hazards;

2. Identification of the characteristics of defences to protect against those hazards; and

3. Drafting of regulation to obligate the correct people to enact those defences.

26
Voltaire (1694-1778).

10 Ways to Better Aviation Regulation | 1. Employ Hazard-Based Regulation 25

 Box 4: In-practice technical airworthiness regulations are not hazard based

Figure 9 is the proportion of regulatory clauses in the technical airworthiness regulations

(TAREGs) that are not regulation. Almost 40% of the 2013 suite of TAREGs are not
regulation of any kind (let alone hazard-based). Many are administrative in nature (red
tape not contributing to safety) or good process guidance which need not be regulation.
The concern for the long term is that the focus on the hazards and their safety risks will
be lost.

100%

80%
Proportion Not Regulation

60%

40%

20%

0%
All 1 2 3 4 5

TAREG

Figure 9. The proportion of TAREGs that are not regulation (red indicates highest proportions).

PBP BOW TIE FOR REGULATION

Hazard identification for a specific task or activity is often challenging enough. It is not practical for
an aviation safety regulator to attempt this for every aircraft and support system. Hazard mapping
using an adaptation of the bow-tie model is a more practicable way of identifying and justifying
safety regulation and identifying ineffective and unnecessary regulation. The resulting bow-tie not
only describes what regulations are in place today, but why they will still be there tomorrow.

The Process Behaviour Product (PBP) bow tie model for regulation is based on a generalised bow-
tie model and modified by considering causes and consequences in generic threat groups across
life cycle categories.27 The PBP bow tie model provides a structure for the systematic representation
and analysis of safety regulation regimes. The features and concepts of the model are:

27
The PBP bow tie originates in Purton, L (2014) Mutual Recognition of National Airworthiness Authorities: A
Streamlines Assessment Process, International Journal of Aeronautical & Space Science, vol 15 no 1, pp 54-62.

10 Ways to Better Aviation Regulation | 1. Employ Hazard-Based Regulation 26

Top Event. For aviation and aviation support systems the top level event could be an undesirable
operational state or similar.

Loss of Control. The loss of control of the top event is dependent on the responsibilities of the
regulator. Where separate technical, operational and aviation support regulatory regimes are in
place then the loss of control states would be:

(i) Loss of technical integrity;

(ii) Loss of operational integrity; or

(iii) Loss of aviation support integrity.

Loss of Control Considerations. Although a little difficult to conceptualise, a technique that has
shown some promise in helping to maintain a generic and complete outlook when trying to identify
generic causes, consequences and controls has been to consider the loss of integrity in terms of
pluses and minuses. It may be possible to use this method to capture and consider failure or loss of
control modes. The technique considers:

(i) A physical (P) attribute or presence that we have but do not want (+P), e.g. foreign object.

(ii) A physical attribute that we want but do not have (-P), e.g. missing or unserviceable item or
service.

(iii) A physical attribute or component that interferes with another physical attribute or
component (P/P), e.g. oversize cargo load.

(iv) A functional (F) attribute that we have but do not want (+F), e.g. excessive power, a
product that hardens before it can be applied properly.

(v) A functional attribute that we want but do not have or is missing (-F), e.g. insufficient power,
absence of electronic counter measures.

(vi) A functional attribute that interferes with another Functional attribute or operation (F/F), e.g.
electromagnetic interference.

(vii) An operator (O) exceeds operational parameters (+O), e.g. excessive G forces, or over
torque/tightening a fitting/cable.

(viii) An operator fails to understand or operates incorrectly through ignorance or denial (-O),
e.g. fails to follow operating instructions.

(ix) An operator interferes with or impedes another operator (O/O), e.g. collision, or distraction
during a vital operation.

Service Events. Each type of service (or information) needs to be considered separately to
determine what could go wrong to cause a loss of control. To help do this in a generic and
independent way, each of these events are considered along product, process and behaviour threat
lines, and subcategory activities are used to denote significant activities in the life cycle and use of

10 Ways to Better Aviation Regulation | 1. Employ Hazard-Based Regulation 27

technical equipment. Product, process and behaviour threat lines are proposed because throughout
the world regulations are generally applied to products, processes and behaviour, regardless of
what is actually being regulated. For technical integrity the subcategory activities could be design,
production, maintenance and distribution. Similar subcategories exist for operational integrity.

Consequences. For each type of service and loss of control, we determine what could be the likely
consequences of a loss of control.

Preventive Controls. To reduce the likelihood of a loss of control occurring, practicable and
appropriate controls are proposed (where possible) for each threat line and activity.

Recovery Controls. To reduce the consequences resulting from a loss of control, practicable and
appropriate controls are proposed (where possible).

The basic layout of the PBP bow tie model structure is illustrated in Figure 10. Conventional bow tie
models in aviation safety are utilised to assess the controls in place, to reduce the likelihood, or
reduce the consequence of a potential hazard, triggered by a specific event. In the PBP bow tie
model in this example the top event is the loss of technical integrity. Technical integrity can be
maintained by assuring product, behaviour and process (PBP) integrity. The three elements
establish a set of threat lines potentially leading to a loss of technical integrity, and in turn, ultimate
consequences.
N

DE
SIG

SIG
DE

PR
N

N
TIO

OD
UC

UC
CE

MA
OD

T
AN

INT

ION
PR

EN

EN
INT

AN
MA

CE

PREVENT RECOVER
TOP EVENT

LOSS OF
TECHNICAL
INTEGRITY DURING
OPERATION

CO
ION NS
EQ
CT U
DU EN
RE C EM
OD IT I
IHO GA
EL TIO
LIK N

Figure 10. Composition of the PBP bow tie model for loss of technical integrity.

10 Ways to Better Aviation Regulation | 1. Employ Hazard-Based Regulation 28

Controls can be put in place to reduce the likelihood of any threat line eventuating. Within the
technical item lifecycle, there are three distinct activities: design, production and maintenance.
Controls are grouped in relation to these three activities. Controls are also grouped as preventative
(grouped on the left-hand side of Figure 10) or recovery (grouped on the right-hand side).

Each control represents one or more specific regulatory obligation.

While this bow tie example focuses on the top event of loss of technical integrity during operation,
operational controls could also be introduced into the same bow tie on the recovery (right-hand) side
to recover from a loss of technical integrity.

HAZARD-BASED REGULATION AND THE PBP BOW-TIE

The purpose of describing the PBP bow tie model here is to show how it can be used to identify and
justify safety regulation and identify ineffective and unnecessary regulation. Firstly, a hazard-based
regulation enacts a control or escalation control on the PBP Bow-Tie. If a regulation cannot be
visualised on the model it is most likely not addressing a hazard and its utility to safety assurance
should be questioned. Secondly, the bow tie not only describes what regulation is in place today, but
can be used to explain why it should still be there tomorrow.28

It is important to note that risk-based regulation does not lend itself to visualisation on the PBP bow
tie model. While the model would look the same, bow tie does not consider likelihood because it
was never designed for the detailed quantitative assessment of risk. Therefore plotting risk-based
regulation on a PBP bow tie model is potentially misleading.

28
Readers should be aware the PBP bow tie model was developed for the purposes of comparing different
aviation safety regulatory regimes by identifying test-points within each regulatory framework. The use of the
model here to identify and justify controls to be enacted through regulation is therefore a more simple use of a
tool that is of further utility but beyond the scope of this publication.

10 Ways to Better Aviation Regulation | 1. Employ Hazard-Based Regulation 29

 Box 5: In-practice comparison of process regulation and product standards

Most product-based standards are inherently risk based regulation because product
standards set risks levels (during certification). For example the certification of a gas
turbine engine is based on a possible (albeit very small) chance of catastrophic structural
failure. The design of gas turbine engines is regulated through the certification process to
prevent catastrophic failure resulting in the loss of the aircraft based on an assessment of
risk. Such failure modes could occur due to the failure of critical high speed rotating
components whose failure will likely result in the release of high-energy debris that cannot
be contained within the engine such as disks and shafts.

One failure mode of such parts is through fatigue. Fatigue is not managed by hazard-
based regulation. Instead regulation (certification) requires that the likelihood of an engine
critical part failure is so remote as to be considered airworthy (FAR 33 requires
hazardous engine effects are predicted to occur at a rate not in excess of one in 10
million events per flight hour).29

The Defence publication AAP7001.054Airworthiness Design Requirements Manual

sets Defences preferred design standards. Design standards are mostly risk-based
regulation because they incorporate some level of inherent risk of failure (which may be
impossible to quantify but nevertheless considered small enough based on past
experience). With its focus on product standards risk-based regulation is similar to quality
control.

The Technical Airworthiness Regulations (TAREG) in AAP7001.053Technical

Airworthiness Management Manual is mostly process type regulation. These regulations
tend not to incorporate risk levels because they are less scientific. Instead they should
remain hazard-based where the level to which a hazard is managed on a case by case
basis and in many cases left up to the regulated entity. With its focus on process, hazard-
based regulation is similar to quality assurance.

29
United States Code of Federal Regulations, 14CFR33.75 (2007).

10 Ways to Better Aviation Regulation | 1. Employ Hazard-Based Regulation 30

 Box 6: In-practice example of a hazard-based aviation software regulation

Hazard Preventive Control Example Regulation

The applicant must ensure

that:

Unspecified anomalies within Apply software assurance to Aviation software must be

aviation software could result ensure that appropriate assured to perform its
in hazardous behaviour of rigour has been applied intended specified functions,
aircraft software (control or during design commensurate to provide confidence that
aircraft, system performance with the worst-case failure performance will be
or misleading displays) condition associated with appropriate and safe.
aviation software. Software
assurance provides
confidence that software will

EXAMPLE
provide appropriate
behaviour when it needs to
provide that behaviour.

Improper assessment of Provide oversight to assure The means of establishing

software assurance could that the evidence, assurance for aviation
result in unintended or methodology and compliance software providing or
unsafe behaviour of aircraft finding agencies that will be controlling functions that
software. used establish compliance could cause a fatality or
with design benchmarks are serious injuries must be
reasonable. submitted to the TAR prior to
the conduct of compliance
finding.

Incorrect software Integrity - Load Control / CM Controls that assure only the
configuration could result in intended approved aviation
hazardous behaviour of software configuration is
aircraft software. installed in aircraft equipment
shall be established.

Known anomalies could Integrity - Problem Reporting Unsafe anomalous behaviour

result in hazardous of aviation software must be:
behaviour of aircraft - identified and tracked; and
software. - unsafe behaviour that
results in aircraft hazards
should be subject to risk
management

Unsafe behaviour specified Safety - System Safety, The safe appropriate

in the design of aviation Software Safety behaviour of aviation
software, failure to provide software must be defined
protection from erroneous within software design, such
input, or failure to that (i) requirements state
appropriately specify safe expected safe behaviour,
behaviour could result in and (ii) software behaves in a
hazardous behaviour of safe manner when subjected
aircraft software. to abnormal input.

10 Ways to Better Aviation Regulation | 1. Employ Hazard-Based Regulation 31

10 Ways to Better Aviation Regulation | 1. Employ Hazard-Based Regulation 32
2. MAXIMISE OUTCOME-BASED
REGULATION

SUMMARY

WHAT
Where possible, focus regulation on the outcome needed to treat threats to safety and not the
means to achieving that outcome.

WHY
Affords more flexibility to regulated entities to develop and implement a range of lower cost
management solutions while achieving the same level of safety assurance. Regulated entities
are better placed than regulators to determine what processes and actions suit their business to
satisfy a regulatory objective. So regulators, instead of focussing on prescribing actions that
organisations must take, step back and define the outcome required.

HOW
Draft each hazard-based regulation to focus on what the control is intended to achieve, rather
than the means by which it should be achieved.

Outcome-based regulation is a characteristic of the REGULATE function in the regulatory model.

Regulator Regulated Entity

REGULATE

INTERPRET
MANAGE R ELATIONSHIP

MANAGE RELATIONSHIP

E DUCATE

IMPLEMENT

OVERSIGHT

ADHERE

ENFORCE

10 Ways to Better Aviation Regulation | 2. Maximise Outcome-Based Regulation 33

 MAXIMISE OUTCOME-BASED REGULATION IN THEORY

INTRODUCTION
The primary aim of regulation is to influence behaviour and affect a particular outcome.
Conceptually this sounds simple. But it involves communicating requirements with written language.
As with any written language, the style, tone and level of detail can significantly influence
comprehension.

Regulation can prescribe required behaviour or it may set goals for the outcome of behaviour
without detailing how it should be achieved. The level of specificity may also vary widely. The style
of regulation therefore is an important characteristic that should be understood and considered. The
following terminology is often used to describe regulation. Hazard-based was explained in chapter 1.
Those in bold are regulatory styles and discussed in this chapter. The remaining terms are
introduced in later chapters.

Outcome-based Purposive approach Rules-based

Prescriptive
Hazard-based Principles-based

Literal approach Performance-based

Management-based Risk-based

REGULATORY STYLE DEFINITION

Consider the following examples, which all aim to achieve the same objective:

Example 1. The applicant shall review for accuracy 10% of all flight manual pages every month.

Example 2. The applicant shall establish procedures to ensure that flight manuals are correct.

Example 3. The applicant shall establish a system to ensure all documentation and publications
used to support safe operation of the aircraft are correct and authorised.

Which of these examples will be most effective? Or efficient?

There is no simple answer. It depends on the circumstances, the extent and history of previous
regulation, organisational cultures of members of the regulated community and the ability and
capability of the regulator to conduct oversight and enforcement. This chapter provides some theory
to understand why the above examples are different and when each of the styles would be best
utilised.

10 Ways to Better Aviation Regulation | 2. Maximise Outcome-Based Regulation 34

STYLE OF INTERVENTION
Very simply, organisations exist to achieve something. They do this by:

1. Planning to do something;

2. Acting to output something; and

30
3. Achieving an outcome (and comparing against what was planned).

The introductory chapter explained that regulation exists to influence behaviour to affect a particular
outcome or prevent an undesirable occurrence. Regulation targeted at any stage of production will
potentially affect outputs. We can therefore distinguish between styles of intervention based on the
location of the regulation in the stage of production, as explained in Figure 11.31

Stage of Planning Acting Output

Production

MANAGEMENT- PRESCRIPTIVE OUTCOME-

Style of
BASED REGULATION BASED
Regulation
REGULATION REGULATION

Figure 11. Regulation can be designed to influence organisations at different stages of production.

Working from the right, outcome-based regulation specifies the required output to be achieved. No
reference is made as to the manner in which the outcome is to be achieved and therefore provides a
degree of freedom to the regulated to determine compliance.

Regulation intervening in the acting stage is prescriptive regulation by specifying technologies to

be used or processes to be followed.

Management-based regulation neither explicitly imposes the means nor the ends. It intervenes at
the planning stages and directs those regulated to engage in a planning process that aims toward

30
This aligns with the quality management principles of plan, do check.
31
Adapted from Coglianese, C and Lazer, D (2003) Management-Based Regulation: Prescribing Private
Management to Achieve Public Goals, RPP-2003-08, Center for Business and Government, John F. Kennedy
School of Government.

10 Ways to Better Aviation Regulation | 2. Maximise Outcome-Based Regulation 35

the achievement of the regulatory goal, offering organisations flexibility in how they achieve those
goals.

OUTCOME BASED
Also known as performance-based regulation.

Outcome-based regulation is regulation that specifies the required outputs to be achieved. In other
words it defines the required performance. It makes no reference to the manner in which the
outcome is to be achieved and therefore provides a degree of freedom to the regulated community
to achieve compliance.

Attractive for its flexibility, outcome-based is appropriate when the output can be measured and its
impact evaluated against the desired objective of the particular regulatory defence it enacts. In
addition to flexibility, benefits include ability to accommodate technological change in ways
prescriptive regulations focusing on a specific technology cannot.

On the negative side, regulatory staff who are used to enforcing relatively straightforward
prescriptive standards are more likely to be uncomfortable with the discretion inherent in assessing
outcomes. Finally, outcome-based regulation makes it difficult to assess the potential resource
impact of a regulatory change on the regulated community due to its focus on outcomes rather than
the actual work undertaken.

PRESCRIPTIVE
Also known as technology-based regulation.32

Traditionally regulators have relied primarily upon prescriptive regulation. In varying levels of detail,
this type of regulation specifies how work is to be done with the expectation of strict compliance.
The presumption is following the rules will bring about the desired regulatory outcome.

Prescriptive regulation is appropriate where there are single, commonly agreed means of controlling
a hazard. Advantages include certainty and clarity. The most important use of prescription is to
enable interoperability amongst the regulated community through standardisation. Examples in civil
aviation include rules of the air, carriage of dangerous goods and aircraft marshalling.

Disadvantages include inflexibility, the liability of becoming outdated, stifling of innovative regulatory
compliant solutions and high administrative and compliance costs. Prescriptive rules can also
contribute to their own defeat through creating a culture of regulatory decision making according to

32
Design standards are included within the definition of regulation if there is an expectation the standard is to be
satisfied and a process of verification is undertaken to confirm compliance. Design standards are typically a
combination of prescriptive and outcome-based requirements focusing on the end product. For example, a
structural design standard requiring a 9G limit load is an outcome-based requirement. A standard requiring the
wing to incorporate at least three wing spars is more prescriptive.

10 Ways to Better Aviation Regulation | 2. Maximise Outcome-Based Regulation 36

rule, a key finding of the investigation into the Piper Alpha offshore oil rig disaster on 6 July 1988
where 167 workers died as a result of a series of explosions.

Imposition of detailed requirements cannot anticipate all the variances of differing practice,
location, organization and size that exist. In fact, prescriptive regulation or over-detailed
guidance may at times result in the overall objective actually being compromised. Innovation, on-
going improvement and objectivity will be stifled; and the more prescriptive the regulation the
33
more unclear it is who has the responsibility for total safety.

MANAGEMENT-BASED
Also known as process-based, systems-based, supervised regulation, meta-regulation or
safety-case regulation.

Management-based regulation focuses on the planning and management activities involved in

producing an outcome. It seeks to take advantage of the regulated communities understanding of
the relationship between their behaviours and their outputs, compelling those parties to conduct
their own evaluations and find their own control solutions. It has the considerable attraction of
providing the most flexibility for enterprises to devise their own least-cost solutions and of facilitating
their going beyond compliance with minimum standards. Regulators tend to employ it to address
systemic problems rather than individual hazards, most commonly used where there are multiple
risk sources and multiple feasible risk controls. Problems where the regulator lacks performance
measures, at least short of the dire consequences regulators seek to prevent in the first place is
also a potential candidate for management based regulation.

Management-based approaches hold a number of potential advantages over other styles of

regulation. Firstly, they place the responsibility for decision-making with those who possess the most
information about the hazards and potential controls. Therefore the actions taken may prove to be
less costly as well as more effective than under imposed solutions or outcomes. Secondly, by
allowing organisations to make their own decisions, personnel are more likely to view their
organisations rules as reasonable and as a result there may be greater compliance.34

Safety management system (SMS) regulation is an example of management-based regulation. The

purpose of an SMS is to manage latent, organisational-type defects. The regulation of SMS in
aviation is typically introduced in parallel with existing regulation because it focuses on threats and
escalation factors that are difficult to control through traditional prescriptive regulation.

33
Cullen, W.D. (1990), The Public Inquiry into the Piper Alpha Disaster, Her Majestys Stationary Office, London,
para 21.4. Lord Cullen examined the (then) existing UK off-shore and on-shore oil and gas safety systems. One
of the major outcomes was a transition to outcome-based regulation.
34
For more information regarding management-based regulation readers are directed to Coglianese, C (2010)
Management-Based Regulation: Implications for Public Policy in Risk and Regulatory Policy Improving the
Governance of Risk, Organisation for Economic Development, chap 5.

10 Ways to Better Aviation Regulation | 2. Maximise Outcome-Based Regulation 37

MAKING THE CHOICE
According to the limited academic research in this area there are 2 dimensions to think about when
deciding which style to use as presented graphically in Figure 12:35

(i) The regulators ability to measure outcomes accurately; and

(ii) The degree of uniformity across the regulated community.

Management-based regulation is recommended when the sector is varied and where the capacity
to assess output is low.

Outcome-based regulation is recommended when the ability to measure outcomes can be

established through indicators or metrics.

Prescriptive regulation is recommended when uniformity across the regulated community is high,
but the ability to measure outcomes is not possible.

High
Capacity/Ability to Assess Output

OUTCOME-
BASED
REGULATION

PRESCRIPTIVE-
BASED
MANAGEMENT- REGULATION
BASED
REGULATION

Low High

Uniformity of Regulated Entities

Figure 12. Graphical representation to assist choosing regulatory style.

35
Coglianese, C and Lazer, D (2003) Management-Based Regulation: Prescribing Private Management to
Achieve Public Goals, RPP-2003-08, Center for Business and Government, John F. Kennedy School of
Government.

10 Ways to Better Aviation Regulation | 2. Maximise Outcome-Based Regulation 38

10 Ways to Better Aviation Regulation | 2. Maximise Outcome-Based Regulation 39
This work has been licensed by Copyright Agency Limited (CAL). Except as permitted by the Copyright
Act, you must not re-use this work without the permission of the copyright owner or CAL.

10 Ways to Better Aviation Regulation | 2. Maximise Outcome-Based Regulation 40

 MAXIMISE OUTCOME-BASED REGULATION IN CONTEXT
the principle regulations in regard to offshore [oil and gas] safety should take the form of
requiring that stated objectives are to be met rather than prescribing that detailed measures are
36
to be taken.

A BETTER WAY TO REGULATION

In Defence, uniformity among the regulated community continues to reduce and the ability to assess
outcomes continues to strengthen. To allow for more flexibility Defence aviation safety regulators
should maximise use of outcome-based regulatory style.

DRAFTING OUTCOME-BASED SAFETY REGULATION

As outsourcing to industry continues and innovative in-service support arrangements continue to
materialise the flexibility available through outcome-based approach must be exploited.

But characterising an already existing regulatory regime as being prescriptive, outcome- or

management-based is not particularly helpful either. Better questions are what is, and what should
be, the relative roles of each. Neither will function particularly successfully without acknowledging
the strengths and benefits of each as well as the context within which they are applied.

The following should be considered when drafting regulation:

(i) Use outcome-based based regulation when it is possible to specify clearly the required
outcome standard, and when there is likely to be different ways of achieving it.

(ii) Use management-based regulation judiciously, where multiple hazards and threats exist
and multiple controls are required.37

(iii) When considering management-based or outcome-based regulation, consider the

capacities of members of the regulated community. Will it be feasible and proportionate to
ask them to undertake the required compliance process?

(iv) Consider the relationship between outcomes, management and prescriptive regulatory
elements to ensure they are mutually supportive rather than opposed.
38
(v) Finally, regardless of the style chosen, draft regulation simply using plain English.

36
Cullen, W.D. (1990) The Public Inquiry into the Piper Alpha Disaster, Her Majestys Stationary Office, London,
para 21.67 (square brackets added).
37
Management-based regulation is the basis of the safety case regulatory approach used widely in the work,
health and safety field and in some other military aviation systems.
38
Office of Parliamentary Counsel (2013) Plain English Manual is the recommended guide to drafting regulation.

10 Ways to Better Aviation Regulation | 2. Maximise Outcome-Based Regulation 41

 Box 7: In-Practice style of the 2013 Defence technical airworthiness regulations

Of the 2013 stock of Defence technical airworthiness regulations (TAREGs), almost half
of the engineering regulations (TAREG 3) were prescriptive. The engineering regulation
originated in the 1990s from previous internal Defence policy and process guidance. The
maintenance regulations developed in the early 2000s (TAREGs 4 and 5) regarded by
many as better drafted were almost entirely outcomes based reflecting a more modern
approach to regulatory development.

Of interest is comparison with the FAA, where it is estimated that only 20% of the Federal
Aviation Regulations (FAR) are outcome-based.39 It is difficult to compare directly
however as all of the inherently prescriptive FAA design standards are codified into
regulations, whereas TAREGs do not include design standards.

It should be stressed the regulations have been reasonably successful since their
introduction as seen in Figure 13. Now with twenty years experience, Defence and
industrys interaction with aviation regulation has matured to the point where many of the
prescriptive provisions are overly restrictive and burdensome, creating inefficiencies and
unnecessary compliance costs for Defence and industry.

Figure 13. Defence fatal accidents 1985-2014.

39
US Government (2010) Aviation Safety Report to Congressional Requesters, Government Accountability
Office GAO-11-14, pp 14.

10 Ways to Better Aviation Regulation | 2. Maximise Outcome-Based Regulation 42

ADDITIONAL CONSIDERATIONS TO IMPLEMENT OUTCOME-BASED REGULATION
Implementation of outcomebased regulation requires supporting mechanisms described elsewhere
in this publication and summarised here.

MEASURING COMPLIANCE
Successful outcome-based regulation is dependent on achieving a fair and objective measure of
what constitutes compliant. This is addressed in chapter 4 Utilise Compliance Proofs.

LEVEL OF SPECIFICITY
Outcome based does not mean vague, motherhood principle statements. This would result in too
much uncertainty. As an example, an independent review of the US Federal Aviation Regulations
found that inconsistent interpretation relating to outcome-based regulation was ranked either first or
second in problem for the flight standards and aircraft certification processes respectively:

Variations in FAAs [Federal Aviation Administrations] interpretations of standards and

certification and approval decisions occurs as a result of factors relating to performance-based
[outcome-based] regulation and the use of professional judgement by FAA inspectors and
40
engineers.

Outcomes may be tightly or loosely defined as required. An adequate amount of detail, but no more,
is necessary to support outcome-based regulation. This is addressed in chapter 5 Ensure
Sufficient Prescription.

EDUCATION
Intuitively duty holders must have a clear understanding of what they must do to comply with their
regulatory obligations. It is not surprising that in support of less prescriptive regulation comes
increased focus on education and training, addressed in chapter 6 Comprehensive Explanation.

SAFETY INDICATORS
Outcome-based regulation relies not only on the regulator but the regulated entity to monitor and
measure the on-going success of their systems and implementation. It is unacceptable to assume
that a lack of accidents is proof of success. Both the regulator and the regulated entity will need to
establish safety indicators, addressed in chapter 7 Safety Indicators.

40
US Government (2010) Aviation Safety Report to Congressional Requesters, Government Accountability
Office GAO-11-14, pp 14 (square brackets added). GAO was tasked to examine the variation in the FAAs
interpretation of standards for certification, approval decisions and overall effectiveness.

10 Ways to Better Aviation Regulation | 2. Maximise Outcome-Based Regulation 43

10 Ways to Better Aviation Regulation | 2. Maximise Outcome-Based Regulation 44
3. TAKE A PURPOSIVE
APPROACH

SUMMARY

WHAT
Express the purpose of the regulatory obligation simply and clearly and interpret and apply
regulation with its purpose at the foremost of mind.

WHY
Increases clarity, protects against tactical, black and white or nonsensical interpretation
disputes between members of the regulated community and the regulator.

HOW
1. Structure regulation so its purpose is clearly apparent, stands alone and places the need for
the obligation beyond dispute.

2. Interpret regulation in light of the purpose for which it was enacted.

Purposive regulation is a characteristic of the REGULATE function in the regulatory model.

Regulator Regulated Entity

REGULATE

INTERPRET
MANAGE R ELATIONSHIP

MANAGE RELA TIONSHIP

E DUCATE

IMPLEMENT

OVERSIGHT

ADHERE

ENFORCE

10 Ways to Better Aviation Regulation | 3. Take a Purposive Approach 45

 TAKE A PURPOSIVE APPROACH IN THEORY
41
All rules have a core of settled meaning surrounded by a penumbra of uncertainty.

INTRODUCTION
Certainty of regulation is good regulation. Certainty comes from common understanding of the
regulation by those who are obliged to follow it and by those who enforce it. In legislation lawyers
spend much of their time interpreting legislation in order to decide how it applies in particular
circumstances. The Defence aviation safety regulation needs to be simpler it is not aimed at
lawyers but personnel with a wide range of backgrounds from Defence and industry, including
engineers, technicians, logisticians, managers, aircrew, commanders and others.

Typically, without a prior understanding through precedence, or a stated presumption about how
regulation should be interpreted, it is natural for people to interpret regulation literally to the letter.
This can result in complications. This chapter looks at a different and often better approach to
writing and interpreting regulation.

The terms literal approach and purposive approach discussed in this chapter originate in the legal
fraternity. In this publication the terms are used quite loosely. It is not the intention to explore the
legal basis of either approach. Just enough is explained to distinguish between the two and to guide
Defence to better aviation regulation.

Outcome-based Purposive approach Rules-based

Prescriptive
Hazard-based Principles-based

Literal approach Risk-based Performance-based

Management-based

THE LITERAL APPROACH

Literal interpretation of regulation has a simplicity about it that is attractive the regulation means
what it says. Ordinarily the meaning will correspond with the grammatical meaning of the words in
the regulation. Unfortunately literal interpretation suffers from a major defect. The literal approach
assumes a word or phrase read in context has only one meaning. But a regulation may have no
single, unambiguous meaning.

Over time as different meanings are developed certainty reduces. Uncertainty drives compliance
costs and, worse still, deviation from the original intent of the regulation. A subsequent response by
the regulator could be to create catch-all provisions, intentionally left vague or undefined, to prevent

41
Hart, H.L.A (1994) The Concept of Law, Oxford University Press.

10 Ways to Better Aviation Regulation | 3. Take a Purposive Approach 46

further literal avoidance. This increases uncertainty. Over time this may be solved by reintroducing
more prescriptive regulation. A pattern of oscillation of literal interpretation commences between
prescriptive regulation and more loosely defined catch-all principles. The best result is a fairly
unsophisticated compromise between the two.

THE PURPOSIVE APPROACH

During the 1970s and early 1980s some Australian courts were criticised for handing down
decisions based on overly literal interpretation of legislation. As a result the Commonwealth, and
later the States and Territories, enacted use of the purposive approach to interpretation, referred to
as a shift from text to context.

Writing and interpreting regulation in such a way as to give best effect to its purpose or object is
called the purposive approach. Purposive regulation aims to establish common understanding by
focusing on the actual intent of the regulation. The success of the purposive approach is based on
the premise that unless the purpose is promoted compliance with literal interpretation cannot
achieve the declared (and sometimes undeclared) objectives.

IMPLEMENTATION OF THE PURPOSIVE APPROACH

Purposive does not mean the words of a regulation are ignored in favour of its purpose. This would
be nonsensical. Rather the purposive approach is used when an attempt to apply the literal
approach produces an ambiguity or uncertainty.

Purposive regulation supports any style of regulatory intervention addressed in chapter 2, whether it
is management-based, prescriptive or outcome-based regulation.

To understand the purpose of a regulation consideration should be given to any extrinsic material
capable of assisting. The strongest piece of extrinsic material is a purpose statement. A purpose
statement accompanies, but is not part of, the regulation and ensures its purpose is clearly
apparent, stands alone and places the need for the obligation beyond dispute.

10 Ways to Better Aviation Regulation | 3. Take a Purposive Approach 47

 TAKE A PURPOSIVE APPROACH IN CONTEXT
We dont have any major issues with the regulations,
42
as long as the regulator interprets them the way we do.

A BETTER WAY TO REGULATION

Until now there has been no common maxim or presumption used in interpreting Defence aviation
safety regulation. Anecdotally the more experienced organisations tend towards a purposive
approach. Those with less experience prefer to take a literal approach. The result is during
compliance audits it would be expected to hear disagreements founded on the difference between
the literal approach and the purposive approach as follows. Individual A: I dont agree with your
interpretation, it doesnt say that here on the page. Individual B: I understand what the words say,
but the real intent is

In order to improve shared understanding and prevent against disputes between members of the
regulated community and the regulator the Defence aviation safety regulation should be drafted
and interpreted with a purposive approach.

HOW
Purposive regulation is achieved in three steps:

(i) The regulator drafts regulation with an accompanying purpose statement;

(ii) The regulator communicates its intention to interpret regulation purposively; and

(iii) The regulator and regulated community interpret regulation with its purpose front of mind.

REGULATOR DRAFTS PURPOSIVE REGULATION

The need to even discuss this chapter has been borne out through experience in Defence where the
specific purpose of a regulation is often assumed to be understood but never expressly stated.

Purposive regulation is best achieved by structuring regulation so its purpose is clearly apparent,
stands alone and places the need for the obligation beyond dispute. A statement of purpose should
therefore accompany each regulation. Individual paragraphs and sub-paragraphs of regulation may
also warrant their own statement depending on the circumstances. In the unusual situation where a
regulation is not directly related to a hazard, such as an administrative detail, its purpose should still
be clearly announced. Otherwise over time the regulated community may tend to recast the
regulation to couple with a hazard that was never intended.

42
Attributed to the chief engineer of a commercial organisation providing engineering services to Defence in
2012, in response to enquiries as to how the regulations could be improved (name and company withheld).

10 Ways to Better Aviation Regulation | 3. Take a Purposive Approach 48

The purpose statement should be short, no longer than a paragraph. The statement is not a
description of the regulation. But rather an explanation of the why and what it aims to achieve. One
approach is to construct the purpose from three distinct pieces of information:

(i) The context;

(ii) The hazard; and

(iii) The defence.

For example, using a hypothetical regulation the engineering organisation must enact a review
process upon receipt of technical information. The deconstructed purpose could be as follows.

Context. Over the life of an aircraft, deficiencies in the original design and degradation due to wear,
corrosion and other causes are identified by designers, operators and maintainers of the aircraft.

Hazard. The deficiencies and/or the degradation identified may mean the aircraft is no longer as
safe as when new or as safe as intended.

Defence. This regulation requires engineering organisations to identify, collect, analyse and take
action upon pertinent information to ensure the aircraft remains safe.

These three pieces of information could be reconstructed into the following purpose statement.

Purpose. Over the life of an aircraft, deficiencies in the original design and degradation due to wear,
corrosion and other causes are identified by designers, operators and maintainers of the aircraft.
The deficiencies and/or the degradation identified may mean the aircraft is no longer as safe as
when new of as safe as intended. This regulation requires engineering organisations to identify,
collect, analyse and take action upon pertinent information to ensure the aircraft is safe.

Establishing the purpose of existing regulation may be time consuming. Disagreement amongst
regulatory staff indicates a variation in understanding of the existing rule. This should be expected
and worked through. The result will be better regulation and a common understanding.

REGULATOR COMMUNICATES PURPOSIVE INTENT

All involved in the regulatory process should have a common understanding of regulatory
interpretation. The regulator should widely communicate its intention to interpret purposively.

REGULATOR AND REGULATED INTERPRET PURPOSIVELY

Interpreting regulation starts with a literal interpretation. If the words of a regulation are clear and
free from ambiguity there is no need to consider its purpose any further. If the words of the
regulation are vague and ambiguous then refer to the purpose statement which accompanies the
regulation closely to interpret the regulations meaning. If no statement exists refer to other extrinsic
material such as guidance, advisory circulars or other documentation issued by the applicable
regulator. The ultimate aim here is to put oneself in the shoes of the regulation drafter to consider
what knowledge they had, and what regulatory objective they had in mind.

10 Ways to Better Aviation Regulation | 3. Take a Purposive Approach 49

 Box 8: In-practice some purposive examples

The following examples are from the 2014 TAREGs. The square brackets have been
added to deconstruct the purpose statement into the three recommended pieces of
information to be included.

TAREG 3.5.11. Special Technical Instructions

a. Each applicant must establish a Special Technical Instruction (STI) management

system for issue of those design changes and Authorised Maintenance Data whose
urgency cannot be satisfied by other types of implementing instructions.

Purpose. [Background] An essential element of any airworthiness management system

is the ability to promulgate urgent safety related information to aircraft operators and
maintainers. [Hazard] A subset of this information concerns technical issues requiring
inspection, test, installation, repair or replacement actions that, if not actioned in a timely
manner, could result in an unsafe condition. [Defence] This TAREG requires that a timely
system is in place to prepare, release, receive and action urgent technical information
necessary for the continued safe operation of an aircraft or configuration item.

TAREG 3.5.14. Management of Type Design Data

a. Each applicant must establish a system for management of Type Design data to
ensure only current and relevant data is used for design activities.

Purpose. [Background] To develop a design change it is important to ensure that the

developer is using current and approved Type Design Data. [Hazard] If a design change
is incorporated into an aircraft that does not represent the configuration described by the
Type Design Data used in the designs development, the design assumptions,
specifications and calculations may all be incorrect, and hence the design change if
implemented could be unsafe. [Defence] This TAREG requires that only current and
approved Type Design Data is used for each designs development.

TAREG 2.2.2. Issue of Type Certification Recommendation

a. The DAR must submit to the TAR copies of [various documentation]:

Purpose. [Background] Type certification activities for an aircraft type occur over an
extended period, and the process produces a large amount of documentation. [Hazard] If
this documentation is not properly collected, collated, analysed and presented in a way
that can be understood by decision makers, an unsafe design may be accepted.
[Defence] This TAREG requires that persons seeking a TAR recommendation for issue
of a type certificate to collect, collate, analyse and present information necessary to show
the compliance of a type design against applicable airworthiness standards.

10 Ways to Better Aviation Regulation | 3. Take a Purposive Approach 50

10 Ways to Better Aviation Regulation | 3. Take a Purposive Approach 51
10 Ways to Better Aviation Regulation | 3. Take a Purposive Approach 52
4. UTILISE COMPLIANCE
PROOFS

SUMMARY

WHAT
Define verification criteria against which to assess compliance.

WHY
Compliance proofs increases certainty as to compliance requirements, resulting in minimised
compliance disputes and gold plating.

HOW
Define specific, measurable, demonstrable and repeatable verification requirements for each
regulation to support unambiguous compliance determination.

To utilise compliance proofs is a characteristic of the REGULATE function in the regulatory model.

Regulator Regulated Entity

REGULATE

INTERPRET
MANAGE R ELATIONSHIP

MANAGE RELATIONSHIP

E DUCATE

IMPLEMENT

OVERSIGHT

ADHERE

ENFORCE

10 Ways to Better Aviation Regulation | 4. Utilise Compliance Proofs 53

 UTILISE COMPLIANCE PROOFS IN THEORY

INTRODUCTION
What constitutes compliance? The potential benefits of outcome-based regulation can be offset by
uncertainty in the absence of clearly identified standards for compliance obligations. Furthermore,
where combinations of regulatory styles are used, clarification is needed on whether regulatory
compliance is achieved through inspection of the process or the outcome.

Successful enforcement is dependent on achieving a fair and objective measure as to whether the
regulated entity has achieved a particular outcome. This can be difficult to achieve. Quantitative
measures are only ever proxies for quality of behaviour, can skew activities away from the
achievement of substantive compliance and are themselves open to creative compliance. As a
result organisations can hit the target but miss the point.

COMPLIANCE PROOFS
Compliance proofs are specific, measurable, demonstrable and repeatable test points for each
regulation to support unambiguous compliance determination:

Compliance proofs are explicit requirements against which evidence is assessed to

determine compliance.

Proofs reflect the intent of the regulation but drafted in words useful to assess compliance.

In most cases all compliance proofs must be met to prove compliance against the
regulation.

Compliance proofs are comprehensive. More proofs cannot be added later there should
be no need to unless an error was made in establishing the proofs in the first instance.

Satisfying compliance proofs is mandatory. But proofs are not regulatory because they are drafted
at regulatory staff, although they will provide the regulated community with the expectations that the
regulator is looking for. Nevertheless if a discrepancy exists between a regulation and the
corresponding compliance proofs, the regulation shall always take precedence and the compliance
proof shall be disregarded to the extent it conflicts with the regulation.

To explain lets turn to an example from the Defence Force Disciplinary Act 1982. The DFDA
includes essential characteristics that define success. This is the equivalent of compliance proofs.

DFDA section 24 (1):

A defence member who is absent without leave is guilty of an offence.

The following proofs set out exactly what must be shown to be true in order for the regulation (the
charge) above to be satisfied. To gain a conviction under this charge, the prosecutor must satisfy all
of the proofs:

10 Ways to Better Aviation Regulation | 4. Utilise Compliance Proofs 54

 a. that the accused was a defence member;

b. that the accused was absent from his or her unit (or ship etc) as specified; and

c. that the absence was without authority of anyone competent to give the accused leave.

The proofs listed here in no way suggest what evidence the prosecutor may use to satisfy them,
they simply state the outcome that must be shown. This is the difference between proofs and
evidence. Where possible compliance proofs should not indicate what evidence must (or even
may) be produced. If they did, the regulation has instantly limited the regulated entity for no good
reason.

The following 4 step graphical representation depicts compliance proofs in action.

1. Regulations are 2. Compliance proofs

prescribed to manage identify the characteristics
hazards. of the control to defend
against the hazard

! !
3. Organisations place a 4. Regulator checks the
control in place to satisfy evidence provided is at
the intent of the least to the standard
regulation. required by the proofs.

! !
Figure 14. Graphical representation of the purpose of compliance proofs.

10 Ways to Better Aviation Regulation | 4. Utilise Compliance Proofs 55

SYSTEMS ENGINEERING ANALOGY
For readers familiar with the field of systems engineering, compliance proofs are similar to
verification requirements. Recapping, in systems engineering there are three types of requirements:

Functional requirements what is the system to do?

Performance requirements how well does the system do it?

Verification requirements how do we check the system does what it is meant to?

Function and performance are important parts of a requirement. However the requirement is not
complete until we know a way of verifying that function and performance. In systems engineering it
is good practice to write verification requirements at the time of developing functional and
performance requirements. It is also a good test of the requirements itself. If you do not know how to
confirm the system has met the functional and performance requirements, chances are the
requirement is not well drafted.

This is similar to regulation. There is no point in drafting a regulation when compliance can never be
verified. The system engineering analogy is to think of regulation as the functional and performance
requirements and the compliance proofs as the verification requirements.

COMPLIANCE PROOFS ORIGINATE IN LOGIC

Verifying compliance against individual regulation can result in one of three conclusions:

(i) Compliance;

(ii) Non compliance; or

(iii) Cannot be determined.

The process involved in reaching a conclusion will always involve discretion and judgement on the
part of regulatory staff. But it should be a logical process. The origin of compliance proofs lie in the
theory of logic, specifically deductive reasoning.

DEDUCTIVE REASONING
Deductive reasoning is a logical process of reasoning from one or more statements (called
premises) to reach a logically certain conclusion. The process links premises with a conclusion. A
deductive argument asserts that the truth of the conclusion is a logical consequence of the
premises. Premises are either true or not based on evidence. If and only if the premises are true
then the conclusion must be true.

The truth of the conclusion depends on the truth of all of the premises. If one of the premises is not
true, the truth of the conclusion is no longer guaranteed. In deductive arguments the burden of
proof is shifted from the conclusion to the premises.

10 Ways to Better Aviation Regulation | 4. Utilise Compliance Proofs 56

An example of deductive reasoning follows. This example consists of two premises:

Premise 1: Fred is a dog.

Premise 2: All dogs have four legs.

For a moment, assume that both of these premises are true. If both are true then the conclusion
must be that:

Conclusion: Fred has 4 legs.

In deductive reasoning the truth of the premises guarantees the truth of the conclusion. So the
conclusion could only be rejected on one or both of the following grounds: (i) that Fred was not a
dog; or (ii) that all dogs do not have 4 legs.

PREMISES
Premises are either:

(i) Generally agreed on without the need for further consideration;

(ii) Shown to be valid by supporting evidence; or

(iii) Unsupported assertions.

EVIDENCE
Evidence is an alleged fact which supports a premise. How strongly the evidence supports the
premise will depend on subjective belief. In the Fred the dog example you either believe the
premises or you dont based on the available evidence. If no evidence is provided then it is less
likely that you will believe the premises. One quasi-rational approach to this question is to evaluate
the likelihood of the claim, assuming the evidence is true. If you dont believe the premise then ask
for more evidence.

DEPENDENT VS INDEPENDENT PREMISES

In deductive reasoning theory, the premises (compliance proofs) must be treated as a set. They are
a set of dependent premises. If the logical chain is complete then they actually prove the
conclusion. If one premise is removed and the logical chain is broken then they no longer prove the
conclusion.

COMPLIANCE PROOFS SUPPORT UNITY OF PURPOSE

A piece of writing can usually perform only one purpose well. If the need arises to achieve different
purposes for different audiences, a better way is to write two pieces each directed at the intended
audience. This is the aim of compliance proofs. The regulation is directed at the regulated
community. Compliance proofs are directed at regulatory staff. Both regulation and compliance

10 Ways to Better Aviation Regulation | 4. Utilise Compliance Proofs 57

proofs are intended to be read by both groups but this delineation acknowledges the two different
audiences and aims to achieve a single unity of purpose as depicted here:

ONE
DRAFTER Drafter

DIFFERENT Regulated Regulatory

AUDIENCES Community Staff

COMMON Understanding
UNDERSTANDING

Figure 15. Compliance proofs aim to bring the two audiences of the regulation to the same understanding.

COMPLIANCE PROOFS AND EVIDENCE

In regulatory language the regulation is the conclusion, the compliance proofs are the premises, and
compliance evidence is the evidence to support the premises. If each compliance proof is true, then
compliance with the regulation is achieved. The truth of each compliance proof is determined by
examining compliance evidence.

It is important to distinguish between compliance proofs and compliance evidence. Evidence is not
proof. It satisfies proofs have been met. Compliance evidence is anything a regulated entity
produces to demonstrate a compliance proof is met. Existence of compliance evidence does not
prove compliance. The assessment of that evidence by regulatory staff against the proofs
determines compliance.

10 Ways to Better Aviation Regulation | 4. Utilise Compliance Proofs 58

 UTILISE COMPLIANCE PROOFS IN CONTEXT
The inspector has formed the opinion that the workplace does not comply. That must mean that the
inspector has in mind what compliance would look like. I want the inspector to tell me here and now
43
what needs to be done, so I can get on with doing it.

A BETTER WAY TO REGULATION

Many lawyers spend much of their time interpreting legislation in order that they can decide how it
applies in particular circumstances. Interpretation of legislation is not achieved by merely reading
the text. The common law, previous court interpretations of that legislation and interpretations made
in relation to aspects of other legislation (such as the meaning of particular terms) must also be
considered. Correct interpretation is never really tested until a challenge arises in a court.

Interpretation of Defence aviation safety regulation is not supported by a body of precedence.

Furthermore the regulation needs to be understood and applied by practitioners, not lawyers. As a
result, the regulations need to be all encompassing and aimed at individuals with a wide range of
backgrounds.

With this in mind, the Defence aviation safety regulators should utilise compliance proofs. The
aim of compliance proofs is to set specific, measurable, demonstrable and repeatable test points for
each regulation to support unambiguous determination of compliance.

THE OVERSIGHT PROCESS

The oversight process will always involve discretion and judgement on the part of regulatory staff.
But it should be a logical process. The logic is as follows:

1. The regulated entity is asked to show compliance against the applicable regulation during
initial certification or ongoing audit.

2. The entity provides evidence to support their claim that they are compliant with the
applicable regulation.

3. Regulatory staff assess the evidence provided against each compliance proof and make
either of two determinations:

a. The compliance proof is met when supported by the evidence provided.

b. The compliance proof is not met even when supported by the evidence provided and
to say otherwise is an unsupported assertion.

43
The manager of a small business subjected to a safety audit as quoted in Maxwell, C. (2004) Occupational
Health and Safety Act Review, Victoria.

10 Ways to Better Aviation Regulation | 4. Utilise Compliance Proofs 59

 4. Regulatory staff may need to seek further evidence from the regulated entity. If no further
evidence is available to support satisfaction of the compliance proof, that compliance proof
is assessed as unsupported and the argument for compliance with the regulation is invalid.
Appropriate enforcement action would then be initiated.

5. Where all compliance proofs are met the regulated entity is assessed compliant with the
regulation.

Compliance proofs do not imply that the regulated entity must in some way take on a burden of
proof to demonstrate compliance with the proofs. Compliance proofs are written for regulatory staff.
The regulated entity would be expected to note the proofs but they do not necessarily need to use
them as a checklist.

DRAFTING COMPLIANCE PROOFS

Drafting regulation with compliance proofs means doing the following:

(i) Ensure the regulation is directed at the correct audience. Regulation is aimed at those
obliged to follow it the regulated community. Compliance proofs are for regulatory staff.

(ii) In order to satisfy a regulation each and every proof would usually need to be satisfied
unless indicated otherwise. There should be nothing undocumented that regulatory staff
can refer to in order to determine noncompliance when all compliance proofs are satisfied.

(iii) It may be helpful to parse the language of compliance proofs into standard format to ease
understanding, such as questions requiring a yes/no assessment.

(iv) In existing regulations drafted without the use of compliance proofs, the proofs may already
exist in the regulation as sub-paragraphs and sub-sub-paragraphs, and may simply require
redrafting.

(v) Compliance proofs are more detailed and obvious to draft where the regulation is outcome-
based and not highly detailed. A detailed prescriptive regulation may not warrant any
compliance proofs as it is self evident. In such cases there would only be one compliance
proof by restating the regulation in the correct parsing.

(vi) If compliance proofs are developed based on firm logic they should not need to be
amended or added to in future.

PREFERRED SOLUTIONS
Preferred solutions are neither compliance proofs nor compliance evidence. They are a description
of a solution that would satisfy the proofs and therefore the regulation. The concept of the preferred
solution is a practical way of assisting the regulated community however it is often incorrectly
treated as being mandatory. Therefore it is recommended to minimise use of the term preferred,
instead focussing on examples of acceptable evidence against each compliance proof.

10 Ways to Better Aviation Regulation | 4. Utilise Compliance Proofs 60

ACCEPTABLE MEANS OF COMPLIANCE
The concept of acceptable means of compliance (AMC) remains valid together with compliance
proofs. An AMC is an accepted solution. Like above, it is a description of a solution that is regulatory
compliant. The solution is compliant because it has been shown to have been compliant before
against the compliance proofs. Therefore where an AMC solution is utilised it should not be
necessary to use the compliance proofs to determine compliance. Instead it would require
comparison of the solution against the AMC to ensure no significant deviations were present. Where
deviations were apparent and the AMC has not been followed exactly, regulatory staff would need to
go back to the compliance proofs to determine compliance.

10 Ways to Better Aviation Regulation | 4. Utilise Compliance Proofs 61

 Box 9: In-practice TAREG 2.6.5.a Notification of Unairworthy Conditions

TAREG 2.6.5.a. (As of April 2014) The DAR [Design Acceptance Representative] must
immediately notify the TAR whenever an aircraft has, or is likely to be operated in an
unairworthy condition. The DARs notification must include the:

1. Details of the source of information;

2. Potential risk to safety;

3. Number of aircraft affected; EXAMPLE

4. Proposed action to rectify the unairworthy conditions; and

5. Potential impact on operations.

Section 3 Chapter 2 Guidance. The DAR is responsible for notifying both the TAR and
the Operational Airworthiness Authority (OAA) whenever an aircraft type has a condition
or defect which is unairworthy in accordance with the regulations. This includes
conditions that although they may not be directly unairworthy, may involve a substantial
restriction in operations or operational capability.

To write compliance proofs for this regulation requires a rewording and removal of the
criteria in the regulation and reparsing into proofs.

Possible Regulation. The DAR must notify the TAR within a reasonable timeframe
whenever an aircraft has been, or likely to be operated in, an unairworthy condition.

Compliance Proofs

CP1. Instances of unairworthy conditions occurred during the period. (Yes/No)

CP2. The DAR notified the TAR in each instance. (Yes/No)

CP3. The time taken to notify the TAR was reasonable in the circumstances, such that
the TAR was not unduly delayed from making informed decisions. (T/F)

Compliance Evidence

Examples of evidence to support each proof could be:

1. Collate the number of unairworthy conditions experienced by the platform in question

either through a declaration by the SPO, independent count of risk assessments etc.

2. Correlate the number of unairworthy conditions against number of TAR notifications.

3. Advice from TAR.

Note. The April 2014 TAREG example includes specific detail about the content of the
notification. Treatment of that detail is not addressed in this example.

10 Ways to Better Aviation Regulation | 4. Utilise Compliance Proofs 62

10 Ways to Better Aviation Regulation | 4. Utilise Compliance Proofs 63
10 Ways to Better Aviation Regulation | 4. Utilise Compliance Proofs 64
5. ENSURE SUFFICIENT
PRESCRIPTION

SUMMARY

WHAT
Decompose outcomes into constituent parts so that obligations are comprehensively specified
and ambiguous principles are avoided.

WHY
Enable clarity and certainty of outcome-based regulation protecting against interpretation and
compliance disputes.

HOW
Decompose outcomes and compliance proofs into sub-outcomes and sub-proofs so that
obligations are comprehensively specified.

Sufficient prescription is a characteristic of the REGULATE function in the regulatory model.

Regulator Regulated Entity

REGULATE

INTERPRET
MANAGE R ELATIONSHIP

MANAGE RELATIONSHIP

E DUCATE

IMPLEMENT

OVERSIGHT

ADHERE

ENFORCE

10 Ways to Better Aviation Regulation | 5. Ensure Sufficient Prescription 65

 ENSURE SUFFICIENT PRESCRIPTION IN THEORY

INTRODUCTION
The degree of specificity is another, and different, characteristic of regulation that should be
manipulated to achieve the best regulatory outcome. Highly specific regulation is precise and
detailed. In comparison less specific regulation uses principles and more general language. The
degree of specificity may be considered along a continuum as depicted in Figure 16. Unspecific
language lies at one end and is referred to as principles-based regulation. Highly specific language
lies at the other end and is referred to as rule-based regulation. These descriptions only make sense
in relative terms. There is no such thing as entirely principles-based regulation and likewise for
rules. Each form has advantages and disadvantages.

Degree of Specificity
Lower Higher

PRINCIPLES-BASED REGULATION RULE-BASED REGULATION

Figure 16. The degree of specificity is another, different characteristic of regulation.

Use of prescriptive in the chapter title should not be confused with its use in chapter 2 which refers
to a style based on its timing of intervention, rather than level of detail.44 The level of specificity,
whether detailed or principles-based, is compatible with any intervention point (whether
management-based, prescriptive or outcome-based).

Outcome-based Purposive approach Rules-based

Prescriptive
Hazard-based Principles-based

Literal approach Performance-based

Management-based

44
Specificity means the state of being specific or precise in detail. Accordingly, the title of this chapter could
have been, rather awkwardly, sufficient specificity.

10 Ways to Better Aviation Regulation | 5. Ensure Sufficient Prescription 66

CONTRASTING TWO EXTREMES

PRINCIPLES-BASED REGULATION
Principles-based regulation means relying on high-level broadly stated principles to set the
standards by which members of the regulated community must conduct their business. They set the
express fundamental obligations to be observed. Principles are largely behavioural based and
qualitative. Principle-based terms include fair, care, diligence and reasonable.

An example of a principle-based regulation is from ICAO Annex 8 Airworthiness of Aircraft which

stipulates that details of design and construction shall be such as to give reasonable assurance that
all aeroplane parts will function effectively and reliably in the anticipated operating conditions.45

The benefits of principles-based regulation include:

Offers more flexibility in determining how to comply particularly beneficial for more
knowledgeable entities (stop telling me what to do);

Tends to focus on the purpose where otherwise the purpose can be obscured by the detail;

More difficult to manipulate, reducing creative compliance;

Can simplify the regulatory language, allowing focus on the important aspects;

Provides a basis for open dialogue between regulators and the regulated community,
facilitating a co-operative and educative approach to oversight rather than a box ticking
approach.

RULES-BASED REGULATION
Rules are more specific and precise in their requirements. They tend to be lengthier with sub-
paragraphs and sub-sub-paragraphs, relying less on guidance material as more detail is within the
regulation itself.

Rules can never provide an airtight seal against unimagined or unimaginable contingencies. They
can never be infinitely specific. Nevertheless the benefits of rules over principles include:

Gives comfort to those regulated of knowing their obligations as compared to principles

which may not be sufficiently certain particularly beneficial for less knowledgeable entities
(just tell me what to do).

Less reliance on guidance material to explain what is required, as opposed to principles

where in an attempt at creating consistency a proliferation of internal guidance amongst
regulatory staff may become at odds with the regulation itself.

45 th
ICAO (2010) Annex 8 to the Convention on International Civil AviationAirworthiness of Aircraft, 11 edn,
para 4.1.

10 Ways to Better Aviation Regulation | 5. Ensure Sufficient Prescription 67

 Less open to regulatory creep obligations are less likely to expand under rules as they
are more tightly defined, unlike principles where the intent can change over time.

AIM OF SUFFICIENT PRESCRIPTION

Both of the above approaches share the same goal to achieve good regulation. Two of the most
important criteria for good regulation are certainty and flexibility. Rules tend to achieve more
certainty. Principles tend to achieve more flexibility. The optimal condition therefore is to maximise
the combined benefits. This should be expected to occur somewhere between the two extremes.

CERTAINTY AND FLEXIBILITY

Rules tend to achieve certainty but only up to a point. One must recognise that complete certainty is
rarely attainable. In seeking to provide certainty through an ever-increasing proliferation of detail, not
only will any level of flexibility be lost but certainty can decrease as organisations are never sure if
they have followed all the guidance that the regulator might deem to be relevant. Both regulator and
regulated therefore need to exercise self-restraint: on those regulated in asking for ever greater
predictability, and on the regulators part for trying to provide it.

Whether a regulation is certain depends on understanding.46 Understanding depends not only on

the level of detail provided in the regulation and guidance but whether all those applying the
regulation agree on what it means. In situations of uncertainty, humans tend to draw on past
experiences. This means that experience, not only competency, should be taken into account when
considering the level of detail to be included in a regulation.

For some people, principles have an uncomfortable vagueness about them and leave too much
discretion to those regulated. Even those begging for freedom from the regulators detailed grasp
may not have the time or the inclination to engage in development of solutions to principles. As a
result principles-based regulation is often supported by more specific guidance that may have all the
characteristics of rules.

CONTEXT
Finally, all organisations have a preference as the extent to which rules or principles are better
suited to their culture. So to say principles are too vague makes sense only against a background of
more specific regulation. Likewise to say that a rule is too constraining makes sense only against a
background of an existing precedence for principles. No way is the right way, it is all about context.

46
It is useful to distinguish between certainty and predictability. Certainty with respect to regulation means a
common understanding within and between the regulator and the regulated. Predictable refers to the regulatory
response, and means the regulator will always respond to similar situations in similar ways.

10 Ways to Better Aviation Regulation | 5. Ensure Sufficient Prescription 68

 ENSURE SUFFICIENT PRESCRIPTION IN CONTEXT

A BETTER WAY TO REGULATION

Regulation is rarely congruent with its purpose it is either over inclusive or under inclusive.
Utilising sufficient prescription aims to get the balance right by taking advantages of both principles
and rules-based approach. The goal is a practical, pragmatic response to the competing needs of
clarity, flexibility, simplicity and certainty. With this in mind Defence aviation safety regulation
should be drafted with a sufficient level of prescription, but no more.

HOW
A sufficient level of prescription may be achieved by:

(i) Appreciating that aiming for more precision in the regulation can ultimately create more
uncertainty.

(ii) Ensuring the regulation is directed at the correct audience. The regulation should be written
to the regulated community. If additional detail is considered necessary to assist regulatory
47
staff, consider incorporating this detail into the compliance proofs.

(iii) Ensuring the regulation and compliance proofs are congruent. Compliance proofs should
link to the regulation and must not create new regulatory requirements.

GUIDANCE
The choice of how specific or not to make a regulation should be made on a case by case basis.
The general principle is to be as precise as required but no more. The following factors should be
considered to inform a decision:

As the complexity of regulation increases, certainty moves from being positively associated
with specificity to being negatively associated with specificity.

Perform a stock take of the regulations on occasion, and determine the most rule-based
and the most principles-based regulation in the set. Use as a yardstick or reference.

When the type of hazard to be regulated is simple and stable specific regulation creates
higher certainty than principles.

With complex actions in changing environments principles are more likely to enable
certainty.

Principle-based regulation is likely to see fewer exemptions than detailed prescription. A

continuing trend of requests for exemption may indicate a regulation more detailed than it

47
Compliance proofs are discussed in chapter 4.

10 Ways to Better Aviation Regulation | 5. Ensure Sufficient Prescription 69

 needs to be. Be aware that many members of the regulated community avoid seeking
exemptions unless considered critical because the process itself can be time consuming
and uncertain.

The degree of specificity depends on the choice about how much discretion a regulated
entity should have.

The regulation should be drafted so it is possible for a regulated entity to comply even
without meeting any of the acceptable means of compliance (AMC).

AMC can sometimes work too well in the sense that regulatory staff sometime seem
unprepared to deviate from the written guidance. AMC and other guidance should never be
mandatory or considered a default position. It is but one means that has proven successful
in the past and documented purely for the benefit of others rather than as an obstacle to
alternatives.

Many in the regulated community will treat guidance as binding, particularly the less
knowledgeable entities, unless it is convenient for them not to do so.

If amending existing regulation, base the changes on the general understanding of the
existing regulation rather than the way it is actually worded. In other words when
determining a new level of precision avoid using the existing regulation as a yardstick use
what the regulated community understands it to mean.

Beware of adopting wording directly from other regulators as their level of specificity is
sometimes dictated by factors that are not relevant to the regulatory regime at hand. For
example, EASA regulations are drafted in a manner such that they can be translated into
any language of the European Union.

10 Ways to Better Aviation Regulation | 5. Ensure Sufficient Prescription 70

10 Ways to Better Aviation Regulation | 5. Ensure Sufficient Prescription 71
10 Ways to Better Aviation Regulation | 5. Ensure Sufficient Prescription 72
6. PROVIDE COMPREHENSIVE
EXPLANATION

IN SUMMARY

WHAT

Implement a comprehensive and ongoing education program on aviation safety regulation.

WHY

Regulations and guidance material alone are never enough to convey intent. A comprehensive
program of explanation contributes to consistent safety performance and protects against
conservative application inherent with poor understanding of regulation.

HOW

Implement a comprehensive and ongoing education campaign to explain the regulations.

Comprehensive explanation is a characteristic of the EDUCATE function in the regulatory model.

Regulator Regulated Entity

REGULATE

INTERPRET
MANAGE RELATIONSHIP

MANAGE RELATIONSHIP

EDUCATE

IMPLEMENT

OVERSIGHT

ADHERE

ENFORCE

10 Ways to Better Aviation Regulation | 6. Provide Comprehensive Explanation 73

 PROVIDE COMPREHENSIVE EXPLANATION IN THEORY
48
Effective regulation is not just about carrots and sticks, its also about sermons.

INTRODUCTION
Comprehensive explanation of the regulations to both regulatory staff and the regulated community
is important. Only through extensive regulatory conversations can a shared understanding be
achieved about the objectives of the regulatory regime, about respective roles and responsibilities,
and about interpretation and application of the regulatory requirements.

Rather than adopting a solely punitive approach regulators need to develop a more educative and
advisory approach, one in which it is prepared to validate decisions of members of the regulated
community in certain circumstances. In other words to give straight answers to straight questions.

Equally, regulatory staff need to be able to answer those straight questions. Staff that lack the
relevant technical knowledge will be incapable of enforcing regulation effectively, irrespective of
what regulatory style is adopted.

APPROACH
Comprehensive explanation is characterised by educative material that is comprehensive and
relevant. It is not the intention of this guide to delve into the depths of education theory. The key
point here is to acknowledge that a regulator has an educational responsibility to enable the
regulatory regime to operate most effectively.

The regulator may conveniently categorise the audience into two distinct groups:

(i) Regulated Community. The approach taken in this chapter to providing comprehensive
explanation to members of the regulated community is guided by the ICAO State Safety
Programme requirements on civil aviation regulatory authorities.

(ii) Regulatory Staff. The approach to comprehensive explanation of regulatory staff is also
guided by the ICAO State Safety Programme requirements but also by the ICAO Safety
Oversight Manual.

COMPREHENSIVE EXPLANATION FOR REGULATED ENTITIES

A regulators role is not only to prescribe regulations, but explain, in various formats what they
mean. The aim is to help the regulated community understand the regulations for their own context.

48
Unknown.

10 Ways to Better Aviation Regulation | 6. Provide Comprehensive Explanation 74

The regulator should provide education and promote awareness of safety hazards and risks to the
regulated community. The regulator should have appropriate communication mediums, in addition to
the regulation itself, to facilitate understanding and implementation of regulatory obligation by those
regulated. This may be an integrated medium for all those regulated or dedicated educative
channels. The basic content should include explanation of the regulations and guidance material.
Without going into further detail here, this may involve:49

(i) Establishing processes to communicate regulatory-related information to the regulated

community.

(ii) Developing guidance material on implementation of regulatory obligations.

(iii) Establishing the means to communicate safety-related issues through mechanisms such as
newsletters, bulletins or websites.

(iv) Promoting exchange of safety information with and amongst different organisations and the
regulator.

(v) Facilitating regulatory training for the regulated community where appropriate.

COMPREHENSIVE EXPLANATION FOR REGULATORS

A regulator should provide internal training, foster awareness and encourage two-way internal
communication of safety-relevant information to achieve an effective and efficient regulatory regime.
This may involve:50

(i) Developing internal training policy and procedures.

(ii) Developing a training programme for relevant staff.

(iii) Developing means of communicating safety related information within the regulators office.

As the regulatory system evolves, new processes, procedures or regulations may come into effect
or existing procedures may change. To ensure these changes are effectively understood and
implemented by all personnel involved in regulatory roles it is vital that training and communication
51
remain as ongoing activities. For more information see the ICAO Safety Oversight Manual.

49 rd
ICAO (2013) Safety Management Manual Doc 9859 3 edn, SSP Element 4.2 External training,
communication & dissemination of safety information, para 4.4.19.
50
Ibid, SSP Element 4.1 Internal training, communication & dissemination of safety information, para 4.4.18.
51 nd
ICAO (2011) Safety Oversight Manual Doc 9734 Part B 2 edn, chap 8 Training programme of the regional
safety oversight organization.

10 Ways to Better Aviation Regulation | 6. Provide Comprehensive Explanation 75

 PROVIDE COMPREHENSIVE EXPLANATION IN CONTEXT
A recurring theme in discussions with stakeholders was the lack of understanding of the
52
airworthiness management system.

A BETTER WAY TO REGULATION

For a non-prescriptive regulatory regime to function successfully, members of the regulated
community must have a clear understanding of their obligations. In support of less prescriptive
regulation comes increased focus on education and training. Defence aviation safety regulators
should implement a comprehensive and ongoing education program on aviation safety
regulation. The alternative is to allow less effective or unnecessarily conservative implementation,
potentially resulting in the bare minimum or excessive compliance cost.

To satisfy both the regulated community and regulatory staff there needs to be a standard suite of
regulatory training material, a structured way of delivering it and a governing administration
processes. This can be achieved with the application of resources and focus on aligning regulatory
education to the Defence Training Model (DTM) and Australian Qualifications Framework (AQF).
Furthermore, less formal modes of explanation through the genuine engagement processes are
explained in chapter 10.

Some examples of ways to explain regulation are in the following break-out boxes. Detailed
explanation of how to provide a comprehensive education program is beyond the scope of this
publication.

52
Smith, N.A. (2007) ADF Airworthiness Management System Review, unpublished.

10 Ways to Better Aviation Regulation | 6. Provide Comprehensive Explanation 76

 Box 10: In-practice use of web-based training

Web-based training, once established provides an easy and cost-effective delivery of

training. Another advantage is the flexibility it provides for trainees in terms of not having
to accommodate various schedules or physical locations.

An example is the Defence Airworthiness online course. The course provides an overview
of the Defence airworthiness management system, including its scope, key appointments,
airworthiness regulatory system, airworthiness instruments and the key reference
publications. The course is relevant to anyone interested in or directly involved in Defence
aviation and is a prerequisite for some specific appointments and courses across
Defence.

10 Ways to Better Aviation Regulation | 6. Provide Comprehensive Explanation 77

 Box 11: In-practice use of interactive electronic manual technology

Another approach to comprehensive explanation of regulation is through the utilisation of

interactive electronic manual technology. Comprehensive explanation was described
earlier as characterised by educative material that is comprehensive and relevant. An
interactive regulatory manual is based on the same technology used for interactive
electronic technical manuals. The aim would be to compress the volumes of regulation,
guidance material, previous decisions and other supporting material into a single source
enabling readers to easily find relevant information far more rapidly than in paper
manuals.

With a comprehensive database of information, an interactive electronic manual

technology could:

State the regulations;

Explain the regulations (provide guidance material);

Provide further explanatory material (audio, video, background explanations);

Provide examples of how the regulations have been successfully satisfied

(acceptable means of compliance);

Explain what regulations apply in a particular context (smart filtering);

Support self-assessment of compliance (provide compliance proofs);

Support demonstration of compliance to regulators (create expositions);

Explain what regulatory staff determined and what needs to be done for
compliance (audit reports);

Broadcast approvals under the regulations (create certificates);

Help those regulated ask questions (of the regulators and perhaps even each
other);

Answer questions (database of previously asked questions linked to particular

regulations or topics);

Confirm potential compliance approaches will satisfy regulations (provide

determinations);

Assist in managing acceptable noncompliances (exemptions);

Provide channel for feedback and suggestions (publication improvement request

function)

10 Ways to Better Aviation Regulation | 6. Provide Comprehensive Explanation 78

10 Ways to Better Aviation Regulation | 6. Provide Comprehensive Explanation 79
10 Ways to Better Aviation Regulation | 6. Provide Comprehensive Explanation 80
7. UTILISE SAFETY
INDICATORS

IN SUMMARY

WHAT

Utilise a wide-range of indicators for continuous monitoring and assessment of safety

performance.

WHY

To drive continuous improvement in safety performance.

HOW

Establish measures of safety and critically assess performance against safety targets.

Utilisation of safety indicators is a characteristic of the OVERSIGHT function in the regulatory

model.

Regulator Regulated Entity

REGULATE

INTERPRET
MANAGE R ELATIONSHIP

MANAGE RELATIONSHIP

E DUCATE

IMPLEMENT

OVERSIGHT

ADHERE

ENFORCE

10 Ways to Better Aviation Regulation | 7. Utilise Safety Indicators 81

 UTILISE SAFETY INDICATORS IN THEORY

INTRODUCTION
Traditional regulators saw their responsibilities predominantly in terms of auditing and enforcing
prescriptive regulations. This approach stems from the provisions of the Convention on International
Civil Aviation of 1944 (Chicago Convention). The Convention was based on an assumption that
safety can be controlled by anticipating, at a sufficient level of detail, various operational scenarios
and prescribing for each of them a normative behaviour developed on the basis of previous
experience, lessons learned from accident investigation, and expert knowledge.

To continue the trend of improved safety performance aviation requires a more sophisticated
approach. The role of modern-day safety regulatory authorities world-wide is more proactive,
involving oversight, which is both surveillance and auditing, not only to check for regulatory
compliance but to analyse voluntarily disclosed safety reports and to assess the effectiveness of
safety systems rather than just policing in an attempt to compel compliance.

Safety indicators are tactical monitoring and measurement tools to assess safety performance at
various levels: the organisational, the aircraft type, or the regulatory system as a whole. Monitoring
is done by using basic quantitative data trending tools that generate graphs or charts incorporating
alert/target levels commonly used in technical, quality or reliability control systems.

Safety indicators can be used by anyone with access to the data. In this publication we focus on
their use by the regulator. Safety indicators allow regulators to assure safety performance by:

(i) Assessing the performance of organisations against outcome-based regulations.

(ii) Assessing the performance of organisations against management-based regulation which

includes self-defined safety outcomes.53,54

(iii) Providing intelligence to support prioritisation of inspections, audits and surveys towards
those areas of greater safety concern or need, as identified by the analysis of the data on
hazards, and their consequences and likelihood in operations.

Finally, regulators should be able to give an account of their performance by:

(iv) Assessing the performance of the system as a whole by examining regulatory system level
performance.

53
Readers unfamiliar with the distinction between outcome-based regulation and management-based regulation
should refer to chapter 2.
54 rd
For example ICAO (2013) Safety Management Manual Doc 9859 3 edn, SSP Element 3.1 Safety
performance monitoring and measurement requires organisations to develop safety indicators to determine
whether their system is operating in accordance with safety performance expectations in addition to meeting
prescriptive regulatory requirements.

10 Ways to Better Aviation Regulation | 7. Utilise Safety Indicators 82

DIMENSIONS OF SAFETY INDICATORS
Two dimensions of safety indicators can be distinguished: personal versus process safety, and lead
versus lag indicators.55 The distinction between personal and process safety is relatively clear. The
distinction between lead and lag indicators is somewhat more problematic.

PERSONAL VERSUS PROCESS SAFETY

The first dimension relates to the type of hazard. Personal safety hazards give rise to events that
primarily affect individual workers. Such events could include slips, trips and electrocutions.
Personal safety in workplaces is regulated by work, health and safety legislation.

Process safety hazards give rise to major accidents with the potential to have catastrophic effects
and are the result of dangerous materials or uncontrolled changes in energy states. Such events
could include explosions, fires and large scale impact. Process safety in aviation is regulated by
aviation safety regulators.

The distinction is important for the development of safety indicators. Management of process safety
should not rely on personal safety indicators such as injury or fatality data but rather develop its own
set of process safety indicators. A focus on personal safety at the expense of process safety was an
organisational failing that contributed to the BP Texas City refinery accident in 2005 resulting in the
deaths of 15 workers and injuring 180 others. The subsequent investigation found those responsible
incorrectly used personal safety metrics to drive safety performance. The investigation concluded
that personal safety metrics are not a reliable measure of the risk for a major accident.56

LEAD VERSUS LAG

The second dimension is whether the indicator leads or lags the event which you are trying to
prevent. An accident, incident or near miss is a sequence of events that unfolds over time.
Contributing factors or precursors can be traced back by unfolding the chronology of events as
illustrated in Figure 17.57 Each point in time is a potential source of a safety indicator.

Lag indicators are measures of performance from data sourced following the occurrence of
accidents, incident or near misses. They have a long history of use and are relatively easy to
calculate given the data. Lag indicators provide information on past and current performance that
may or may not reflect future performance. An example lag indicator is accident rate.

55
Hopkins, A (2007) Thinking about Process Safety Indicators, Working Paper 53, Regulatory Institutions
Network (RegNet), Australian National University.
56
US Chemical Safety and Hazard Investigation Board (2007) Investigation Report: Refinery Explosion and Fire,
BP Texas City, report no 2005-04-I-TX, pp 185.
57
Sparrow, M (2008) The Character of Harms: Operational Challenges in Control, Cambridge University Press,
pp 137.

10 Ways to Better Aviation Regulation | 7. Utilise Safety Indicators 83

 The Event
Precursors Accident, incident
or near miss

Time

LEAD LAG
INDICATORS INDICATORS

Figure 17. Safety indicators either lead or lag events depending on what and when they are measured.

Lead indicators are measures of performance from data representing potential precursors to future
accidents, incidents or near misses. The advantage of lead indicators is that they may potentially
indicate negative trends before that which you are trying to affect or ultimately prevent (accident,
incident or near miss). They also enable recognition and celebration of success. Lead indicators
provide information on past and current performance that is more likely to reflect future
performance. An example lead indicator is the level of competence of a workforce based on the
premise that a more competent workforce is less likely to succumb to poor decision making.

DEVELOPMENT OF SAFETY INDICATORS

Functional and process work creates straightforward performance measures about activity or
outputs. The regulator issued X number of enforcement notices indicates the regulator may be
working hard and keeping busy. The regulator made Y number of formal visits and audits indicates
workload volumes. Unfortunately neither of these metrics is particularly indicative of safety
performance or of demonstrating effective hazard control. Whether the regulatory tools are effective
remains unanswered.

The search for indicators that count can be challenging. The key is to ensure the metrics contribute
to answering the question show me the hazards you have controlled? The most important task in
developing safety indicators is to enable measurement of the effectiveness of the controls upon
which the risk control system relies.

10 Ways to Better Aviation Regulation | 7. Utilise Safety Indicators 84

DEFINE HIGHER-VOLUME, PRECURSOR CONDITIONS
The setting of aggregate indicators for catastrophic harms which scarcely happen requires a
nuanced approach.58 To say the accident rate is zero in a small sample size is unsatisfactory
because the nature of catastrophic accidents means there are simply not enough to analyse
effectively. Where accidents are occurring frequently enough to be able to talk about a rate, this rate
can be used to measure safety. Where accidents are rare, we must look to more frequently
occurring precursor events such as major incidents and near misses. From the database of near-
misses and major incidents other reportable patterns can be identified and defences regulated or
otherwise to manage the hazard, and to monitor their impact on specific problems over time.

When the number of near misses and major incidents have been driven down to a level where there
are no longer enough of these to support meaningful analysis then broaden the definition to bring in
more data by focussing on minor incidents, gaining the opportunity to ratchet safety up another
notch. By viewing it this way there are circumstances in which lag indicators are perfectly good
indicators of how well safety is being managed. Furthermore, and referring to Figure 17, depending
on the focus of the event (accident, major or minor incident, near miss), a lead indicator may
become a lag indicator. The distinction between lead and lag indicators therefore is somewhat
irrelevant.59 The most important point is that safety indicators must be chosen so as to measure the
effectiveness of the controls upon which the risk control system relies.

PROVING CAUSALITY
A perennial problem around any metric or indicator is to prove causality between an intervention and
a reduction in safety risk. This issue is fundamental in justifying the value of any hazard-reduction
strategy. Whether the safety indicators show an increase or decrease in risk may have little or
nothing to do with specific interventions and might be just random.

According to the best available theory the only way to prove causality from a particular course of
hazard reduction is to show a continuing trend of reduction in comparison with a similar control
sample. This is often not possible. The best and rather insufficient alternative is to describe the
specific course of action and qualitatively justify how it links to the improvement in safety
performance.60

58
For a discussion on the challenges of measuring performance see Sparrow, M (2008) The Character of
Harms: Operational Challenges in Control, Cambridge University Press, pp 123-48.
59
Hopkins, A (2007) Thinking about Process Safety Indicators, Working Paper 53, Regulatory Institutions
Network (RegNet), Australian National University.
60
Sparrow, M (2008) The Character of Harms: Operational Challenges in Control, Cambridge University Press,
pp 130-4.

10 Ways to Better Aviation Regulation | 7. Utilise Safety Indicators 85

SOURCES OF SAFETY DATA
To support safety indicators the regulator should establish mechanisms to ensure the capture and
storage of data on hazards and safety risks at both an individual and aggregate level. The regulator
should have established mechanisms to develop information from the stored data, and to actively
exchange safety information with all stakeholders as appropriate.

Sources of data include accident and incident notification systems, hazard and risk management
databases, oversight audits, safety management system reporting, contextual data such as aircraft
fleet size, age of fleets and the pedigree of certification bases.

Source data can arise from either mandatory or voluntary reporting systems. Mandatory incident
reporting systems facilitate the collection of actual or potential safety deficiencies. Voluntary
reporting systems facilitate the collection of further actual or potential safety deficiencies that may
not be captured by the mandatory reporting system. Voluntary incident reporting systems must be
non-punitive and afford protection to the sources of information.61 Either way the availability of these
data sources enables the development of further safety indicators.

A word of warning in regards to the use of voluntary disclosed indicators. A review of the FAAs
approach to safety by an independent review team espoused the use of voluntary disclosure
programs as a well accepted component of any modern regulatory toolkit. However they cautioned
when utilising voluntarily disclosed data for safety performance indication as they are a composite
measure, explained as follows.62

The rate at which a regulated entity voluntarily discloses problems is the product of the underlying
problem rate they experience multiplied by the rate at which they report those problems. When a low
underlying problem rate (all is well) combines with a high reporting rate the indicator is mid range.
However exactly the same mid range might be produced if the underlying problem rate were high
and the reporting rate low. When such composite measures move up or down one may not be able
to tell which is different, the underlying problem rate or the willingness to report. So in the absence
of systematic or scientific approaches to unbundling them, it is potentially misleading to interpret
variation in such metrics as either good or bad.

61
Mandatory and voluntary reporting systems are required as part of a State Safety Programme, see ICAO
(2013) Safety Management Manual Doc 9859 3rd edn, SSP Element 3.2 Safety data collection, analysis and
exchange.
62
Report of the Independent Review Team (2008) Managing Risks in Civil Aviation A Review of the FAAs
Approach to Safety, recommendation 6.

10 Ways to Better Aviation Regulation | 7. Utilise Safety Indicators 86

 UTILISE SAFETY INDICATORS IN CONTEXT

A BETTER WAY TO REGULATION

The traditional approach of enforcing compliance with prescriptive regulation is no longer sufficient
and needs to be supplemented with a more proactive reality check process based on safety
performance monitoring. To enable the benefit of proactive oversight, analysis of voluntarily
disclosed safety reports and assessment of the effectiveness of safety systems Defence aviation
should utilise safety indicators.

ICAO APPROACH
An approach for developing and implementing safety indicators comes from the ICAO State Safety
Programme applicable to civil aviation regulatory authorities. The ICAO SSP approach to safety
indicators is considered the most relevant and useful for Defence aviation purposes. The steps
involved in developing and implementing safety indicators are as follows:63

(i) Establish a mandatory or reportable occurrence procedure for certificated/approved service

providers of each aviation sector to report accidents and serious incidents on a mandatory
basis;

(ii) Establish requirements for service providers to have an internal occurrence investigation
and resolution process that documents the investigation results and makes the reports
available to their respective regulatory organisation;

(iii) Ensure there is an appropriate integration, consolidation and aggregation of data collected
from the various aviation providers at the State Safety Programme level. Safety data should
not exist as independent or stand-alone databases at the individual provider only. This
integration aspect should also be addressed for the respective safety databases of the
regulator and the independent accident investigatory authority;

(iv) Establish basic high consequence safety indicators and their associated target and alert
settings. Examples of high consequence safety indicators are accident rates, serious
incident rates and monitoring of high-risk, regulatory noncompliance outcomes;

(v) Establish a State level voluntary reporting system, including information for safety
information protection;

(vi) Establish lower-consequence safety and/or quality indicators with appropriate target and
alert monitoring; and

63 rd
ICAO (2013) Safety Management Manual Doc 9859 3 edn, para 4.4.2 and 4.4.16. Here State refers to ICAO
contracting states countries who are signatories to the Chicago Convention. For our purposes this can be
taken to mean Defence. State Safety Programme can be taken to mean Defence Aviation Safety Program.

10 Ways to Better Aviation Regulation | 7. Utilise Safety Indicators 87

 (vii) Promote safety information exchange and sharing amongst the States regulatory and
administrative organisations and service providers, as well as with other States and
industry.

The remainder of this chapter provides examples of safety indicators. Methods of collection are
beyond the scope of this publication.

EXAMPLES OF INDICATORS
The following is a consolidated list of potential safety indicators.64,65 Each may be used for multiple
purposes, discussed later in this chapter.

ACCIDENTS
Aircraft accidents are the most obvious lag indicator which, in a development aviation community,
should be rare and therefore not a useful indicator on which to base ongoing management
decisions.

INCIDENTS
Aircraft incidents provide a better source of information concerning the performance of aviation
safety and airworthiness. When trended these lag indicators provide a good indication of system
health, particularly when considering an incident as a precursor to an accident if combined with
other precursors.

Such incidents may be grouped by hazard types, such as environmental hazards, materiel failures,
aircrew errors and maintenance errors. Some examples include:

(i) Breakdown of separations;

(ii) TCAS Resolution Advisories;

(iii) Violations of controlled airspace;

(iv) Runway incursions;

(v) Birdstrikes;

(vi) Mission abort rate;

(vii) Engine in-flight shutdown rate;

64
Smith, N.A. (2007) ADF Airworthiness Management System Review, unpublished.
65
Australian Transport Safety Bureau (2005) Aviation Safety Indicators A Report on Safety Indicators Relating
to Australian Aviation, B2005/0046, Canberra. This publication and its predecessors were published as a
response to a recommendation from the 1995 Plane Safe report by the House of Representatives Standing
Committee on Transport, Communications and Infrastructure.

10 Ways to Better Aviation Regulation | 7. Utilise Safety Indicators 88

WORKFORCE
Deficiencies in workforce may lead to an increased likelihood of an unsafe situation. Workforce
focused safety indicators are lead indicators. Some examples are described here.

Workforce competence can roughly be measured as the combination of training and experience.
With respect to the operational workforce, this is adequately captured by the various categorisations
schemes. Generally, maintenance workforce competency is measured as average time in trade,
average time on aircraft type and as average time in rank. However, experience levels are often
obscured by the use of average experience. A better means is the number of trained personnel with
more than two years of competence in their role. For example, an acceptable level of experience
(for technicians with two years on type) might be set at 60%. A figure below this level would indicate
that the supervision in the unit is stretched and thus would be an area of concern.

Workforce strength/task ratio assumes that the unit has been established to meet authorised
tasking rates. The ratio then is simply a ratio of the two percentages: percentages strength
(compared to establishment) divided by the percentage tasking. For example, if the unit is manned
to 80% of its establishment and is being tasked 110%, then the strength/task ratio is 73% and
should be cause for concern. The strength/task ratio needs to be considered in conjunction with the
competence measure.

Other workforce stress indicators of a maintenance workforce under stress may include
unrecompensed overtime (leave in lieu bill), recreation leave liability per member, rate of carried
forward unserviceabilities and cannibalisation rates.

Workforce attitude and culture surveys are essential to fill the data gap that audits cannot
capture. An argument exists that the exposure of the results to senior command conflicts with the
principles that the confidentiality of members must be preserved to ensure ongoing validity of survey
results. This is valid on an individual basis but should not apply to aggregated data sets. This data
should be available on a need to know basis, which includes regulatory staff for the purpose of
assessing safety performance.

OPERATIONAL ACTIVITY
Hours/sorties flown may be used to prioritise surveillance inspections and also to normalise other
data to a rate.

Age of aircraft may be used to prioritise surveillance inspections, relative to other similar aircraft.

Hazard assessments reviewed/updated is not particularly helpful alone but can be normalised
and compared against other organisations/aircraft fleets to determine a comparative level.

Safety meetings as for hazard assessments.

10 Ways to Better Aviation Regulation | 7. Utilise Safety Indicators 89

LEVEL OF REGULATORY COMPLIANCE
Audit compliance is another possible indicator for assessing the safety performance of an
organisation. A crude means is the number of corrective action requests issued through internal and
external surveillance audits. This is not a recommended approach as it tends to measure activity of
the regulator rather than the actual level of compliance. A better indicator is the time to close out
action requests and the quality of the close out information provided.

HOW TO USE SAFETY INDICATORS

Measuring the workforce strength/task ratio at an operating unit tells us nothing about the
performance of the Defence aviation regulatory regime. Likewise the aggregate Defence aviation
accident rate tells us little about the safety performance of an operating unit. Safety indicators need
to be utilised at the right level and for the right purpose. The usefulness of safety indicators are
predicated on asking what are you trying to determine, and what can the indicator tell you?

Additionally, indicators which measure how hard the regulator is working are not at all related to
safety outcomes. Numbers of corrective action requests, numbers of audits, visits and phone calls
are all process measures that are useful for the regulator to understand from a process
management point of view, but are not necessarily indicators related to safety.

OPERATING/SUPPORTING UNIT PERFORMANCE

At the operating or supporting unit level, safety indicators are used to quantify the stressors that are
applicable to an operating or supporting unit that may lead to an increased likelihood of an unsafe
situation by:

(i) Assessing the performance of organisations against outcome-based regulations;

(ii) Assessing the performance of organisations against management-based regulation which

includes self-defined safety outcomes; and

(iii) Providing intelligence to support prioritisation of inspections, audits and surveys towards
those areas of greater safety concern or need, as identified by the analysis of the data on
66
hazards, and their consequences and likelihood in operations;

66
In other words support risk-based oversight, discussed in chapter 8.

10 Ways to Better Aviation Regulation | 7. Utilise Safety Indicators 90

 Box 12: In-practice example safety indicator engine-related mission abort rate

Figure 18 is an example of the engine-related mission abort rate per 1,000 air frame
hours for a fictitious aircraft type. As of 2013 this metric was measured by the regulator in
order to provide an indication of the safety performance of aircraft fleets, as part of
oversight of organisations subject to TAREG 3.5.5 Engine Structural Integrity
Management.

Normal, marginal and high levels are calculated using basic statistical methods. As a lag
indicator this only presents past performance however it is still of value as marginal and
high rates of engine-related mission abort can be further investigated and rectified to
prevent further poor performance.

3.0 6
12monthrollingaverage

High

XMACount/month
2.5 5
MA/1000AFHR

2.0 Marginal 4
1.5 Normal 3
1.0 2
0.5 1
0.0 0
Feb May Aug Nov Feb May Aug Nov Feb May Aug Nov Feb May Aug Nov
10 10 10 10 11 11 11 11 12 12 12 12 13 13 13 13

Figure 18. Engine-related mission abort rate. An example lag indicator for an aircraft fleet.

REGULATORY SYSTEM PERFORMANCE

Safety indicators are also used to assess the performance of the aggregate regulatory regime or
wider aviation safety program. The Defence Aviation Safety Health Indicators on the following page
are such an example.

FURTHER READING
Readers are advised to see the ICAO Safety Management Manual for further information on
developing safety indicators. The manual provides further information on developing safety
indicators for the State Safety Programme level by focussing on aggregated data for assessing
systemic performance and for organisations safety management systems (organisational level).
The same principles apply.

10 Ways to Better Aviation Regulation | 7. Utilise Safety Indicators 91

 Box 13: In-practice 2014 Defence aviation safety health indicators (DASHI)

DASHI have been reported annually since 2009 in the Defence Aviation Safety Health
Assessment (DASHA). The DASHA is intended to provide a consolidated view of key
aviation safety issues and trends across the Defence Aviation Safety Program.

DASHI 01. Level of Regulation Compliance. Assesses the level of regulatory

compliance against Defence airworthiness regulations.

DASHI 02. Workforce Competency Levels. Assesses the general competency and
experience of the aviation workforce to deliver required outcomes. This indicator is
assessed against the three key aviation workforce categories of aircrew, maintenance
and engineering.

DASHI 03. Workforce Strength to Task Ratio. Assesses whether the strength of the
aviation workforce is sufficient to deliver current tasking levels. This indicator is assessed
against the three key aviation workforce categories of aircrew, maintenance and
engineering.

DASHI 04. Workforce Culture. Assesses whether the current workforce culture is
consistent with maintaining a robust and effective airworthiness system, in terms of level
of commitment, level of safety culture and degree to which violating behaviour is present.

DASHI 05. Application of Risk Management Practices. Assesses the risk management
practices.

DASHI 06. Aviation Safety Occurrences. Assesses whether any fundamental trends
exist in the reporting rate of safety occurrences, grouped by accidents, serious incidents,
incidents and events.

DASHI 07. OIP and Aeronautical Information. Assesses whether Orders, Instructions
and Publications (OIP), including the delivery of Aeronautical Information Services, are
being maintained at an appropriate level to ensure the airworthiness of aviation assets.

DASHI 08. Platform Supportability. Assesses the degree to which platform

supportability (including the requirement for upgrade programs) is having a negative
impact on aircrew competency levels and maintenance workload.

DASHI 09. Aviation Support Environment. Assesses whether there are any
deficiencies within the Aviation Support Environment that are having a negative impact on
the health of the Airworthiness Management System, or on discrete aviation assets within
this system.

DASHI 10. Management of ACARs. Assesses the management of Airworthiness Board

Corrective Action Requests (ACARs) in ensuring their timely and effective closure.

10 Ways to Better Aviation Regulation | 7. Utilise Safety Indicators 92

.

10 Ways to Better Aviation Regulation | 7. Utilise Safety Indicators 93

10 Ways to Better Aviation Regulation | 7. Utilise Safety Indicators 94
8. APPLY RISK-BASED
OVERSIGHT

IN SUMMARY

WHAT

Use a robust risk-based assessment process to allocate finite oversight resources most
effectively.

WHY

Improves airworthiness assurance by focussing scarce resources according to risk.

HOW

Use a robust risk assessment process to allocate oversight resources most effectively.

Utilisation of safety indicators is a characteristic of the OVERSIGHT function in the regulatory

model.

Regulator Regulated Entity

REGULATE

INTERPRET
MANAGE R ELATIONSHIP

MANAGE RELATIONSHIP

E DUCATE

IMPLEMENT

OVERSIGHT

ADHERE

ENFORCE

10 Ways to Better Aviation Regulation | 8. Apply Risk-Based Oversight 95

 APPLY RISK-BASED OVERSIGHT IN THEORY
The characteristics of an effective state safety oversight system includeemploying risk
67
management strategies to assist in the effective use of resources.

INTRODUCTION
Oversight is the function by which regulators assure that each member of the regulated community,
and the products or services they produce, is compliant with the regulation. Oversight encompasses
both the review that is done when issuing an approval for the first time and the continuous
surveillance thereafter.

To undertake oversight a regulator will conduct audits, analyse operations, identify deficiencies,
make recommendations, impose operating restrictions, as well as grant, suspend, revoke or
terminate certificates or other approvals.68 More challenging is the expectation that a regulator is to
apply their limited oversight resources across each member of the regulated community, each with
differing responsibilities and safety cultures, to ensure optimal performance across the entire sector
and do it continuously.

Given the main objectives of the safety oversight function, the generic components of a safety
oversight system are:

(i) Monitoring of safety performance;

(ii) Verifying compliance with applicable safety regulatory requirements;

(iii) Safety regulatory auditing;

(iv) Oversight of new or changed systems, operations, products or procedures;

(v) Publication of regulatory instructions or advisory material based on findings of oversight

activities; and

(vi) Generation and maintenance of safety oversight records.

This chapter examines how best to apply finite resources to the oversight function.

FACTORS THAT INFLUENCE OVERSIGHT

A range of factors influence how a regulator may choose to undertake its oversight responsibility,
including:

Financial or human resource constraints;

67 nd
ICAO (2006) Safety Oversight Manual Doc 9734 Part A 2 edn, para 2.4.7.
68
Ibid, para 3.8.4.

10 Ways to Better Aviation Regulation | 8. Apply Risk-Based Oversight 96

 Ability to adapt to changing regulatory environments;

Ability to cater for changes to regulation;

Ability to cater to the dynamics of regulated organisations;

Nature of the organisations under oversight, for example, Defence or industry;

Alignment with other national and military airworthiness authorities;

Ability to respond to safety incidents and events; and

Historical approach to oversight.

RISK-BASED OVERSIGHT
A risk-based approach to oversight means differentiating engagement with each member of the
regulated community based on an assessment of their relative likelihood of noncompliance and the
consequences of any potential noncompliance. This not only enables allocation of oversight
resources where they can do the most good in areas of higher risk and consequence but also
reduces the overall compliance costs by reducing unnecessary inspections or data requirements. To
determine compliance risk a relative risk profile is developed for each regulated entity and used to
differentiate oversight activities.

Risk-based approach to oversight is the pre-eminent trend in oversight across diverse domains
including safety, finance and tax. ICAO use a risk-based approach to auditing of civil regulators
through their Universal Safety Oversight Audit Program Continuous Monitoring Approach (USOAP
CMA) and recommend civil regulators adopt a risk-based approach in their own oversight
,
function.69 70

The risk associated with noncompliance of regulations by members of the regulated community can
be assessed by examining a range of factors, including the nature of the organisation and the
environment in which it operates. The consequence of noncompliance may be assessed against the
nature of the aviation materiel that the organisation is responsible for, its intended use and an
understanding of impact if the aviation materiel failed due to a latent design, construction and
maintenance or supply-chain error. The overall outcome of this risk-based approach determines the
level of oversight that the organisation shall be subject to.

69 rd
ICAO (2011) Universal Safety Oversight Audit Programme Continuous Monitoring Manual Doc 9735 3 edn,
chap 3 The Continuous Monitoring Approach.
70 rd
ICAO (2013) Safety Management Manual Doc 9859 3 edn, SSP Element 3.3 Safety-data-driven targeting of
oversight of areas of greater concern or need, para 4.2.36.

10 Ways to Better Aviation Regulation | 8. Apply Risk-Based Oversight 97

THE RISK-BASED APPROACH
Risk profiles are based on various safety risk indicators that identify or highlight specific information
related to that organisation that needs to be considered in identifying and prioritising the regulators
oversight activities. The USOAP CMA process consists of the following four components:

(i) Collection of safety information;

(ii) Determination of safety risk profile;

(iii) Prioritisation of oversight activities; and

(iv) Updating summary safety capability metrics for each regulated entity.

The risk profile of each regulated entity should be reviewed regularly. In the event of deterioration in
a risk profile, the regulator should:

(i) Increase monitoring;

(ii) Provide or facilitate assistance;

(iii) Consider financial or technical aid; and/or

(iv) Reassess or more closely monitor existing technical assistance projects.

DETERMINATION OF RISK PROFILE

Data used to develop risk profiles may include safety performance indicators related to the specific
regulated entity or relevant wider metrics, results from previous surveillance activities or audits of
the particular entity. More comprehensive intelligence could include safety risk data external to the
regulator itself, such as organisational safety culture surveys.

10 Ways to Better Aviation Regulation | 8. Apply Risk-Based Oversight 98

 APPLY RISK-BASED OVERSIGHT IN CONTEXT
With the introduction of safety management systems, the oversight function has assumed even
71
greater importance.

A BETTER WAY TO REGULATION

The traditional approach of conducting oversight on a fixed-schedule is burdensome and resource
intensive. Defence aviation safety regulators should apply a risk-based safety oversight
approach to identify and prioritise oversight activities. This not only enables allocation of
resources where they can do the most good in areas of higher risk and consequence but also
reduces the overall compliance costs by reducing unnecessary inspections or data requirements.

A PRACTICAL DEFENCE AVIATION APPROACH

The Defence aviation regulators would usually not reduce the level of oversight during initial
compliance assessments, when issuing instruments of organisational or key personnel approvals.
However the risk-based approach and the benefits it entails is enacted following initial assessment.
After initial assessment, the level of oversight can be reduced or increased as required.

The risk-based approach to oversight requires a subjective assessment of an organisations risk of

regulatory noncompliance, over a set period such as twelve months. The assessed risk of
noncompliance is then normalised relative to the other organisations. This approach ensures that
the limited oversight resources are applied for the best effect for the entire Defence system.

One of the benefits of the aviation regulator existing within Defence is that it is able to use risk
based oversight processes more easily because it is more familiar with the owner, the operators and
their risks.

COLLECTION OF SAFETY INFORMATION

Safety information could consist of any of the example safety indicators discussed in chapter 7.
Suffice to say good quality risk-based oversight is challenging without access to an adequate array
of safety indicators.

DETERMINATION OF RISK PROFILE

A simple tool to support risk-based oversight is assigning a risk-hazard index (RHI) based on
established criteria. Organisations are mapped on the RHI and allocated a surveillance level based
on the band in which they fall.

71 nd
ICAO (2006) Safety Oversight Manual Doc 9734 Part A, 2 edn, para 3.8.8.

10 Ways to Better Aviation Regulation | 8. Apply Risk-Based Oversight 99

 Box 14: In-practice an example risk-hazard index for determining risk profiles

Almost Certain Raised Targeted

Most Likely
Raised

Likely Routine

Not Likely Reduced

Rare Minimal

Likelihood

Consequence Insignificant Minor Moderate Major Critical

CONSEQUENCE OF ORGANISATIONAL NONCOMPLIANCE WITH REGULATION

The underlying purpose of the horizontal axis of an RHI is to differentiate organisations based on the
consequences of regulatory noncompliance, with a clear understanding that noncompliance means
that the defences may not be sufficiently strong to protect the aviation materiel from latent error.
Latent design and maintenance errors are a problem because they might not be discovered before
the equipment is operated, and a loss of function during operation could jeopardise safety, capability
or efficiencies.

As such the following factors can be used to collectively help distribute organisations based on the
consequences of organisational noncompliance with the regulations:

(i) Whether the aviation materiel is classified as whole aircraft/engine, or whether the aviation
materiel is classified as equipment-only with a functional/physical interface to the aircraft.

(ii) The extent to which an organisation conducts maintenance or design on whole of

aircraft/engines, or maintenance or design on safety critical systems.

10 Ways to Better Aviation Regulation | 8. Apply Risk-Based Oversight 100

 (iii) The extent to which a potential latent error in maintenance or design could be discovered,
or not, by a different organisation that is further along the chain before the equipment is
used by an operator.

(iv) The total number of people placed at risk of injury should the aircraft/engine/equipment
lose a safety critical function due to latent error.

(v) The potential impact on safety, capability or efficiencies should aviation equipment - with a
functional/physical interface to the aircraft/engine - lose its most critical function due to a
latent error.

Many of these factors are interrelated, while others are mutually exclusive. This complex
relationship can make it difficult to assess organisations in a standardised manner. A better
understanding may be achieved by incorporating the PBP bow tie described in chapter 1.

LIKELIHOOD OF ORGANISATIONAL NONCOMPLIANCE WITH REGULATION

The underlying purpose of the vertical axis on the RHI is to differentiate organisations based on the
likelihood of organisational noncompliance with the regulations in the near term. An organisations
likelihood assessment is expected to be somewhat volatile, while the consequence assessment is
expected to be somewhat fixed. This means the organisations will move vertically within an RHI
column as the organisation experiences internal changes, or suffers pressures from the environment
within which it operates.

While past compliance history will be a factor in determining a likelihood level, past compliance
history will not be the prime driver. Rather the likelihood ratings are to be predictive in nature and
regulatory staff are still expected to exercise professional judgement in assigning a likelihood rating.

LIMITATIONS IN USING RISK HAZARD INDEX

RHIs are usually developed for a very specific context and used internally by the owning
organisation to provide a simple and reasonably reliable means of assessing risk. However, when a
RHI is used out of context the results can become nonsensical, which often occurs when external
organisations and stakeholders use the relative and subjective results of a RHI as an absolute
measure of risk, performance or health. These limitations need to be understood and clearly
explained when surveillance level descriptors are communicated to organisations and stakeholders.

Furthermore, surveillance level descriptors are not measures of organisational performance or

health themselves. The extent to which an organisation complies with regulations is but one
contributor in assuring safety performance. Hence RHI cannot and should not be used to infer a
measure of organisational health because the RHI neither assesses workforce attitudes nor
assesses the culture of an organisation in determining the surveillance level. Organisations may
operate in a fully compliant manner and yet continue to have incidents in which individual people
demonstrate unsafe behaviours, indicative of inappropriate workforce attitudes and/or poor

10 Ways to Better Aviation Regulation | 8. Apply Risk-Based Oversight 101

organisational cultures. A compliant culture is one example, where an organisation focuses entirely
on regulatory compliance at the expense of measuring safety performance.

10 Ways to Better Aviation Regulation | 8. Apply Risk-Based Oversight 102

10 Ways to Better Aviation Regulation | 8. Apply Risk-Based Oversight 103
10 Ways to Better Aviation Regulation | 8. Apply Risk-Based Oversight 104
9. TAKE A GRADUATED
RESPONSE

IN SUMMARY

WHAT

Escalate enforcement remedies to elicit acceptable and compliant behaviour proportional to the
observed behaviour and intent of the regulated entity.

WHY

Incentivise strong safety culture rather than penalising minor infractions.

HOW

Escalate remedies to elicit acceptable and compliant behaviour proportional to the observed
behaviour and intent of the regulated entity.

Graduated response is a characteristic of the ENFORCE function in the regulatory model.

Regulator Regulated Entity

R EGULATE

INTERPRET
MANAGE RELATIONSHIP

MANAGE RELA TIONSHIP

E DUCATE

IMPLEMENT

OVERSIGHT

ADHERE

ENFORCE

10 Ways to Better Aviation Regulation | 9. Take a Graduated Response 105

 TAKE A GRADUATED RESPONSE IN THEORY

INTRODUCTION
Regulatory enforcement is the function undertaken by the regulator to ensure compliance when
noncompliance is observed.

Noncompliance may be for a variety of reasons: the regulated entity may not know about their
regulatory obligations, may not agree with them, may not be capable of complying with them, may
find them too costly to comply, or simply may not care. The challenge for regulators is to develop
enforcement approaches which address all these obstacles. Much innovative work has been
undertaken in Australia on this topic. Some theory is presented here to introduce the graduated
response to enforcement.72

PERSPECTIVES ON COMPLIANCE MOTIVATORS

Regulatory theorists explain the motivators behind regulatory compliance in various ways. According
to classical deterrence theory compliance is one dimensional: individuals are self interested and
comply if it is in their best interest by balancing the risks of detection of non compliance. A more
modern theory has two elements:73

(i) Negative motivations arise from fears of the consequences of being found in violation of
regulatory requirements; or

(ii) Affirmative motivations arise out of good intentions and a sense of obligation to comply.

The line between the two is not always clear because they interact in influencing compliance.
Nonetheless, the distinction is useful in helping to think about better regulatory outcomes.

The basic logic of enforced sanctions or punitive measures of enforcement is that of a criminal law
model of deterrence. From this perspective, individuals and organisations comply because they fear
the consequences of being found in violation of the regulatory requirements.

The second perspective is that of an unwritten social contract involving expectations and obligations
on the part of both regulators and regulated. Regulators approve either explicitly with licenses or
certificates, or implicitly by not issuing sanctions actions to address harms taken by members of
the regulated community, while in exchange members offer compliance with regulations and gain
social benefit associated with that compliance. This contract is more likely to develop over repeated
actions together and with a shared common goal. Inherent in this perspective is a more positive
assessment of the willingness of the regulated community to comply, to do the right thing.

72
Graduated response described here is referred to in academic literature as responsive regulation.
73
May, P.J. (2004) Compliance Motivations: Affirmative and Negative Bases, Law & Society Review, vol 38
no 1.

10 Ways to Better Aviation Regulation | 9. Take a Graduated Response 106

The notion of the social contract is tempered somewhat by the understanding that social norms act
less upon complex organisations than individuals. The study of organisational deviance tells us that
one should think of personnel as actors who assume certain roles as defined by the organisation,
not by their personality. This explains how an organisation can act in a way that is prejudicial to
safety in ways that most personnel within the organisation individually would not condone.

EFFECT OF ENFORCEMENT STYLE ON COMPLIANCE MOTIVATORS

The approach by regulatory staff to interaction with the regulated community constitutes
enforcement style. Enforcement style also affects compliance motivators, so it is useful to
understand the different styles and how they will affect compliance.

Case studies have shown enforcement styles can vary along two dimensions:74

(i) The degree to which enforcement style is facilitative from helpful and friendly, to
unhelpful and threatening; and

(ii) The degree to which enforcement style is formal from flexible and less picky, to rigid and
picky.

While different combinations of these styles are possible, of interest here is how more or less of
each influences motivations to comply. Increased facilitation fosters affirmative motivations while
detracting from negative motivations. This is intuitive, because facilitation, by definition, leads to a
greater understanding of the basis of regulations and means for complying with them.

Increased formalism contributes to negative motivations while detracting from affirmative

motivations. Formalism adds little to the understanding of the basis for rules and as such does not
enhance affirmative motivations. Formalism undermines affirmative motivations if the regulatory staff
member is indifferent, threatening or similar, because it undermines confidence in the system.

Formality and minimal facilitation is an appropriate enforcement approach in some circumstances.

Informality and high facilitation is also an appropriate approach in some circumstances. Naturally,
combinations exist in between these two extremes.

GRADUATED RESPONSE
Armed with a better understanding of the motivations of the regulated community it becomes clearer
as to why punitive enforcement action in all cases of noncompliance makes the job of the regulator
more difficult. The regulator needs a better enforcement approach that avoids using persuasion on
those with no will to comply and using punishment on those trying to do their best. Such an
approach exists and is called the graduated response to enforcement.

74
May, P.J. (2004) Compliance Motivations: Affirmative and Negative Bases, Law & Society Review, vol 38
no 1.

10 Ways to Better Aviation Regulation | 9. Take a Graduated Response 107

The basic principle of the graduated response approach to enforcement is for the regulator to
respond to the culture, conduct and context of those they seek to regulate when deciding whether a
more or less interventionist response is needed, rather than default to punitive measures.75

The theory of responsive regulation, upon which the graduated response method is based, has
been an influential policy idea and utilised across many Australian jurisdictions including the
Australian Taxation Office, Australian Securities and Investment Commission and Office of
Transport Security (OTS).

The 1992 book Responsive Regulation by Ian Ayres and John Braithwaite was influential in defining
an enforcement pyramid, up which regulators would progress depending on the seriousness of the
regulatory risk, and the noncompliance of the regulated business. An example is in Figure 19. The
theory is that regulatory compliance is best secured by persuasion in the first instance, with
inspection, enforcement notices and penalties being used for more risky businesses further up the
pyramid.

Responsive regulation strikes a balance between, and helps to answer the question, when to punish
and when to persuade. Its popularity has come about because not only is it simple but it reconciles
better than any other theory the clear empirical evidence that sometimes punishment works and
sometimes it backfires and likewise with persuasion.

Attitude to compliance Possible compliance responses

Seriously disengaged Revoke license/certificate

Able but not willing Enforceable undertakings

Willing but not always able Counselling

Willing and able Educate

Fully compliant Compliance responses aim to Maintain awareness

promote full voluntary compliance

Figure 19. An example enforcement pyramid.

75
Wood, C et al (2010) Applications of Responsive Regulatory Theory in Australia and Overseas, Occasional
Paper 15, Regulatory Institutions Network (RegNet), Australian National University.

10 Ways to Better Aviation Regulation | 9. Take a Graduated Response 108

The theory states that compliance is more likely when a regulator displays an explicit enforcement
approach and sticks to it. The left side of the pyramid represents the regulated entitys attitude to
compliance. At the base is fully willing and able, in the middle is willing but not always able and at
the top is unwilling and recalcitrant.

The right side represents the regulators response. At the base are advisory and persuasive
measures, the middle mild administrative sanctions and at the top are more punitive sanctions,
determined to be sufficiently undesirable to halt the behaviour of the most determined offenders.

There are three critical elements to its implementation. Firstly, a full explanation of the process to
the regulated community so they know what to expect. Secondly, inherent respect for each
member of the regulated community by always executing enforcement procedures from the bottom
of the triangle. Thirdly, an escalation of response in the absence of a genuine effort by a member of
the regulated community to meet the required standard. The speed of escalation should depend on
an assessment of the organisations motivational stance, prior interaction, and capacity to meet the
regulations:76

(i) Entities found noncompliant who were willing and made a genuine effort should receive
education and counselling;

(ii) Entities not willing or who lack genuine effort should be met with deterrence tools if
unwilling to change their attitude rapidly; and

(iii) Incompetent and/or irrational actors should be incapacitated with limited notice.

Escalation and de-escalation is possible throughout the course of the relationship between the
regulator and regulated, and indeed possible within the same encounter.

76
Baldwin, R. & Black, J. (2007) Really Responsive Regulation, LSE Law, Society and Economy Working
Papers 15/2007, London School of Economics and Political Science.

10 Ways to Better Aviation Regulation | 9. Take a Graduated Response 109

 TAKE A GRADUATED RESPONSE IN CONTEXT
The Panel recommends thatthe Civil Aviation Safety Authority reintroduces a use of discretion
procedure that gives operators or individuals the opportunity to discuss and, if necessary, remedy a
77
perceived breach prior to CASA taking formal action.

A BETTER WAY TO REGULATION

The traditional enforcement strategy is to apply punitive measures to any noncompliant member of
the regulated community and to threaten anyone who is compliant to remain compliant. A better way
is to take an enforcement approach that recognises and takes account of compliance motivators. In
order to improve the level of compliance and enable the benefit of proactive oversight Defence
aviation safety regulators should take a graduated response to regulatory enforcement.

ENFORCEMENT ACTION IN CIVIL AVIATION

ICAO takes a relatively simple approach to expressing a graduated response to regulatory
enforcement (without using such terminology) by promoting the following principles to State
78
regulatory authorities:

(i) The States regulatory authority will take action against those who consistently and
deliberately operate outside civil aviation regulations;

(ii) The States regulatory authority will seek to educate and promote training or supervision of
those who show commitment to resolving safety deficiencies; and

(iii) The States regulatory authority will give due and equitable consideration to distinguish
premeditated violations from unintentional errors or deviations.
79
In Australia CASA implement these principles and bring about compliance in four ways:

(i) Assisting the regulated community to comply, generally and on an individual basis through
educational activities, advice and safety promotion.

(ii) Encouraging or exhorting compliance through counselling, remedial training or infringement

notices.

(iii) Compelling compliance through the suspension or cancelling of authorisations, the

variation of authorisations which may include imposition of conditions, and by enforcing
voluntary undertakings.

77
Australian Government (2014) Aviation Safety Regulation Review, Canberra, recommendation 18. The review
was commissioned by the Australian Government to improve the civil aviation regulatory system.
78 rd
ICAO (2013) Safety Management Manual Doc 9859 3 edn, appx 10 to chap 4.
79
Civil Aviation Safety Authority (2013) CASA Enforcement Manual, v 4.3, chap 2, pp 4.

10 Ways to Better Aviation Regulation | 9. Take a Graduated Response 110

 (iv) Initiate penalty action (by recommendation to the Commonwealth Director of Public
80
Prosecution) including fines and custodial sentences.

CASA only refers to the latter two as enforcement actions. The first two are referred to as
compliance actions. When deciding whether to take enforcement actions CASA considers the
nature of the noncompliance including whether it was intentional, nature of the evidence, the kind of
action required to address the noncompliance, the obligation to be fair and consistent, and any
policy of the Commonwealth Director of Public Prosecutions.81

ENFORCEMENT ACTION IN DEFENCE AVIATION

The enforcement approach should involve the development of a Defence aviation safety regulation
enforcement triangle tailored to account for the motivations of the regulated community.
Furthermore, it should provide advice to regulatory staff as to the appropriate use of
formality/informality and facilitative/threatening approaches as follows:

(i) A compliant position due to affirmative motivations should be reinforced with a facilitative
and informal enforcement approach;

(ii) A compliant position due to negative motivations should be reinforced with a formal and
punitive approach;

(iii) A noncompliant position with an affirmative approach should be corrected with a facilitative
and informal enforcement approach;

(iv) A noncompliant position with a negative motivation should be corrected with a formal and
more threatening approach; and

(v) A noncompliant position due to intentional disregard should be corrected with the most
formal and most threatening approach.

80
Only a court (not CASA or the Commonwealth Director of Prosecutions) has the authority to impose a penalty
for a breach of the Civil Aviation Act or Civil Aviation Safety Regulations.
81
Civil Aviation Safety Authority (2013) CASA Enforcement Manual, v 4.3, chap 3, pp 15.

10 Ways to Better Aviation Regulation | 9. Take a Graduated Response 111

 Box 15: In-practice an example aviation safety regulation enforcement triangle

Figure 20 is an example of a possible enforcement triangle which could form the basis of
an enforcement policy.

Attitude to compliance Regulators response

Revoke certificate and

Seriously disengaged advise Defence
Airworthiness Authority

Able but not willing Corrective Action

Requests
Willing but not
always able Observations and
Counselling

Willing and able Educate

Maintain
Fully compliant awareness

Recognition
for good work

Figure 20. An example aviation safety regulation enforcement triangle.

10 Ways to Better Aviation Regulation | 9. Take a Graduated Response 112

10 Ways to Better Aviation Regulation | 9. Take a Graduated Response 113
10 Ways to Better Aviation Regulation | 9. Take a Graduated Response 114
10. ESTABLISH GENUINE
ENGAGEMENT

IN SUMMARY

WHAT

Aim to develop mutual respect, appreciating the natural tension between regulators and the
regulated community.

WHY

Regulatory outcomes will be enhanced if mutual respect exists between regulators and the
regulated community.

HOW

Engage with the regulated community using formal and informal approaches. Appreciate the
natural tension between regulators and the regulated community.

Genuine engagement is a characteristic of the MANAGE RELATIONSHIP function in the regulatory

model.

Regulator Regulated Entity

REGULATE

INTERPRET
MANAGE R ELATIONSHIP

MANAGE RELA TIONSHIP

E DUCATE

IMPLEMENT

OVERSIGHT

ADHERE

ENFORCE

10 Ways to Better Aviation Regulation | 10. Establish Genuine Engagement 115

 ESTABLISH GENUINE ENGAGEMENT IN THEORY
Regulators, by their conduct in interpreting, administering and enforcing regulatory requirements,
can take considered, well designed regulation and produce regimes which discourage
82
compliance, squander government resources or add to business costs and delays.

INTRODUCTION
Genuine engagement theory is almost a contradiction in terms. Engagement is after all practical. But
it is critical. According to the Productivity Commission regulator behaviour can potentially have as
large an effect on compliance costs as the regulations themselves.83 The intent of this chapter is to
provide a clear guide to regulatory staff for better engagement between the regulator and the
regulated community.

AIM OF GENUINE ENGAGEMENT

In a regulatory environment the relationship between parties ought to be one of mutual respect.
Such is the aim of genuine engagement. No worse, no better. Tensions that arise from time to time
between parties are expected and not necessarily a bad thing. But punitive regulatory responses at
all times is most definitely destructive. The result of a heavy handed approach has been seen in
Australian civil aviation where many in industry have been critical of, and reportedly now actively
avoid, the safety regulator unless necessary.84

BENEFITS OF MUTUAL RESPECT

Interacting genuinely with the intent of establishing mutual respect between the regulator and the
regulated community opens advantages that would otherwise be inaccessible.85

Regulated Community. Members of the regulated community are more likely to want to take an
interest in, rather than avoid the regulators operations and thereby better understand their
compliance expectations.

Regulator. Trust fosters voluntary compliance. Respect will also see improvement in the
understanding of the motivation and abilities of the regulated community to meet compliance
obligations and thus informs the graduated response approach to enforcement (discussed in

82
Australian National Audit Office (2007), Administering Regulation: Better Practice Guide, Canberra.
83
Productivity Commission (2013) Regulator Engagement with Small Business, Research Report, Canberra.
84
Australian Government (2014) Aviation Safety Regulation Review, Canberra, s. 4.66 The use of discretion.
The review was commissioned by the Australian Government to improve the civil aviation regulatory system.
85
Productivity Commission (2013) Regulator Engagement with Small Business, Research Report, Canberra.

10 Ways to Better Aviation Regulation | 10. Establish Genuine Engagement 116

chapter 9). Furthermore it helps to identify compliance challenges and support proactive strategies
to address them.

RISKS OF MUTUAL RESPECT

A closer relationship does not come without risk. A regulator that does not appropriately manage its
close relationships is in danger of capture, which ultimately can compromise the integrity of the
regulatory regime. Capture is when a regulator draws so close to those whom it regulates that the
agency ends up elevating others concerns at the expense of their own core mission.

Once captured there is no quick fix. With the integrity of the regime destroyed, incentive for
compliance reduces. The result is a step backwards into a punitive compliance culture that, as we
have seen elsewhere in this publication, is not the best way to manage the myriad of potential
hazards exposed to aviation.

REGULATORS ARE NOT CUSTOMER FOCUSSED

Enforcing regulatory requirements while adhering to customer service principles can be a difficult
balance to achieve. So customer focussed strategies typical of the private sector are not as helpful
for regulators as it may seem. All regulators have to deal with the fundamental reality that being a
regulator involves different relationship characteristics, including:86

1. A regulators job is to deliver obligations, not products or services. Trying to emulate the
private sectors treatment of customers will not lead to improved regulatory outcomes.

2. In most cases the person dealt with by the regulator is not usually paying for the service,
has no choice as to whether or not to accept the service, and is often not the one that
benefits from the service. The person or their organisation is obliged to absorb a loss for
the benefit of a greater good. So there is generally no reason to expect that the person
dealt with will be pleased.

3. Regulators need a more nuanced vocabulary than business to describe the parties to
regulatory action. Using the term customer is confusing, misleading and potentially
dangerous. Instead regulators use a broader set of terms to describe the various parties
they deal with. Common terms include stakeholders, regulated community, regulated
entities and industry.

4. Regulators are obliged to treat all parties with respect and dignity. This notion should not in
any way conflict with an uncompromising focus on regulatory objectives.

86
Sparrow, M (2000) The Regulatory Craft: Controlling Risks, Solving Problems, and Managing Compliance,
Brookings Institution Press, chap 4 Customer service: merits and limits.

10 Ways to Better Aviation Regulation | 10. Establish Genuine Engagement 117

TECHNIQUES FOR MANAGING RELATIONSHIP
Genuine engagement is an approach. Establishing mutual respect is the desired outcome. How
does the regulator achieve this soft target? According to the Australian National Audit Office the
answer lies in clarity and openness which are enhanced when:87

Relationship goals are defined and agreed;

All parties understand their roles, commitment and obligations;

Modes of interaction are available that facilitate two-way communication; and

Procedures for handling disagreements are in place.

DEFINE RELATIONSHIP GOALS

Defining relationship goals means having an understanding between regulators and regulated as to
the relationships purpose. Once defined it is then possible to develop mechanisms and processes
of interaction. For example a collaborative relationship looks very different to one of inform.

INFORM CONSULT INVOLVE COLLABORATE EMPOWER

Regulator Regulator obtains Regulator and Regulator and Regulator places

provides industry feedback from industry work industry create rulemaking
with balanced, industry on directly partnership to decision-making
objective, analysis, throughout the develop power in the
GOAL

accurate and alternatives rulemaking alternatives, hands of industry.

consistent and/or outcomes. process to ensure make decisions
information concerns and and identify
regarding needs are preferred
rulemaking. understood and solutions.
considered.

Regulator keeps Regulator keeps Regulator works Regulator looks to Regulator

REGULATORS ACTIONS

industry informed. industry informed, with industry to industry for implements what
listens to and ensure concerns advice in industry decides.
acknowledges are reflected in formulating
concerns, the alternatives solutions;
provides developed and incorporates
feedback on how provides advice into the
industry input feedback on how outcome to the
influences input influences maximum extent
outcome. outcome. possible.

Figure 21. Relationship goals and corresponding actions of the regulator.

87
Australian National Audit Office (2007) Administering Regulation: Better Practice Guide, Canberra, chap 4
Relationship management.

10 Ways to Better Aviation Regulation | 10. Establish Genuine Engagement 118

Referring to Figure 21 above to collaborate and empower involve a significant risk of regulatory
capture. Such an approach may be appropriate however for discrete tasks such as where the
regulator is unable to resource or influence a lower priority task and it is in the interest of safety
assurance for the regulated community to develop a solution on its own.

ARTICULATING ROLES, OBLIGATIONS AND COMMITMENTS

Articulating the roles, obligations and commitments of the regulator and the regulated community
defines the terms and conditions for the relationship. For example a client service charter is a tool
that enables a regulator to define the parameters of an interactive relationship.

Another tool is a regulatory code of conduct, developed in consultation with stakeholders. A

regulatory code of conduct outlines the regulators priorities and enables the regulated community to
formulate their own expectations about how the regulator will administer regulation. Details could
include:

Consultation processes for developing regulations;

Provision of information on regulations and compliance requirements;

Approaches to enforcement and penalties, including where breaches are voluntarily

notified;

Processes for dealing with complaints; and

Timeframes for responses.

Such a code could also define the responsibilities of members of the regulated community.

CONSULTATION PROCESS FOR DEVELOPING REGULATION

The following principles should be considered when developing consultation processes:88

Continuous. Relationships with stakeholders already exist. If seeking out people to discuss the
policy proposal when developing consultation documentation then the point is missed. Build
consultative relationships whenever the opportunity presents itself, not merely when needed.

Broad-based. Consultation should capture the diversity of stakeholders affected by the proposed
changes. Use intermediaries where efficient. For example smaller organisations are more likely than
larger organisations to rely on third parties to receive information on regulatory requirements,
including industry and professional associations. Similarly, junior staff within an organisation are
more likely to receive regulatory information from their senior managers rather than directly from the
regulator. Ensure the approach is suitably tailored to suit the diversity.

88
Australian Government (2014) Australian Government Guide to Regulation, Canberra. These seven
consultation principles are articulated in Regulatory impact statement question 5 who will you consult and how
will you consult them?

10 Ways to Better Aviation Regulation | 10. Establish Genuine Engagement 119

Accessible. Channels for consultation should be relevant to the groups with whom you are
consulting. Consider strategies to assist stakeholders who might be significantly impacted by the
regulation but do not have the resources and/or ability to prepare a response.

Not burdensome. Many people you wish to consult with have full time jobs so dont make
unreasonable demands on them or assume they have unlimited amounts of time to devote to the
consultation process. Too much written material may result in information overload and lack of
clarity on priorities.

Transparent. The regulator should explain the objectives of the consultation process and when and
how the final decision will be made.

Consistent and Flexible. Consistent consultation processes demonstrate you are an experienced
and professional regulator. But dont be a slave to the process if there is a simpler way to consult in
the circumstances.

Evaluation and Review. Regulators should evaluate consultation processes and continue to
examine ways of making them more effective.

Not Rushed. When detailed information is provided as a part of consultation, people need time to
understand it, consider it and respond. Give as much time as is reasonable. This could be as much
as 60 days and should not be less than 30 days. Rushing the process is likely to lose trust and
respect of stakeholders.

A Means Rather than an End. Consultation should be used to improve decisions, not as a
substitute for making decisions.

Furthermore, the Regulatory Impact Statement (RIS) is a widely used tool that should be
incorporated into the regulation development process. The RIS is designed to encourage rigour,
innovation and better policy outcomes from the beginning. In summary the RIS seeks policy and
regulatory developers across to government to ask seven questions:89

(i) What is the problem you are trying to solve?

(ii) Why is government action needed?

(iii) What policy options are you considering?

(iv) What is the likely net benefit of each option?

(v) Who will you consult about these options and how will you consult them?

(vi) What is the best option from those you have considered?

(vii) How will you implement and evaluate your chosen option?

89
Australian Government (2014) Australian Government Guide to Regulation, Canberra, pp 5.

10 Ways to Better Aviation Regulation | 10. Establish Genuine Engagement 120

FACILITATING TWO-WAY COMMUNICATION
Communication mechanisms established by a regulator are influenced by the:

Outcomes sought from the relationship;

Characteristics and motivations of key stakeholders;

Cost of establishing, maintaining and operating the interface for the regulator; and

Capacity of the regulated community to use a wide-range of mechanisms.

Effective communication methods include electronic interfaces, single points of contact, formal
consultative arrangements, conferences and informal ad-hoc forums.

PROCEDURES FOR HANDLING DISAGREEMENTS

Even though all the above may be in place, disputes will still arise. As a consequence the regulator
should develop a well-defined dispute handling process to manage these circumstances. For
guidelines to develop a dispute handling process see the ANAOs Administering Regulation: Better
Practice Guide.90

UNDERSTANDING COSTS OF REGULATION

A professional regulator must aim to understand the cost of its regulation on the regulated
community. The cost of regulation is the sum of the resources given up by an organisation to
achieve regulatory compliance. This includes responding to perceived regulatory requirements
which may not be expressly intended by the regulator. It does not include costs that would be
expended regardless of the regulation.

There are multiple units of cost measurement. The most common are financial or time based but
others less tangible include loss of goodwill and impact on staff morale.

Cost of regulations is primarily borne by the regulated community and, to a lesser extent, the
regulator itself. There are three broad categories of costs applicable to the regulated community
described as follows and illustrated in Figure 22.

Administrative Compliance is the cost of demonstrating compliance which does not directly
contribute to achieving regulatory outcomes. Time spent interacting with the regulator to prove
compliance (such as hosting an audit), cost associated with completing forms for the regulator and,
importantly, learning about the regulations are all examples.

Substantive Compliance cost is associated with achieving regulatory outcomes. Recruiting an

additional maintenance member solely to meet a supervision requirement is an example of a

90
Australian National Audit Office (2007) Administering Regulation: Better Practice Guide, Canberra, chap 4
Relationship management.

10 Ways to Better Aviation Regulation | 10. Establish Genuine Engagement 121

substantive cost because it satisfies a regulatory obligation and would not have occurred had the
regulation not existed. Costs associated with preparing documentation to satisfy regulatory
requirements may be either administrative or substantive depending on their intended use.

Inefficiency/Indirect cost is realised as an indirect consequence of achieving regulatory

compliance. In other words second order effects. These are the least understood costs as they are
not paid for directly. For example, suppose a regulation requires a particular person to sign off on
release of an aircraft and that person was unavailable for a period of time (and no delegation was
possible), the cost of waiting (through reduced capability) is an inefficiency cost.

Cost of Regulation

Regulator Regulated Community

Regulation Administrative Substantive Inefficiency/Indirect

Administration Compliance Cost Compliance Cost Cost

Figure 22. The constituent costs of regulation.

MINIMISING COST OF REGULATION

A regulatory regime is the most cost effective when:

(i) Administrative compliance costs are kept to the bare minimum, just enough to provide the
regulator with enough evidence for them to undertake their oversight role.

(ii) Substantive compliance costs are zero because the regulated entity meets all the
regulatory requirements as a part of normal business and would do so even if the
regulations did not exist.

(iii) Enough flexibility exists within the regulations to ensure inefficiency costs are zero.

Costing of existing regulatory regimes is near impossible other than broad estimates. The best a
regulator can do is to cost changes to regulation. This is core to the Regulatory Impact Statement
tool. Detailed costing of regulation remains challenging and is beyond the scope of this publication.
Cost/benefit analysis and cost effectiveness analysis tools exist to assist with such tasks.

10 Ways to Better Aviation Regulation | 10. Establish Genuine Engagement 122

 ESTABLISH GENUINE ENGAGEMENT IN CONTEXT
The Civil Aviation Authority was never captured by the aviation industry.
91
On the contrary, the regulator offered itself as a willing captive.

A BETTER WAY TO REGULATION

Attitudes among regulatory staff as to the best style and methods of interacting with the regulatory
community will vary widely. Some will believe in the importance of tough enforcement and see
relationship building as inherently dangerous and potentially corrupt. Others see the value of close
collaborative partnerships and worry that harsh and inflexible enforcement will damage trust forcing
members of the regulated community to withdraw from collaboration. The truth lies somewhere in
between. It is only through engagement with regulators in their role of administering and enforcing
regulation that the regulated community primarily experience regulation and feel much of the
associated compliance burden. Good engagement should be core business of a regulator.

Appreciating the natural tension between regulators and the regulated community the aim should be
to develop mutual respect between both parties. Defence aviation safety regulators should
engage genuinely, using formal and informal approaches whenever undertaking regulation,
oversight or enforcement.

INTERACTION STYLE
Chapter 9 discussed two dimensions of enforcement style: formal/informal and
facilitative/threatening. These same dimensions are applicable to any interaction not just
enforcement. So with the aim of establishing mutual respect, regulatory staff should be aware of the
different styles and aim to adapt accordingly as the situation requires. Some staff may find a
preference for a particular style too strong. In such cases use of appropriately balanced teams or
allocating staff to particular tasks is a solution.

CONSULTATION MECHANISMS
The purpose of consultation is to seek comment from affected or interested parties regarding
proposals to introduce new or change existing regulations. But is consultation what is actually
desired? Perhaps it is to inform, or involve, or collaborate?

Sometimes less consultation is appropriate, particularly where a rapid decision is required for the
benefit of the aviation community. Consultation can be slow after all. But where consultation is
desired it can sometimes be difficult for the regulated community to know the intentions of the

91
Australian Parliament, House of Representatives, Standing Committee on Transport, Communications and
Infrastructure (1995), Plane Safe Inquiry into Aviation Safety: the Commuter and General Aviation Sectors,
Australian Government Publishing Service, Canberra.

10 Ways to Better Aviation Regulation | 10. Establish Genuine Engagement 123

regulator. For example the disadvantage of the Notice of Proposed Rulemaking (NPRM) process is
that the regulated community may fail to respond because they believe the regulator has already
made up their mind. Regulators, in seeking to communicate what they are doing and why, can easily
default into a process of telling.

The solution is to engage early and send clear messages as to the type of consultation the regulator
expects. If the regulator is seeking real consultation regarding a regulatory change (as opposed to
only informing) then engage prior to drafting the regulation, again after a draft has been prepared,
and again once finalised.

Box 16: In-practice some example consultation mechanisms

Name Intent Format

Discussion Paper High level, prior to actually Standalone document, titled

drafting any regulatory Discussion Paper
changes

Notice of Proposed For new or amendments to Standalone document, titled

Rulemaking existing regulations Notice of Proposed
Rulemaking (NPRM)

Notice of Final Rulemaking Issued following NPRM and Standalone document, titled
feedback Notice of Final Rulemaking
(NFRM)

Industry Consultative To obtain focussed and Roundtable meetings once or

Committees coordinated industry input twice a year.

Conferences and Coordinated input and public Presentations and open

Symposiums discussion of dedicated discussions of various
topics durations and timings

10 Ways to Better Aviation Regulation | 10. Establish Genuine Engagement 124

 Box 17: In-practice cost of Defence aviation safety regulation

The cost of Defence aviation safety regulation is not well understood. It is believed that
most regulatory costs are indirect and difficult to attribute to specific regulations.

There is agreement that the cost of the regulation is not insignificant. The cost to the
Commonwealth for the provision of Defence aviation materiel is over $4 billion every year
and rising. Therefore efficiency gains, however small in proportion, can make a material
difference in dollar terms. Conversely, poor regulatory practices can add significant costs
to the Commonwealth with little benefit to safety performance.

We know from empirical studies the drivers of burdensome and costly regulations in
Defence are driven by underlying factors affecting both the regulator and regulated.

Uncertainty in the regulatory requirement leads to conservative interpretation and

application by the regulated community and conservative enforcement by the regulator.
Uncertainty can be introduced simply in the process of drafting regulations. That is, even
assuming that all the affected parties understand and agree with the spirit of a given
regulation, it is often difficult to find the exact words to capture that spirit without leaving
room for interpretation (and thereby creating uncertainty). If uncertainty is created, who
owns it? It is lazy and most costly for the regulator to push uncertainty onto the regulated
community.

Inflexibility results from overly prescriptive regulation which impedes the regulated
community from doing business in the manner appropriate for their circumstances and
context.

Disproportionateness means focussing too much on particular hazards or risks which

leads to the regulated community having to divert their resources in a manner not
consistent with their circumstances or context.

Lack of Prior Recognition involves repeating certification or compliance activities

unnecessarily which drives cost for no increase in safety.

Double Treatment occurs when the same two risks or hazards are treated independently
and in isolation, leading to increased administrative cost of compliance for no increase in
safety.

Untimeliness by the regulator can place the regulated community in a position of

uncertainty and extend periods of risk associated with waiting for regulatory decisions.
Risk always increases cost.

10 Ways to Better Aviation Regulation | 10. Establish Genuine Engagement 125

10 Ways to Better Aviation Regulation | 10. Establish Genuine Engagement 126