The International Standards of Supreme Audit Institutions, or ISSAIs, are issued by INTOSAI,

ISSAI 3200
ISSAI XX – Title
the International Organisation of Supreme Audit Institutions. For more information visit
www.issai.org

INT OSAI
Guidelines for the
performance auditing
process

EXPOSURE DRAFT

Please send your comments before

September 30, 2015 to:

Ms. Maridel Piloto de Noronha,

PAS Secretariat
[email protected]

Inge Laustsen
[email protected]
 INTO S AI Pr o f e s si o n a l S t an d ar ds Co m mitt e e
PSC-Secretariat
Rigsrevisionen • Store Kongensgade 45 • P.O. Box 9009 • 1022 Copenhagen K • Denmark
Tel.:+45 3392 8400 • Fax:+45 3311 0415 •E-mail: [email protected]

IN TO S AI

EXPERIENTIA MUTUA
EXP ERIENTIA M UTUA

OMNIBUS PRODEST
OMNIBUS
P RODEST

INTOSAI General Secretariat – RECHNUNGSHOF

(Austrian Court of Audit)
DAMPFSCHIFFSTRASSE 2
A-1033 VIENNA
AUSTRIA
Tel.: ++43 (1) 711 71 • Fax: ++43 (1) 718 09 69

E-MAIL: [email protected];
WORLD WIDE WEB: http://www.intosai.org
 ISSAI 3200 – Performance Auditing Process

TABLE OF CONTENTS
INTRODUCTION .................................................................................................................................................. 2
PLANNING ........................................................................................................................................................... 3
Selection of topics ........................................................................................................................................... 3
Designing the audit .......................................................................................................................................... 5
CONDUCTING ................................................................................................................................................... 18
Evidence ........................................................................................................................................................ 18
Findings and conclusions .............................................................................................................................. 21
REPORTING ...................................................................................................................................................... 25
Content of the report ...................................................................................................................................... 25
Recommendations ......................................................................................................................................... 28
Communicating with the auditee ................................................................................................................... 30
Distribution of the report ................................................................................................................................ 31
FOLLOW-UP ...................................................................................................................................................... 32
 ISSAI 3200 – Performance Auditing Process

INTRODUCTION
1. Professional standards and guidelines are essential for the credibility, quality and
professionalism of public-sector auditing. The Fundamental Principles of Public-Sector Auditing,
amongst other things, defines the purpose and authority of ISSAIs and the framework for public-
sector auditing. The Fundamental Principles of Performance Auditing build on and further develops
the fundamental principles of ISSAI 100 to suit the specific context of performance auditing.

2. ISSAI 3000 is the International Standard for Performance Auditing and should be read
and understood in conjunction with ISSAI 100 and ISSAI 300. It provides the requirements for the
professional practice of performance auditing followed by explanations in order to enhance the clarity
and readability of the standard. ISSAI 3000 is the authoritative standard for performance auditing
and consequently each requirement must be complied with if an SAI choose to adopt it.

3. For each requirement set out in ISSAI 3000, supporting guidelines are provided in
ISSAI 3100, on central concepts for performance auditing and ISSAI 3200, on the performance
auditing process. These guidelines describe good practices that are based on the experience of SAIs
with a long tradition and well-established performance audit function. They are meant to help the
auditor interpret the requirements set out in ISSAI 3000, provide advice to the auditor on how to fulfil
these requirements and how to apply his/her professional judgment. Therefore, compliance with
these guidelines is not mandatory.

4. ISSAI 3200 deals with planning, conducting, and communicating and following-up on
the results of the performance audit. This guideline has four different parts, structured according to
the different phases in the performance audit process:

Planning Conducting Reporting Follow up

5. The first section relates to planning the audit – how to select audit topics and design
the audit. The second section relates to conducting the audit in order to obtain sufficient, appropriate
evidence to support the auditors’ findings and conclusions. The third section relates to reporting –
the format of the report, the report contents and report distribution. The fourth section relates to
follow-up of performance audit reports, to identify and document the impact of the audit and the
progress made in implementing recommendations.

6. ISSAI 3100 and ISSAI 3200 should be read together to get a deeper understanding of
how the central concepts are considered throughout the audit process.

2
 ISSAI 3200 – Performance Auditing Process

PLANNING

Planning Conducting Reporting Follow-up

7. This section contains planning requirements and guidelines for performance audits.
The purpose of these requirements is to establish the overall approach for the auditor to apply when
planning the performance audit. This section has two main parts. The first part is on selecting topics
and relates primarily to the SAI’s strategic planning process. The second part of planning relates to
the individual auditor’s design of each audit, focusing on what to audit, what criteria to use and what
methods to use.

Selection of topics

Requirements

8. The auditor shall select audit topics through the SAI’s strategic planning process
by analysing potential topics and conducting research to identify risks and problems.

9. The auditor shall select topics that are significant and auditable, and reflect the
SAI’s mandate.

10. The auditor shall conduct the process of selecting audit topics to maximise the
expected impact of the audit while taking account of audit capacities.

Good practice

Selecting an audit topic as part of the strategic planning process

11. Determining which audits will be carried out is part of the SAI’s strategic planning
process. The SAI’s strategy documents the main direction of the SAIs performance auditing. It covers
several years and involves the selection of topics, programmes or themes to guide the audits. While
the number of potential topics, programmes and themes is usually high the SAI’s capacity is usually
limited. Consequently, audit selection decisions must be made with care.

12. The SAI’s approach to selecting performance audit topics is a balancing act between
the mandate of the SAI, the expectations of different stakeholders such as the relevant legislative

3
 ISSAI 3200 – Performance Auditing Process

committee (for example the Public Accounts or Budgetary Committee), and the strategy of the SAI.
The strategic planning process would normally result in an audit programme for the SAI covering
one or several years. Considering the dynamic of the context in the public sector and the changing
priorities in public policies it is recommended to revise the audit programme annually.

13. Some SAIs may choose topics based on strategic considerations regarding the type of
performance audit and reforms within the public sector. One possible strategic choice is to decide to
contribute to the modernization of the government administration and focus on auditing government
programmes with significant effectiveness problems. Other SAIs may choose topics based on
selection criteria, for example with regard to a specific type of performance audit. An alternative
choice might be to simply focus on auditing individual government agencies and their performance
towards meeting objectives and goals in relation to economy, efficiency and effectiveness.

14. The audit programme for the SAI will serve as a basis for operational planning and
resource allocation. The programme can list the audit areas and provide a brief account of the
possible problems, questions, and other arguments supporting each one of them. The approach to
selecting the audit topics that are to be included in the audit programme may vary. Some SAIs have
a bottom-up approach, where the auditors participate in the selection process. Other SAIs have a
top-down approach, where the management selects audit topics and the auditor does not take part
in the selection process. Some SAIs have a mix of both approaches.

Assessing potential audit topic in terms of materiality and risk

15. Performance audit topics are generally selected on the basis of problem and/or risk
assessments and an assessment of materiality. Materiality relates not only to financial, but also
social and/or political aspects.

16. In performance auditing risks also involve areas where there is reason to suspect
inefficiency that concern citizens or can have great impact on specific groups of citizens. The
accumulation of such indicators or factors linked to an entity or a government programme may
represent an important signal to the auditor and can induce the auditor to plan audits based on the
risks or problems detected. Factors that may indicate higher risk include:
 The financial or budgetary amounts involved are substantial, or there have been significant
changes in the amounts involved.
 Areas traditionally prone to risk (for example procurement, technology, environmental issues
and health).
 New or urgent activities or changes in conditions (requirements, demands) are involved.
 Management structures are complex, with possible confusion about responsibilities.
 There is no reliable, independent, and updated information on the efficiency or the
effectiveness of a government programme.

17. The analysis of potential topics must give consideration to maximizing the expected
impact of an audit. As part of analysing potential topics and conducting research to identify risks and
problems, the auditor is advised to consider the following:
 The greater the risk for consequences in terms of economy, efficiency, and effectiveness or
public trust, the more important the problems tend to be.
 Adding value is about providing new knowledge and perspectives. The better the prospects
of carrying out a useful audit of good quality, and the less the policy field or subject has been
previously covered by audits or other reviews, the greater the added value might be.

18. Selection of topics depends on the assessed risks, that is the likelihood and impact of
an event with the potential to affect the achievement of an organisation's objectives. A risk analysis
is a two-step process:

4
 ISSAI 3200 – Performance Auditing Process

 Identifying possible events that, should they occur, would prevent the entity/functional
area from attaining its objectives; and
 Assessing the possible magnitude and likelihood of each event.

19. This assessment helps the auditor identify the inherent risks. The auditor may also
focus on residual risk – the risk that remains even when controls are in place to mitigate the inherent
risk – or on areas of suspected weaknesses.

Selecting audit topics that are auditable

20. Assessing auditability is an important requirement in selecting the audit. At this stage,
determining whether a topic is auditable or not depends on whether the audit topics are in compliance
with the audit mandate of the SAI and whether the SAI has the audit capacities (for example human
resources and technical skills) to conduct the audit. It is also important that the auditor takes into
account the expected net impact of the audit as defined in the SAI’s strategy. When designing the
audit, auditability will have to be considered yet again, more in detail (See section on Designing the
audit below).

How requests affect the selection process

21. Some SAIs get requests to conduct audits on a specific topic. In this case it is not
always possible to meet the specific requirements on topic selection. However, it is still important to
follow the requirements for designing the audit.

Designing the audit

Requirements

22. The auditor shall plan the audit in a manner that contributes to a high-quality
audit that will be carried out in an economical, efficient, effective and timely manner and in
accordance with the principles of good project management.

23. The auditor shall acquire substantive and methodological knowledge during the
planning phase.

24. During planning, the auditor shall design the audit procedures to be used for
gathering sufficient and appropriate audit evidence that respond to the audit objective(s).

25. The auditor shall submit the audit plan to the audit supervisor and SAI’s senior
management for approval.

Good practice

26. After the SAI has chosen an audit topic, the auditor has to design the specific audit.
According to ISSAI 3000, the auditor shall plan the audit in a manner which ensures that an audit of
high quality is carried out in an economic, efficient and effective way and in a timely manner. A well
thought-out plan is in general indispensable in performance auditing. Before starting the conducting
phase, it is consequently important to define the subject matter, ‘what is audited’. The audit topic is
often a rather broad area of interest so it has to be narrowed down by determining the audit
objective(s) and the audit scope. This is often done in the form of a pre-study.

27. The purpose of a pre-study is to establish whether the conditions for an audit exist and
if so, to produce an audit proposal with a work plan and a research design. A pre-study helps provide
answers to questions like: Is this subject auditable and worth auditing? What information is needed
and how should the audit be conducted? In addition, it provides background knowledge and

5
 ISSAI 3200 – Performance Auditing Process

information needed to understand the entity or entities to be audited (programme, function, service
or organisation).

28. A pre-study enables the auditor to produce an appropriate audit proposal and makes it
easier to ensure that the performance audit coverage is comprehensive and realistic. The pre-study
is normally carried out in a fairly short period. The pre-study of a specific topic can be a conducted
in less than a month, while a broader approach to the audit can normally be conducted within 3
months. The time spent on the pre-study is not recommended to exceed the time used on the main-
study. The outcome of the pre-study phase is the audit proposal.

29. As part of the pre-study it is advisable to test some of the hypotheses that led to the
topic selection. It is also important to check the availability of data. For example, if the audit objective
is efficiency in public hospitals, it is advisable to visit a hospital and collect central material in the pre-
study phase.

30. The auditor is advised to consider the needs and interests of the primary intended
users, including the responsible parties when designing the audit. The needs and interests of the
users could influence the selection of audit objectives and the types of analysis conducted by the
audit team. Ultimately, by taking into account the needs and interests of the primary intended users,
the auditor can ensure that the audit report is useful and understandable.

Understanding what is audited

31. The aim at the beginning of the design phase is to develop a basic understanding of
the subject matter (‘what is audited’), and of risks and challenges in the area. Obtaining the required
knowledge is a continuous and cumulative process of gathering and assessing information at all
stages of the audit. Therefore, it might be necessary to gather further information and to test initial
hypotheses in the planning stage once the audit topic is decided. This information will help the auditor
decide on the most relevant approach of the audit. It is important that the auditor weighs the costs
of obtaining information against the additional value of the information to the audit. The information
gathered in the planning phase may make it necessary to adjust what is to be audited.

32. Sources of information for understanding what is audited may include:

 enabling legislation and legislative speeches;

 ministerial statements, government submissions, and decisions;
 audited entity risk profile;
 recent audit reports, reviews, evaluations, and inquiries;
 scientific studies and research (including those from other countries);
 strategic and corporate plans, mission statements, and annual reports;
 policy files and management committee and board minutes;
 organisation charts, internal guidelines, and operating manuals;
 programme evaluation and internal audit plans and reports;
 viewpoints from experts in the field;
 discussions with the audited entities and key stakeholders;
 management information systems or other relevant information systems;
 official statistics;
 reports from other SAIs;
 press coverage.

6
 ISSAI 3200 – Performance Auditing Process

33. Past evaluations and audits are often a useful source of information. They can help
avoid unnecessary work in examining areas that have been under recent scrutiny and highlight
deficiencies that have not yet been remedied.

Defining the audit objective(s)

34. The auditor shall set a clearly defined audit objective(s) that relates to the principles of
economy, efficiency or effectiveness (see ISSAI 3000/37). The objective(s) determines the approach
and the design of the audit.

35. The audit objective(s) can be thought of as the overall audit question concerning the
subject matter (for instance a government programme or activity) that the auditor seeks the answer
to. Or put differently, it can be thought of as the hypothesis that will be tested through the collection
and analysis of evidence. The audit objective therefore needs to be framed in a way that allows a
clear and unambiguous conclusion. Many audit objectives can be expressed in the form of one
overall audit question which is then broken down into more detailed/specific sub-questions.

36. The audit objective(s) addressed by performance auditors do not have to be exclusively
based on a retrospective (ex post) audit approach. In a performance audit, the auditor can take an
early initiative and provide ex ante audits where appropriate, and if this is allowed by their legal
mandate.

37. It is important that the audit objective is based on rational and objective considerations.
In determining the audit objective(s), the auditor must determine where the greatest risks are, and
where the audit can add most value. To help defining appropriate audit objective(s) the auditor can
conduct interviews with major stakeholders and experts, and analyse potential problems from
various viewpoints. If an audit takes place on request, the audit objective(s) might however be more
or less determined or obvious.

38. The objective(s) is typically based on the requirements of the relevant legislation,
regulations, and policies, government objectives, or reflect what is expected based on sound
principles and best practices, or what could be given better conditions.

39. The audit objective(s) must give sufficient information to the audited entity and other
stakeholders about the focus of the audit. Well-defined audit objectives relate to a single entity or an
identifiable group of government undertakings, systems, operations, programmes, activities or
organisations.

40. A good practice is to describe the audit objective(s) as simply as possible. Presenting
the audit objective(s) as clearly and concisely as possible prevents the audit team from undertaking
unnecessary or overly ambitious audit work. Ambiguous or vague audit objectives must be avoided.
The auditor is advised to avoid multiple objectives where more than one question is asked. This will
enable the auditor to reach clear conclusions.

41. One way of examining whether the potential audit objective(s) can be concluded
against is by considering:
 whether the objective(s) identified is auditable (e.g. whether audit criteria are available
or can be developed, whether resources and competence are available, etc.);
 whether audit evidence exists or can be generated and is accessible by the auditor;
and
 whether audit methodologies can be used successfully to collect and analyse such
evidence.

7
 ISSAI 3200 – Performance Auditing Process

42. The audit objective(s) and scope are interrelated and need to be considered together.
Even minor changes in the objective(s) or the problem to be studied may have a major impact on
the general scope of the audit.

Defining the scope of the audit and the specific audit questions

43. The scope defines the boundary of the audit. To define the scope, the auditor needs to
identify which entities are to be included in the audit or which particular programme or aspect of a
programme defines the boundary of the audit. The auditor must also identify the period of time to
review and the locations that will be included. To avoid an overly complex audit, the audit scope can
exclude certain activities or entities from the audit, even if they in principle would be relevant to the
audit objective.

The scope of an audit can be determined by answering the following questions:

What? What specific questions or hypotheses are to be examined?
What kind of study seems to be appropriate?
Who? Who are the key players involved and the auditee(s)?
Where? Are there limitations in the number of locations to be covered?
When? Are there limitations on the time frame to be covered?

44. It is good practice to discuss the audit scope with the audited entities at the earliest
opportunity. In some cases, it may also prove useful to explicitly clarify what is not going to be audited
in the actual study (what is not intended to be covered). This may contribute to reduce
misconceptions or false expectations among stakeholders.

Creating audit questions

45. It is good practice for the auditor to create specific audit questions to help define and
structure the audit. The audit questions must therefore, when established, have a further impact on
and define the scope.

46. It is important for the audit questions to be thematically related, complementary, not
overlapping and collectively exhaustive in addressing the audit objective(s) (or main question). The
aim is to cover all aspects of the audit objective by specific audit questions. All terms employed in
the questions need to be clearly defined. The questions are stated in neutral form, even if the auditor
expects to find problems in relation to the question.

47. Audit questions may be analytical, normative or descriptive. Even if it is advisable to

formulate audit questions in a normative or analytical way, descriptive questions can sometimes be
useful in an audit, especially when preparing an audit in an area where information on economy,
efficiency or effectiveness is lacking. However, they seldom add much value to those who seek
comprehensive explanations, or well-founded information on how to significantly improve
performance. They therefore work best when combined with analytical or normative questions.

48. The formulation of audit questions is an iterative process in which the auditor
repeatedly specifies and refines the questions, taking account of known and new information on the
subject as well as the feasibility of obtaining answers. During the planning stage, the purpose of
formulating audit questions is to systematically direct attention to what the auditor needs to know to
accomplish the audit objective. Audit questions may have to be adjusted to better reflect the subject
matter as the auditor becomes more knowledgeable during the audit (see Conducting in this ISSAI).
However, it is advised that this be done infrequently. Since it is recommended that audit questions

8
 ISSAI 3200 – Performance Auditing Process

are communicated with the audited entity, changing the audit questions during the course of an audit
may raise questions as to the professionalism, objectivity and fairness of the audit.

49. The auditor must also define the approach of the study: what kind of study is needed
to answer the audit objective? There are three common approaches in performance auditing:
system-oriented, results-oriented and problem-oriented or a combination thereof (see ISSAI 3100).

50. Performance auditing is often based on an overall perspective, that is, a top-down
perspective. It concentrates mainly on the requirements, intentions, objectives and expectations of
the legislature and central government. However, it is also possible to add a client-oriented
perspective, a focus on service-management, waiting-time, and other issues relevant to the ultimate
clients or consumers involved.

Setting the audit criteria

51. The auditor needs to establish suitable criteria, which correspond to the audit
objective(s) or questions. Criteria are the benchmarks or a standard used to evaluate the subject
matter to determine whether a programme meets or exceeds expectations. The criteria provides a
basis for assessing the evidence, reaching audit findings and developing conclusions on the audit
objectives.

52. The criteria can be qualitative or quantitative and define what the audited entities will
be assessed against. The criteria may be general or specific, they may reflect a normative (ideal)
model for the subject matter under review, they may represent best or good practice, an expectation
of “what should be” according to laws, regulations or objectives. The criteria may also be “what is
expected”, according to sound principles, scientific knowledge and best practice, or “what could be”
(given better conditions). The nature of the audit and the audit questions determines which criteria
are the most suitable.

53. Audit criteria are deduced from authoritative sources. The auditor can use many
different sources to identify criteria, for example:
 Laws and regulations governing the operation of the audited entity
 Political goals or statements by the legislature
 Decisions made by the legislature or the executive
 Key performance indicators set by the auditee or the government
 Detailed procedures for a function or activity
 Standards from research, literature, professional and/or international organisations
 International benchmarks of good performance
 Corresponding performance in the private sector
 Benchmarks – same entity, different years; different entities same activity
 Planning documents contracts and budgets from the audited entity
 General management and subject-matter literature
 Criteria used previously in similar audits or by other SAIs
 Standards set by the auditor, possibly after consultation with subject matter experts
(necessary to agree with the relevant audited entities)
 Identification of what could be (given better conditions)

9
 ISSAI 3200 – Performance Auditing Process

54. Sometimes audit criteria are easy to define, for example when the goals set by the
legislature or the executive branches are clear, precise and relevant. However, this is often not the
case. The goals may be vaguely formulated, conflicting or non-existent. Under such conditions, the
auditor may have to establish criteria that reflect the ideal or expected result to which the
performance of the entity can be measured.

Tips for setting good audit criteria. Ensure that they are:
 short and clear i.e. unambiguous and easy to comprehend
 relevant and logically or causally linked to the audit questions
 mutually exclusive, i.e. different and distinct from one another, and not
overlapping
 collectively exhaustive for each audit question, i.e. taken together, they are
sufficient to answer the audit questions
 specific or testable (in principle capable of a “yes/no” response, even though
an elaborated answer is often required), so that it is possible to identify what
procedures and evidence is needed to provide an answer and to conclude
against the criteria.

55. The audit criteria need to be set objectively. The process requires rational
consideration and sound judgment. It is therefore important that the auditor acquires:

 an understanding of the area to be audited, recent studies and audits in the area;
 knowledge of the motives and the legal basis of the government programme or activity
to be audited and the goals and objectives set by the legislature or the government;
 knowledge of practices and experience in other relevant or similar programmes or
activities.

56. When setting criteria, one possibility is to allow experts in the field to answer questions
such as ‘what ought to be the ideal results under perfect conditions according to rational thinking or
best-known comparable practice?’ Alternatively, to define and obtain support for well-founded and
realistic criteria, it may prove helpful to discuss benchmarks with stakeholders and decision makers.

57. According to ISSAI 3000/51, the auditor shall, as part of planning or conducting the
audit, discuss the audit criteria with the audited entities and possibly with the relevant stakeholders.
Disagreement about criteria can then be identified, discussed, and, perhaps, resolved at an early
stage. It is especially the case when criteria are developed specifically for the engagement or they
are not self-evident and are capable of dispute by audited entity management.

58. It is important that the auditor listens to good arguments from the audited entities but
at the same time be aware of their potential interest to hide their weaknesses. The facts and
arguments presented by the audited entity must be weighed against other relevant facts and
arguments (from other sources, experts etc.) and the auditor may accept the criticism by the audited
entity of a criteria after careful discretion. However, the final decision on criteria belongs to the auditor
and it is important for the auditor to remain independent during the process.

Example of a performance audit design

59. Listed below is an example on the structure in a performance audit design where the
concepts presented above are used.

10
 ISSAI 3200 – Performance Auditing Process

An example of the structure of a performance audit design

Audit topic
 The active labor market policy

Audit objective:
 Does the Ministry of Employment deliver an effective employment programme?

Audit Scope
 There are 18 employment centers across the country, the audit covers 1 in each region,
which means 5 employment centers
 Unemployed persons can be divided into several different groups. The audit covers the
group of unemployed insured persons.
 The audit covers the period from 2013 to 2015

Audit questions (and sub-questions):

 Have employment activities provided by external consultants (for example training
courses on how to prepare a curriculum vitae) been acquired at the lowest possible
cost, while taking quality into account?
 Have the employment centers increased the number of unemployed persons attaining
training courses (output) without increasing the resources such as staff and office space
(input) from 2013 to 2015?
 Have employment centers achieved the expected outcome of the program, as set by the
legislature (for example accessing a new employment)?

Choosing methods to gather audit evidence

60. An important part of planning the conducting phase is to determine the methods to be
used to gather data and analyse it. The audit scope, the audit objective, the audit questions and the
audit criteria are the factors guiding what evidence is needed and the methods most appropriate to
obtain that evidence.

61. To manage audit risk it is important that the auditor in the planning phase gathers
information about availability and quality of relevant data needed to answer the audit question(s). If
there is a problem with the availability of secondary data, or the quality of the data is poor, the auditor
could decide to collect primary data, by developing questionnaires, statistical records, observations
etc. There is also the possibility to report on the poor quality of or missing data. When changing the
design, it is important that this is done as early as possible in the planning phase. The auditor must
also consider the relevance of and value added by these changes.

62. Performance audits can draw upon a large variety of data-gathering techniques that
are commonly used in the social sciences, such as surveys, interviews, observations, collection of
administrative data and of written documents. Sampling methods and surveys might allow general
conclusions to be drawn and case studies provide an opportunity for in-depth analysis.

63. Different types of audit evidence can be obtained by using different methods of
collecting data, as illustrated in the table below.

Table 1. Link between types of audit evidence and different methods

Audit evidence Methods of data collection
Testimonial evidence Interviews
Surveys, questionnaires

11
 ISSAI 3200 – Performance Auditing Process

Focus groups
Reference groups
Documentary evidence Document review
File reviews
Using existing statistics
Using existing databases
Physical evidence Observation of people
Inspection of objects or processes

64. It is important for the auditor to establish an appropriate strategy for the audit. The
challenge is to establish an appropriate strategy combining study designs, methods and techniques
in the audit to suit the context, objective, questions, criteria and the availability of skills and resources
as well as the availability of data.

65. The auditor will normally use a range of techniques to gather and analyse evidence.
The auditor has to decide on which methods are appropriate to use in the audit, i.e. what the
advantages and disadvantages are and whether methods are too costly to use compared with the
expected outcome. When planning the audit, the auditor will have to identify the probable nature,
sources, and availability of audit evidence required.

66. While primary data developed by the auditor is usually the most reliable, secondary
data collected and/or analysed by others (for example performance evaluation reports, internal
reports, etc.), can be an important source of information in performance audits.

67. During the planning stage, the purpose of choosing methods is to systematically direct
attention to what the auditor needs to know to answer the audit questions/criteria, and from where
and how the auditor can obtain the information. The aim is to adopt best practices, but practical
reasons such as availability of data may restrict the choice of methods, and the auditor may often
have to settle for the second-best solution. It is therefore important to have an open mind when
conducting the audit. The choice of methods must not be rigid at this stage.

Assessing auditability

68. Assessing auditability is also an important requirement in the designing process. It

defines whether a topic still is suitable for conducting an audit and whether an audit can be carried
out. A topic must be both auditable and worth auditing in order to be included in the audit scope. The
auditor may have to consider, for instance, whether there are criteria available and whether the
information or evidence required is likely to be available and can be obtained efficiently. Furthermore,
it is important that reliable and objective information exist and that there are reasonable possibilities
of obtaining this information.

69. During the planning phase, the auditor must consider if conducting an audit is still
relevant and cost effective. Even if the selected topic is consistent with the SAI’s strategy, the auditor
might observe during the pre-study that the expected problem is already handled by the auditee.
Similar studies covering similar objectives may already have been conducted by other institutions,
or there are no relevant criteria available. Another reason could be that the information or evidence
required is unlikely to be available and cannot be obtained efficiently. In such circumstances it is
important that the auditor informs the management of the SAI of these important aspects and that
management decides how to proceed.

An example of a design matrix

12
 ISSAI 3200 – Performance Auditing Process

70. In designing the audit the auditor must link together the audit objective, the audit scope
and questions, the audit criteria and the methods and strategy for data collection and analysis.

71. It must be stressed that there is no universally applicable model on how to plan and
design performance audits. The detail in which the audit is planned is another decision to be made.
Careful planning will reduce the likelihood of problems arising at a later stage. At the same time,
planning that is too detailed may sometimes inhibit innovative thinking and openness. Audits are
carried out in a complex environment, and it is therefore rarely possible to devise a comprehensive
audit design that predicts the progress of a performance audit in every detail.

72. The method presented below represents a good practice and is often applicable, but it
may not fit all audits and the more complicated engagements. The auditor must therefore in each
case reflect on whether it is suitable to apply the method presented below, or if there are better
alternatives.

73. The purpose of the design matrix is to clarify the feasibility of concluding against the
main audit objective, and to assure a logical chain from the audit criteria to the specific audit
questions all the way to the main audit objective. The matrix helps the auditor to impose a logical
disciplined pattern on the design of the study and to ensure that all aspects of a question are
considered. It also helps define the scope of the audit. However, when using the technique it may
arise that the main audit objective needs to be re-formulated or clarified. Most importantly the design
matrix requires the auditor, at the planning stage, to clarify what sources of evidence the audit criteria
can be tested against.

74. The figure shows how the audit objective can be broken down into specific audit
questions. For each specific audit question there must be established specific audit criteria. It is
advisable to formulate the audit objective so that the auditors can answer the audit questions and
conclude against the audit objective. Since the audit criteria are the benchmarks used to evaluate
the subject matter they are expressed in terms of what should be, what is expected or what could
be.

75. For example “Did the entity have effective procedures in place to manage its
programme?” Here the auditor must identify, by setting the audit criteria, what precisely are “effective
procedures”.

76. It is important that the auditor clearly outlines in the design matrix what kind of analysis
is needed to be able to obtain sufficient, appropriate audit evidence in order to establish findings.
This requires that the auditor describes what kind of information and data are to be collected, from
the specific sources, the techniques needed to gather data, and the kind of methods to be used to
analyse data.

77. As shown in the figure, the expected audit findings are directly related to and supported
by the audit criteria. The auditor will have to assess whether the programme or entity fulfills the
criteria. If the audit shows that some of the criteria are met while others are not, the auditor must use
his or her professional judgment to consider what the audit conclusion would be.

78. The auditor is advised to consider the expected findings, conclusions and impacts of
the examination to make sure that the proposed outcome of the audit still is expected to be useful
and feasible. The pre-study might provide some indications in this regard and become part of the
basis for the decision on whether to continue with the audit.

13
ISSAI 3200 – Performance Auditing Process

14
 ISSAI 3200 – Performance Auditing Process

Figure 1: The audit design matrix (an example)

Method
Audit (Overall study
objective Audit Audit design, data
Expected Expected Expected
questions criteria needed, sources audit conclusions conclusions
of data, findings (according to (according to
procedures for questions) objective)
data collection (according to
and analysis) criteria)

- - - - -
-
- - - -
- - - - -

Audit proposal Overall professional judgment is required

15
 ISSAI 3200 – Performance Auditing Process

Good project management and submitting the plan to supervisors and management

79. As performance auditing is time-consuming and costly, it is essential that the audit be
properly planned and that the implementation of the plan be regularly monitored and corrective action
taken when appropriate. The auditor shall submit the audit plan to the audit supervisor and SAI’s
senior management for approval. Early discussions with the supervisor and senior programme
management to gain an overall programme perspective are also important.

80. A sufficient audit team and team leader with the right skills need to be assigned. When
making an audit plan, it is important to determine the timetable and the resources needed. The
auditor also has to consider if there is need to consult with internal or external experts (consultants,
other auditors) in order to secure the quality of the audit.

81. A milestone plan may help the team break down the audit process into smaller parts.
This makes it easier for the team to assess how realistic the use of resources is compared to the
work needed. If the plan shows that the timelines are too tight, the team may need to expand the
timelines or consider which audit questions are themost important ones, and spend time and
resources accordingly. Sometimes less important questions can be answered by using less time-
consuming data collection methods, such as using available secondary sources as opposed to using
questionnaires or interview data.

82. The team leader is responsible for the day-to-day management of the audit and for
ensuring that the budget and timelines are adequately documented and that the performance audit
is completed within budget and on time. Where more complex performance audits are concerned,
the SAI may consider appointing an experienced supervisor or a steering committee to guide the
audit team and to monitor the progress of the audit.

83. Audit supervisors provide guidance and direction to staff assigned to the audit in order
to address the audit objectives and follow applicable requirements. At the same time they stay
informed about significant problems encountered, review the work performed, and provide effective
on-the-job training. The nature and extent of the supervision of staff and the review of audit work
may vary depending on a number of factors, such as the size of the audit organisation, the
significance of the work, and the experience of the staff.

Managing audit risks

84. A good practice is to include a discussion of the specific audit risks and how the auditor
plans to mitigate them in the audit plan or pre-study. Risk assessment can take many forms but may
be done by addressing the following questions:
 Is there enough data available and is this data of good quality?
 Does the audit team possess sufficient skills and knowledge for this particular audit?
 Are the time frames and resources (i.e. hours/funds) needed to conduct the audit feasible?
 Is the audit topic sensitive, highly visible or controversial? (e.g. political sensitivity, media
sensitivity, parliamentary sensitivity)
 Is the audit and/or the subject matter very complex?
 Is there a risk related to management integrity or entity relations?

85. If the audit risk is significant it may be necessary to modify the audit plan and develop
additional mitigation strategies. The auditor can then develop and modify the evidence collection
strategy to lower the audit risk. For example, it may be useful to consider:
 Establishing a different staff mix – for example more senior staff

16
 ISSAI 3200 – Performance Auditing Process

 Using additional internal or external specialists

 Adjusting the data collection methods
 Setting up specific communication arrangements with the audited entity
 Establishing specific quality control measures.

Communication in the planning phase

86. It is advisable to plan contacts with the audited entity and the relevant stakeholders
throughout the audit process in order to keep them continuously informed of the audit progress.

87. Practices regarding the communication may vary. Some SAIs prefer to give the audited
entity – especially senior management – detailed information on the pre-study, since their early
involvement can contribute to reassuring the audited entities and the government institutions
concerned. Other SAIs do not provide detailed information at this stage and prefer to provide such
information after the audit proposal has been approved. Even without providing detailed information,
it is generally good practice to provide the audited entity with information on the assumptions and
reasons behind the decision to carry out a pre-study. Preliminary discussions with the audited entity
are vital to inform them about the pre-study, what a possible audit could be about and why it may be
undertaken.

88. Discussions with managers and staff at the audited entity are important to gain basic
knowledge of the audit area and its functions and conditions. It is also important to seek knowledge
from other stakeholders, e.g. clients, researchers, evaluators, scientists, and other experts; but it is
desirable to inform the audited entity involved about this. In addition (but to some extent depending
on the subject matter), it is important to have discussions with the internal auditors and take
advantage of their experiences.

The following topics may serve as examples for discussion during the pre-study:
 whether the audit is requested by others, e.g. the legislature, or is at the initiative
of the SAI itself;
 whether the audit is addressing a general risk, involves a strategic assessment or
whether it relates to economy, efficiency or effectiveness issues, and if so on what
grounds;
 the purpose and the objectives of the pre study;
 the audit design;
 the audit criteria;
 the kind of information the SAI may need to get from the audited entity at this
stage in order to build up knowledge, test potential designs, etc.

89. In addition to meetings and discussions with the audited entities, several methods can
be used to support the communication process in the planning phase:
 Send a letter directly to the head of the audited entity or entities. This will ensure the proper
presentation of the audit to the senior management at the audited entity. A template can be
made to ensure that the information is presented in the same way for all audits.
 Make a leaflet presenting an outline of the audit process. This will facilitate the understanding
of what performance audit is all about and what the audit process will include. It can be
placed on the SAI’s website for a general introduction to performance audit.

17
 ISSAI 3200 – Performance Auditing Process

 Establish contact persons at the audited entities to enable the auditor to have direct contact
with the auditees and make the audit process run smoothly. Nevertheless, it is important to
keep senior management on both sides informed on important matters.

90. It is the responsibility of the auditor to try to foster a proper dialogue and
communication. However, if a disagreement occurs, it is important to handle it in a professional and
fair manner - listen carefully, focus on facts, be objective and keep one’s integrity.

CONDUCTING

Planning Conducting Reporting Follow-up

91. This section contains conducting requirements and guidelines for performance audits.
The purpose of these requirements is to establish the overall approach for the auditor to apply, when
conducting the performance audit. The conducting requirements relate firstly to obtaining sufficient
and appropriate evidence and secondly to using this evidence to answer the audit objective and
audit questions.

Evidence

Requirement

92. The auditor shall obtain sufficient and appropriate audit evidence in order to
establish findings, reach conclusions in response to the audit objective(s) and questions and
issue recommendations when appropriate (

Good practice

93. The auditor has to start by gathering the evidence needed to answer the audit
objective(s) and questions. The decisions on how to proceed with the gathering process will
generally be made when the audit is designed (see Planning above). Depending on how detailed
the general audit plan is, it can be necessary at the beginning of conducting the audit to elaborate
further on where and how to collect the evidence needed. It can be helpful to prepare detailed audit
plans, if this has not been done in the planning phase.
94. When the audit evidence is obtained, the auditor has to assess whether the evidence
is sufficient and appropriate. Based on this assessment the auditor has to decide if more or different
evidence is needed.

How to obtain sufficient, appropriate evidence

18
 ISSAI 3200 – Performance Auditing Process

95. In the conducting phase, the auditor has to continue identifying potential sources of
information that could be used as evidence. Not all situations can be foreseen during the planning
phase, and therefore the auditor, as he or she becomes more knowledgeable during the audit, may
have to adjust the methods and the need for data. However, the aim of the planning phase is to
properly and thoroughly plan in order to be able to apply the methods and collect the data described
in the planning documents during the conducting phase. It is therefore advised that the auditor avoids
material changes in the audit design at this stage. If this cannot be avoided, the management (and
the audited entity) needs to be informed. The auditor also has to evaluate whether the lack of
sufficient and appropriate evidence is due to internal control deficiencies or other programme
weaknesses, and whether the lack of sufficient, appropriate evidence could be the basis for audit
findings.

96. In performance audits evidence is rarely conclusive (yes/no or right/wrong). More

typically, performance audit evidence is persuasive (‘points towards the conclusion that…’). When
working in areas where evidence is persuasive rather than conclusive, it can be useful to have
discussions in the planning phase or in the beginning of the conducting phase with the experts in the
field about the nature of the evidence to be obtained and the way in which it will be analysed and
interpreted by the auditor. This approach reduces the risk of misunderstanding the evidence and
may speed up the process. It is also important that the auditor seek information from different
sources, since organisations, individuals in an organisation, experts, and interested parties have
different perspectives and arguments to put forward.

Assessing whether the evidence is sufficient and appropriate

97. The concept of sufficient, appropriate evidence is integral to an audit. In assessing

evidence, the auditor has to evaluate whether the evidence taken as a whole is sufficient and
appropriate for addressing the audit objectives and supporting findings and conclusions. Audit
objectives may vary widely, as may the level of work necessary to assess the sufficiency and
appropriateness of evidence to address the objectives. For example, in establishing the
appropriateness of evidence, the auditor may test its reliability by obtaining corroborating evidence.
The concepts of risk and significance assist the auditor in assessing the audit evidence (see also
Audit risk and Materiality in ISSAI 3100).

Appropriateness
Is a measure of the quality of the evidence that encompasses the relevance, validity, and
reliability of evidence used for addressing the audit objectives and supporting findings
and conclusions.
 Relevance
o refers to the extent to which the evidence has a logical relationship
with, and importance to, the issue being addressed
 Validity
o refers to the extent to which the evidence is a meaningful or
reasonable basis for measuring what is being evaluated. In other
words, validity refers to the extent to which the evidence represents
what it is purported to represent
 Reliability
o refers to the extent to which the audit evidence is supported by
corroborating data from a range of sources, or produces the same
audit findings when tested repeatedly.
 Sufficiency
o is a measure of the quantity of evidence used for addressing the audit
objectives and supporting the audit findings and conclusions.

19
 ISSAI 3200 – Performance Auditing Process

98. As mentioned above there are different types and sources of evidence that the auditor
may use. Each type of evidence has its own strengths and weaknesses. The following contrasts are
useful in assessing the appropriateness of evidence:

 Evidence obtained when internal control is effective is more reliable than evidence
obtained when internal control is weak or non-existent.
 Evidence obtained through the auditor’s direct observation, computation, and inspection
is more reliable than evidence obtained indirectly.
 Examination of original documents is more reliable than examination of copies.
 Testimonial evidence obtained under conditions in which persons may speak freely is
more reliable than evidence obtained under circumstances in which persons may be
intimidated.
 Evidence obtained from a knowledgeable, credible, and unbiased third party is more
reliable than evidence obtained from management of the audited entity or others who
have direct interest in the audited entity.
 Documentary evidence is considered to be more reliable than oral evidence.
 Testimonial evidence that is corroborated in writing is more reliable than oral evidence
alone.

99. The following presumptions are useful in assessing the sufficiency of evidence:
 The greater the audit risk, the greater the quantity and quality of evidence required.
 Stronger evidence may allow less evidence to be used.
 Having a large volume of audit evidence does not compensate for a lack of relevance,
validity, or reliability.
 Less evidence may allow for a less precise or less detailed finding.

100. The auditor must determine the overall sufficiency and appropriateness of evidence to
provide a reasonable basis for the findings and conclusions, within the context of the audit objectives.
Professional judgment assists the auditor in determining the sufficiency and appropriateness of
evidence as a whole (see also Professional judgment and scepticism in ISSAI 3100). Interpreting,
summarizing, or analysing evidence is typically used in the process of determining the sufficiency
and appropriateness of evidence and in reporting the results of the audit work. When appropriate,
auditors may use statistical methods to analyse and interpret evidence to assess its sufficiency.

101. Sufficiency and appropriateness of evidence are relative concepts, which may be
thought of in terms of a continuum rather than as absolutes. Sufficiency and appropriateness are
evaluated in the context of the related findings and conclusions. For example, even though the
auditor may have some uncertainties about the sufficiency or appropriateness of some of the
evidence, the auditor may nonetheless determine that in total there is sufficient and appropriate
evidence to support the findings and conclusions.

Sufficient and appropriate evidence

 Evidence is sufficient and appropriate when it provides a reasonable basis for
supporting the findings or conclusions within the context of the audit
objectives.
 Evidence is not sufficient or not appropriate when
o using the evidence carries an unacceptably high risk that it could lead the
auditor to reach an incorrect or improper conclusion,
o the evidence has significant limitations, given the audit objectives and
intended use of the evidence, or
o the evidence does not provide an adequate basis for addressing the audit
objectives or supporting the findings and conclusions. Auditors may not use
such evidence as support for findings and conclusions.

20
 ISSAI 3200 – Performance Auditing Process

Findings and conclusions

Requirement

102. The auditor shall analyze the collected information and ensure that the audit
findings are put in perspective and respond to the audit objective(s) and questions;
reformulating the audit objective(s) and questions as needed (ISSAI 3000/120).

Good practice

103. Performance auditing involves a series of analytical processes that evolve gradually
through mutual interaction, allowing the questions and methods employed to develop in depth and
sophistication. The whole process is closely linked to that of drafting the audit report. Reporting can
be seen as an essential part of the analytical process that culminates in answers to the audit
questions (see Reporting below).

104. The analytical steps to reach audit conclusions can be illustrated in this way:

• 1. Audit criteria - 'what should be'

• 2. Audit evidence - 'what is'

• 3. Audit findings - 'what is' compared with 'what should be'

• 4. Determine the causes and effects of the finding

• 5. Develop audit conclusions

105. Step 1 is setting the audit criteria, which is done in the planning phase. Step 2 is the
gathering of evidence and assessing whether these are sufficient and appropriate. (See section on
Evidence above.)

106. Step 3 is where the auditor uses the evidence to answer the audit questions. When
criteria are compared with what actually exists, audit findings are generated.

107. Once the auditor has identified a deviation between ‘what should be’ and ‘what is’, he
or she is advised to, where possible, determine why the deviation (cause) occurred and what the
consequences (effects) of this are. This is done in step 4.

108. In step 5, the auditor will reach a conclusion based on the findings. Formulating
conclusions may require a significant measure of the auditor´s professional judgment and
interpretation in order to answer the audit questions. It is necessary to consider the context and all
relevant arguments, and different perspectives before conclusions can be drawn. The involvement
of the SAI’s senior management is recommended (see Quality control and assurance in 3100).

Reaching audit findings

109. Audit findings are the specific pieces of evidence gathered and analysed by the auditor
to satisfy the audit objective(s), in order to answer the audit questions and verify the stated

21
 ISSAI 3200 – Performance Auditing Process

hypotheses (see the example of an audit design matrix under Planning above). Audit findings
normally contains the following elements: criteria (‘what should be’), condition (‘what is’), effect (‘what
are the consequences’), and causes (‘why is there a deviation from criteria’).

110. Meeting or exceeding the criteria may indicate a “good practice” leading to good
performance. Failing to meet criteria would indicate that improvements are needed. It is however
unrealistic to expect that the audited entity’s performance regarding economy, efficiency, and
effectiveness will always meet the criteria. It is important to appreciate that satisfactory performance
does not mean perfect performance, but is based on what a reasonable person would expect, taking
into account the audited entity circumstances.

111. Most audits involve some type of analysis in order to understand or explain what has
been observed. A wide range of models or methods of analysis are used depending on the objective
of the audit This could be done in the form of more detailed statistical analysis, discussing the
findings between the auditors, systematic analysis of interviews or other evidence, studies of
documentation and working papers. The analysis may sometimes also require comparisons of
findings between for instance:
 elements that work well and those that work less well, and
 the audited area and a similar audit area in another country.

112. When analysing collected information to reach findings and later conclusions, it is
recommended to focus on the audit question and objective(s). This will help to organise data and
also provide the focus for analysis.

Determining cause and effect

113. While it is important to seek explanations for deviations from criteria, causes must be
presented with caution. They have to be supported by evidence. Where possible, the auditor has to
assess causes that are stated by the auditee and make a judgment as to whether they are relevant
or not.

114. The auditor may have to identify possible effects of the criteria not being met. If
possible, in identifying the effects, the actual situation must be compared with the ideal situation
where the criteria would have been met. To a certain extent these possible effects would have been
considered at an earlier stage as a motivation for carrying out the audit of this particular problem.

115. The effects could be noted either as what has already occurred or as likely future
impact based on logical reasoning. The nature of the findings determines whether the auditors can
present actual or potential effects. Actual effects from past or current conditions help to demonstrate
the consequences and generally provide evidence that corrective action is needed. Potential effects
are generally described as the logical consequences that could follow should the condition not meet
criterion. Potential effects are to some degree speculative, so the auditor has to use them with care,
especially in the absence of any related evidence or observed past effects.

116. Cause and effect will have to be carefully scrutinized by doing a full critique of the data
and ensuring that other exogenous (external) factors have been allowed for in the data. It may be
necessary to use enhanced analytical techniques in order to answer questions on cause and effect.

22
 ISSAI 3200 – Performance Auditing Process

It is important to understand the nature of any relationships that may exist. It is not always the
case, for example, that poor funding causes worse conditions. It might be due to the poor quality
of care that funding was reduced for a particular organisation. When scrutinizing data the auditor
has to remember there are many reasons for relationships to exist:
 There may be a direct cause-and-effect relationship. For instance, if a university has
a set intake each year, and increases its intake of part-time students, then it must
reduce its full-time intake.
 There may be a reverse cause-and-effect relationship. For example, bad exam results
could be due to poor attendance but, equally, poor attendance could be due to bad
exam results.
 The relationship may be a coincidence. For example, there may be a relationship
between the quality of health care in a local authority and their exam results, but it is
hard to say that one causes the other.
 There may be a confounding effect. For example, the relationship between quality of
health care and exam results could be due to effective use of resources within the
local authority, which may not have been considered as part of the fieldwork.

Developing conclusions after considering the findings

117. Once the auditor has established the findings, determined why the criteria are not being
met (causes), and possible consequences (effects), the auditor has to draw conclusions.
Conclusions are statements inferred by the auditor from those findings. Since performance audits
point out deficiencies in economy, efficiency and/or effectiveness, the conclusions have to specify
the reasons behind the lack of economy, efficiency or effectiveness.

118. Audit conclusions clarify and add meaning to specific findings in the report. It is not
always easy to make a clear distinction between the findings and the conclusions. One reason for
this is that conclusions are based on findings and can include summaries of the findings. However,
conclusions present the auditor´s opinion and go beyond merely restating the findings. Whereas the
audit findings are identified by comparing ‘what should be’ with ‘what is’, the conclusions reflect the
auditor’s explanations and views based on these findings. Conclusions might include identifying a
general topic or a certain pattern in the findings. An underlying problem that explains the findings
may also be identified.

119. The conclusions must flow logically from the findings, their causes and their effects. All
analytical steps taken beyond the findings will have to be clearly explained and justified.

120. The analysis of data consists in combining results from different types of sources.
There is no general method for doing this. In a properly conducted performance audit, the arguments
put forward are balanced against the best possible counter arguments, and the various contrasting
views are weighed against each other. The conclusions have to be based on the objective(s) criteria,
evidence and findings.

121. The analysis undertaken must be rigorous and objective, using appropriate methods
and sound evaluative criteria. Evidence has to be triangulated and conclusions drawn from the
evidence on the basis of considered and balanced judgment. Triangulation of evidence means
forming findings and conclusions that are supported by different types of evidence from more than
one source. In pulling together the conclusions, the auditor has to regularly test them against the
evidence base. Such conclusions are likely to be more reliable that those based upon only one
source of evidence.

23
 ISSAI 3200 – Performance Auditing Process

122. In the process of developing conclusions it can be necessary for the auditor to adjust
or tweak the audit questions and on rare occasions even the audit objective (s). Slight adjustments
can be needed when it during the analysis becomes clear that it will not be possible to precisely
answer the posed questions with the obtained evidence and findings. The auditor may also realize
that some of the audit questions need to be adjusted as he or she gains more knowledge about the
subject matter. If adjustments are needed they have to be discussed and communicated both
internally and to the audited entity.

123. It is important for the auditor to be goal-oriented and to work systematically and with
due care and objectivity. It is vital that the auditor adopts a critical approach and maintains an
objective distance to the information put forward. At the same time, the auditor must be receptive to
views and arguments. The auditor must be able to see things from different perspectives and
maintain an open and objective attitude to various views and arguments. If the auditor is not
receptive, the auditor may miss the best arguments. This also underscores the importance of making
rational assessments, in that the auditor discounts his or her own personal preferences and those of
others. It is therefore important for the auditor’s involvement to be expressed in a process of reflection
and objective analysis rather than in a conviction that certain standpoints are correct (See
Professional judgment and scepticism in ISSAI 3100).

Managing audit risk in the conducting phase

124. It is important to monitor audit risk and the planned mitigation strategies throughout the
audit, and make adjustments as needed to changing circumstances.

125. Good planning will enable the auditor to manage audit risk when conducting the audit,
as the auditor will have planned for eventualities and different scenarios. For example, if the planned
data collection procedures do not allow the team to collect sufficient evidence, the auditor needs to
develop an alternative plan for adjusting these procedures. Also, the auditor always needs to
consider whether the audit risks have changed in a way that can lead to inappropriate conclusions,
unbalanced information or not adding value. Proper quality control procedures and supervision are
important in this regard.

Documentation during the conducting phase

126. The auditor shall document the audit in a sufficiently complete and detailed manner
according to ISSAI 3000/90. Preparing audit documentation on a timely basis helps to enhance the
quality of the audit and facilitates the effective review and evaluation of the audit evidence obtained
and conclusions reached before the report is finalized. Because it is difficult to reconstruct and recall
specific activities related to gathering audit evidence weeks after the work was actually performed,
work needs to be documented as the audit team completes it in order to reduce risk of inaccurate
audit documentation, improve audit quality, and improve engagement efficiency.

127. The nature and extent of audit documentation for a particular audit are largely a matter
of professional judgment, based on the unique circumstances of each audit. However, an auditor will
typically be expected to document the following:
 the objective (s), scope, and methodology of the audit; and
 the work performed and evidence obtained to support significant judgments and
conclusions.

Discussing the preliminary findings and conclusion – internally and externally

128. As the work continues, the draft report gradually takes shape. The notes and
observations are put into a structured order, and as internal and external discussions progress, text
is drafted, assessed and rewritten; details are checked and conclusions are discussed.

24
 ISSAI 3200 – Performance Auditing Process

Communication is essential in the analytical process because the auditor has to consider the context
and all relevant arguments, and different perspectives before conclusions can be drawn. For this
reason, the auditor needs to maintain an effective and proper communication with the audited entity
and relevant stakeholders.

129. Internal discussions with senior auditors and experienced colleagues can help the
auditor in the analytical process – in weighing findings and assessing preliminary findings and
conclusions.

130. Externally there is a need for exchange of information to discuss major issues that have
emerged during the course of the audit. In this phase, it is necessary to ensure that the factual basis
of descriptions is accurate and fair and that the analyses are comprehensive and address the cause
of identified problems. Various arguments need to be represented and findings put in perspective.
Meetings with the auditee may serve to confirm facts with the audited entities and to promote the
development of audit findings and recommendations.

131. A good practice is having meetings with senior managers or other government officials.
Another is to carry out focus group meetings, in which various stakeholders and experts are invited
to discuss preliminary findings, conclusions and recommendations. Being able to discuss various
issues when all vital stakeholders are present will add value to the audit (See Communication in
ISSAI 3100).

REPORTING

Planning Conducting Reporting Follow up

132. This section contains reporting requirements and guidelines for performance audits.
The purpose of reporting requirements is to establish the overall approach for the auditor to apply in
communicating the results of the performance audit. The reporting requirements for performance
audits relate to the form of the report, the report contents, and report issuance and distribution.

133. The purposes of audit reports are to (1) communicate the results of audits to the
intended user(s); (2) make the results less susceptible to misunderstanding; (3) make the results
available to the public in order to create transparency, unless specifically limited; and (4) facilitate
follow-up to determine whether appropriate corrective actions have been taken.

Content of the report

Requirements

134. The auditor shall provide audit reports, which are a) comprehensive, b)
convincing, c) timely, d) reader friendly, and e) balanced.

25
 ISSAI 3200 – Performance Auditing Process

135. The auditor shall identify the criteria and the source of the criteria in the report.

136. The auditor shall ensure that the findings clearly conclude against the audit
objective(s) or questions, or explain why this was not possible.

Good practice

137. Performance audit reports aim to contribute to better knowledge and highlight
improvements needed. In a performance audit, the auditor reports on the economy and efficiency
with which resources are acquired and used, and the effectiveness with which objectives are met.
Such reports may vary considerably in scope and nature, for example covering whether resources
have been applied in a sound manner, commenting on the impact of policies and programmes and
recommending changes designed to result in improvements.

138. When writing the audit report it is vital that both the audit team, supervisors and quality
control reviewers critically consider the conclusions in relation to the findings, evidence, data
material, and criteria. Findings must be substantiated and conclusions must be supported by solid
evidence. Recommendations, if provided, must be linked to the conclusions. Proper procedures for
clearance and fact validation with the audited entity will also be important.

Writing comprehensive reports

139. It is important that the report promotes an adequate and correct understanding of the
matters and conditions described. To write a comprehensive report, the auditor will typically include
a description of the audit objective(s) and the scope and methodology used for addressing the audit
objective(s) and audit questions. Readers need this information to understand the purpose of the
audit, the nature and extent of the audit work performed, the context and perspective regarding what
is reported.

140. Readers also need to know if there are any significant limitations in audit objective(s),
scope, methodology, or data gathered, so that they can reasonably interpret the findings,
conclusions, and recommendations in the report without being misled.

141. In the report, the auditor will typically identify significant assumptions made in
conducting the audit, describe the methods and the criteria used, including their sources. The auditor
has the ultimate responsibility to define and explain the criteria used in the audit report. (See Planning
in this ISSAI for more information on criteria.)

142. Auditors may provide background information to establish the context for the overall
message and to help the reader understand the findings and significance of the issues discussed.
Appropriate background information may include information on how programmes and operations
work, the significance of programmes and operations, a description of the audited entity’s
responsibilities, and explanation of terms.

Writing convincing reports

143. In a convincing and accurate report the audit findings and conclusions address the
audit questions or objective(s) and are presented persuasively. The report has a logical flow with
findings, conclusions and recommendations clearly linked to the subject matter and audit criteria.
Furthermore, the conclusions and recommendations follows logically and analytically from the facts
and arguments presented. An accurate report is fact-based, with clear statement of sources,
methods and assumptions so that report users can judge how much weight to give the evidence and
conclusions reported. The language and tone used is neutral, and the information presented is

26
 ISSAI 3200 – Performance Auditing Process

sufficient to convince the readers as to the validity of the findings, the reasonableness of the
conclusions, and the benefit of implementing the recommendations. Different perspectives, opinions
and arguments are presented.

144. One way to help audit organisations prepare accurate audit reports is to use an
engagement quality control reviewer, i.e. an experienced auditor who is independent of the audit
checks that statements of facts, figures, and dates are correctly reported, that the findings are
adequately supported by the evidence in the audit documentation, and that the conclusions and
recommendations flow logically from the evidence.

Writing timely reports

145. The report has to provide accessible, concise and up-to-date information, which the
government, the legislature, and government entities can use for improvements. To be of maximum
use, the auditor's goal is to provide relevant evidence in time to respond to legitimate needs of the
intended users. Likewise, the evidence provided in the report is more helpful if it relates to current
issues. Therefore, the timely issuance of the report is important.

146. During the audit, the auditor may provide interim reports of significant matters to the
auditee, if allowed by the SAI’s mandate. Such communication alerts officials to matters needing
immediate attention and allows them to take corrective action before the final report is completed.
(See ISSAI 3100 Communication.)

Writing balanced reports

147. The work underpinning performance audit reporting must be fair and support the overall
findings, conclusions and recommendations. In preparing a balanced and constructive report it is
useful to:
 Present findings objectively and fairly. Present and interpret facts in neutral terms,
avoiding biased information or language that can generate defensiveness and
opposition.

 Present different perspectives and viewpoints. Where different interpretations of the

evidence can legitimately be made, these need to be presented to ensure fairness and
balance. By following the underlying arguments, the reader will better be able to
understand the final conclusions and recommendations.

 Be complete. A complete report includes both good and bad points and gives credit
where it is due. Inclusion of positive aspects may lead to improved performance by other
government organisations that read the report. It is important that the report contains all
information and arguments needed to satisfy the audit objective(s), and promote an
adequate and correct understanding of matters and conditions reported. Facts must not
be suppressed, and minor shortcomings not exaggerated. Explanations, especially from
the auditee, must always be sought and critically evaluated.

Writing reader-friendly reports

148. To effectively add value and promote the better use of resources, it is important that
performance audit reports are clear, concise, logical, and focused on the topic area. Reports are
likely to have the greatest impact on a wide audience when they are reader-friendly.

Meeting the audience’s needs

27
 ISSAI 3200 – Performance Auditing Process

149. A key success factor for reader-friendly reporting is to determine the audience and
understand its needs. The primary audience for performance audit reporting is the legislature and
government agencies. However, there are also other stakeholders such as citizens, academia, the
private sector and the media who all can have an interest, but possibly a different focus, in the
outcome of a performance audit.

An effective report structure

150. At the outset of writing an audit report it is important that a draft structure is determined
to facilitate the organisation and flow of the text. An effective structure enables the report to grab the
reader’s attention, convey the purposes of the audit, communicate complex issues, and provide clear
interpretation of the results. Using a "Dinner Party" approach can help the auditor in making a reader-
friendly report structure. The Dinner Party refers to a real dinner party situation where there is only
a short amount of time to hold fellow guests’ attention. The Dinner Party meeting takes place after
data collection and analysis and the aim is to produce crisp, interesting report conclusions that can
each be stated in 10-15 seconds, and to build up more levels of detail from that basis.

Tips to design the report for easy reading

 Ensure the content of the report flows from the audit objective(s) and the reader is
provided with sufficient information to understand the topic.
 Break up the text with the use of headings.
 Be clear on the main point of each section and paragraph and how it relates to the
broader audit topic;
 Design your report for easy reading, making it appear ordered and uncluttered.
 Avoid the excessive use of cross-referencing and acronyms.

Clear writing

151. A reader-friendly report must be clear. In order to improve clarity:

 avoid jargon. When technical, scholarly or foreign terms and abbreviations are required
they must be explained. It is helpful to the reader if explanations are provided in a
glossary or easy-to-find footnotes;
 avoid ambiguity;
 use the same term consistently for a specific thought or object;
 use active rather than passive voice;
 be concise. Use short sections, paragraphs and sentences;
 use examples that demonstrate audit findings and conclusions;
 use visuals to direct attention to main points. Use lists, tables, diagrams, maps and other
illustrations to present complex and large amounts of data. These can often convey a
message more effectively than text. However, keep tables and graphs simple. Make
sure they illustrate one idea only and that the reader will be able to understand that idea
immediately.

Recommendations

Requirement

28
 ISSAI 3200 – Performance Auditing Process

152. The auditor shall provide constructive recommendations that are likely to
contribute significantly to addressing the weaknesses or problems identified by the audit,
whenever relevant and allowed by the SAI’s mandate.

Good practice

153. Recommendations, where provided, aim to promote improvements to service delivery

and improved governance, as well as help to identify financial savings and efficiencies. The auditor
may recommend actions to correct deficiencies and other findings identified during the audit and to
improve programmes and operations when the potential for improvement is substantiated by the
reported findings and conclusions.

154. In order to be constructive, recommendations will typically:

 be well-founded, practical and add value;
 flow logically from the findings and conclusions;
 be directed at resolving the cause of identified weaknesses or problems;
 be phrased in such a way that avoids truisms or simply invert the audit conclusions;
 neither be too general nor too detailed. Recommendations that are general will
typically risk not adding value, while recommendations that are too detailed would
restrict the freedom of the audited entity;
 clearly state the actions recommended and who is responsible for taking the actions;
 be addressed to the entities having responsibility and competence for implementing
them.

155. This means that recommendations, depending on the audit objective(s) and the
findings, may be addressed to ministries, agencies, local governments or state owned companies.
For recommendations to be practical it is important that they are realistic. In other words, that it will
be possible to implement them with existing resources.

156. It may be relevant to present the arguments for and against various alternative
proposals. By following the underlying arguments, the reader will be better able to understand the
final recommendations.

Questions to consider when developing recommendations

 What needs to be done and why?
 Where does it need to be done?
 Who is responsible for doing it?
 Will the proposed actions remedy the problems observed?
 Could the proposed actions have any negative effects?

29
 ISSAI 3200 – Performance Auditing Process

157. Effective recommendations encourage improvements in the conduct of government

programmes and operations. Recommendations are effective when they are addressed to parties
that have the authority to act and when the recommended actions are specific, practical, cost
effective, and measurable.

Tips to generate recommendations

 Think about potential recommendations early on in the audit process. Teams are
often expected to present the scope of potential recommendations at an early
stage.
 Write the recommendations in a way that allows auditors to evaluate whether or
not they have been implemented.
 Build on good examples from past reports – do not reinvent the wheel.
 Where possible, work with the auditee to identify the necessary changes and
ways of implementing them.

Communicating with the auditee

Requirements

158. The auditor shall give the audited entity the opportunity to comment on the audit
findings, conclusions and recommendations before the SAI issues its audit report.

159. The auditor shall record the examination of the audited entity’s comments in
working papers, including the reasons for making changes to the report or for rejecting
comments received.

Good practice

160. Giving the audit entity the opportunity to comment on the audit findings, conclusions
and recommendations before publishing the report, helps to ensure that the factual basis of
descriptions in the report is accurate and fair and that the analyses are comprehensive and address
the cause of identified problems. Various arguments need to be represented and findings put in
perspective. Recommendations must be well founded and add value. All of these issues need to be
communicated to the responsible authorities concerned by the audit.

161. Providing a draft report with findings for review and comment by responsible parties
helps the auditor develop a report that is fair, complete, and objective. Including the views of
responsible parties results in a report that presents not only the auditor’s findings, conclusions, and
recommendations, but also the perspectives of the audited entities and the corrective actions they
plan to take. Obtaining the comments in writing is advisable.

162. Usually the SAI determines the amount of time given to the audited entity for providing
feedback, but care must be taken to ensure that sufficient time is given for feedback.

Dealing with the comments received

163. All comments received need to be carefully considered. Where responses provide new
information, the auditor needs to assess this and be willing to modify the draft report. All
disagreements must be analysed in order for the final report to be balanced and fair.

30
 ISSAI 3200 – Performance Auditing Process

164. When the responsible parties’ comments are insufficient to address the findings,
inconsistent or in conflict with the findings, conclusions, or recommendations in the draft report, the
auditor is advised to evaluate the validity of the audited entity’s comments. If the auditor disagrees
with the comments, it is good practice to explain in the report their reasons for disagreement.
Conversely, the auditor is advised to modify the report as necessary if they find the comments valid
and supported with sufficient, appropriate evidence.

165. The responses need to be documented. It will be helpful to record in working papers
the examination of the feedback received so that any changes to the draft audit report, or reasons
for not making changes, are documented.

166. At the end of the process it is advisable to keep the auditee(s) informed on the
procedures and timetable for the publication of the final report.

Referring the draft report to third parties

167. In order to ensure that the audit report is fair and balanced, it is a good practice to refer
them to third parties concerned by the audit as well as the audited entity. Third parties concerned by
the report, that is all individuals and organisations referred to in the report, may be given the
opportunity to comment on what is said about them and their actions or views. While the third parties
may be provided the whole draft for comments, the auditor needs to decide how much of a stake
third parties have in the subject matter. In some circumstances, auditors may choose to send third
parties the whole report or major sections of it, but often it will be appropriate only to send them
extracts.

Distribution of the report

Requirement

168. The SAI shall make its audit reports widely accessible.

Good practice

Distributing reports to responsible parties, stakeholders and the public

169. It is recommended that SAIs decide about the method of distribution in conformity with
their respective mandates. Each performance audit will normally be published in a separate report,
either in print or online or both. The reports must be distributed to the legislature and the responsible
parties. It is common practice to make the report accessible to the general public directly and through
media and to other interested stakeholders, unless prohibited by legislation or regulations. It is an
advantage if the reports are available for public discussion and criticism.

170. The auditor is advised to use a form of the audit report that is appropriate for the
intended users and is in writing or in some other retrievable form. For example, the auditor may
present audit reports using electronic media that are retrievable by all intended users.

Results may be presented in different ways

171. In order to make the report widely accessible, the auditor may develop different
material to present in the report. The users’ needs will influence the form of the products and may
include summaries, press releases or other presentation materials. Each product needs to be written
in a style tailored to its specific audience in order to have the maximum impact. Preparing a

31
 ISSAI 3200 – Performance Auditing Process

communication plan can provide a structured way of thinking about how to effectively reach different
audiences and provide timely input to the decision making process.

172. In addition to the published report (on paper or online), the auditor may consider
generating additional products to disseminate the findings in the main report more widely:
 It is recommended that one provides the media with adequate and well-balanced
information, for instance in the form of press releases. This may reduce the risk of the
media misunderstanding or exaggerating findings.
 Individual feedback reports may be issued to survey respondents to show how they are
performing compared to the sector benchmarks, and to spread good practice.
 Auditors generating detailed data analysis may publish additional technical annexes on
the web.
 Other reports on the subject matter by consultants or academics may be placed in full
on the web to give greater backing to the summary provided in the main report.
However, it is appropriate that this only happens where it is felt these reports add
substantial value and do not in any way conflict with the findings and the conclusions of
the audit report.

173. Whatever means used to disseminate the message in the report, the auditor needs to
make sure that the messages are consistent throughout. The auditor also needs to consider whether
the products have been subject to sufficient quality control.

174. In addition to written material, auditors may use a range of means to increase the
influence of the audits by helping organisations to improve their performance, and by spreading good
practices and lessons learned across the public sector. To do this, the auditor can use a variety of
methods, such as workshops with the audited entity to help stimulate and embed beneficial change
and holding conferences as an effective way to reach practitioners and promote discussion on
important issues.

FOLLOW-UP

Planning Conducting Reporting Follow-up

175. The publication of the report is not the end of the auditing process. Beyond publication
there is follow-up on the impact of the audit. The aim of audit reports is to influence the way in which
services are designed and provided to citizens, and recommendations are given to help deliver
improvements in the economy, efficiency and effectiveness of these services. This section contains
follow-up requirements and provides advice on how to do follow-up of performance audit reports.

32
 ISSAI 3200 – Performance Auditing Process

Requirements

176. The auditor shall follow up, as appropriate, on previous audit findings and
recommendations and the SAI shall report to the legislature, if possible, on the conclusions
and impacts of all relevant corrective actions.

177. The auditor shall focus the follow-up on whether the audited entity has
adequately addressed the problems and remedied the underlying situation after a reasonable
period of time .

Good practice

Why follow-up

178. Follow-up of the audit report is an important tool to strengthen the impact of the audit
and improve future audit work and is therefore a valuable part of the audit process. A follow-up
process will facilitate the effective implementation of report recommendations and provide feedback
to the SAI, the legislature and the government on performance audit effectiveness. Following up on
audit findings and recommendations may serve four main purposes:
 identify the extent to which audited entities have implemented changes in response to
audit findings and recommendations;
 determine the impacts which can be attributed to the audits;
 identify areas that would be useful to follow up in future work
 evaluate the SAI’s performance. Follow-up provides a basis for assessing and
evaluating SAI performance and may contribute to better knowledge and improved
practices in the SAI.

The focus of the follow-up

179. Follow-up is typically done periodically as deemed appropriate by the SAI. The priority
of follow-up tasks are usually assessed as part of the overall audit strategy of the SAI. Sufficient time
needs to be allowed for the audited entity to implement appropriate action.

180. When conducting follow-up of audit reports, the aim is to determine whether actions
taken on findings and recommendations remedy the underlying conditions. This means that both
positive and negative reactions regarding the audit and the audit report need to be examined by the
auditors. It is therefore important to adopt an unbiased approach.

181. The impact of the audit may be identified through the effect of corrective action taken
by the responsible parties, or through the influence of the audit findings and conclusions over
governance, accountability, the understanding of the problem addressed or the approach towards it.

182. When conducting follow-up of an audit report, the auditor needs to concentrate on
findings and recommendations that are still relevant at the time of the follow-up. Insufficient or
unsatisfactory action by the audited entity may call for a further audit by the SAI.

183. The results of the follow-up need to be reported appropriately in order to provide
feedback to the legislature, if possible together with the conclusions and impacts of the corrective
actions taken. Follow-ups may be reported individually or as a consolidated report. If several follow-
ups are reported together, it may include an analysis of different audits, possibly highlighting

33
 ISSAI 3200 – Performance Auditing Process

common trends and themes across a number of reporting areas. Whatever the form, the follow-up
reports must be balanced and findings presented objectively and fairly.

How to do the follow-up

184. Different methods may be used to follow-up on the findings and recommendations
made.
 Arrange a meeting with the responsible parties after a certain time has elapsed to find
out what actions have been taken to improve performance and to check which
recommendations have been implemented.
 Request the responsible parties to inform the SAI in writing on what actions they have
taken to address the problems presented in the audit report.
 Stay updated on reactions from responsible parties, parliament and the media and
analyse whether identified problems have been appropriately addressed or not.
 Carry out a follow-up audit, resulting in a new performance audit report.

185. What methods to use will depend on the priorities established by the SAI during the
strategic and annual planning process for performance auditing. It is also influenced by the
importance of the identified problems, the actions expected to be implemented, and the external
interest for information on the actions taken.

186. Whatever method used, the results from the follow-up need to be recorded. A good
practice is to report deficiencies and improvements identified in the follow-up of audits to the
responsible parties or the legislature.

34