INTOSAI Task Force on Public Procurement Audit

Public Procurement Audit Practical Guide

Summary of principal sections
Part 1 provides the necessary general information about Public Procurement process. It
contains a description of Public Procurement process life cycle, Public Procurement principles
and ethics in Public Procurement
Part 2 describes the area of application of the Guide, object and subject of Public
Procurement Audit, Main goals of Public Procurement Audit, auditee and audit domains of
Public Procurement Audit
Part 3 includes a description of Fundamental principles of public sector auditing and its
Ethical principles, Stakeholders and intended users of the Public Procurement Audit, objectives
of the Public Procurement Audit and corruption risks in Public Procurement. Objectives of the
Public Procurement Audit are also considered according to the stages of the goal-oriented
procurement process. Section noted that Public Procurement Audit process can also be
considered as a goal-oriented process
Part 4 describes contents of Public Procurement Audit: the usage of different types of
audit during Public Procurement Audit, the choice of methodological support of Public
Procurement Audit including structure of methodological support, the choice of information
support of Public Procurement Audit including description of Internal and External Information
Sources and a format of presenting information on the Public Procurement Audit Best Practices.
It also contains Public Procurement Audit in the context of application of modern Information
Technologies for the benefit of improvement of the audit quality and specific features of Public
Procurement Audit for the benefit of IT implementation
Part 5 considers the various aspects of organization and implementation of Public
Procurement Audit: planning of an audit and preparation of information requests, audit risk in
Public Procurement Audit, methods and main procedures of Public Procurement Audit,
collection and analysis of audit evidence, audit documentation including types of audit
conclusions and main approaches to evaluating of performance of Public Procurement Audit
including set of practicable criteria might be presented.
Part 6 describes preparation of reporting documents on Public Procurement Audit
findings including: procedure of preparation and presentation of the audit conclusion, the
qualitative estimation scale may be used as a basis, requirements to the composition and
contents of the report on the results of Public Procurement Audit, recommendations on
approximate structure to ensure methodological unity of the process of compilation and
preparation of the final audit report, drawing up conclusions based on the Public Procurement
Audit findings.

2
 Contents

MAIN TERMS AND ABBREVIATIONS.............................................................................5

INTRODUCTION...............................................................................................................7

1 THE PUBLIC PROCUREMENT PROCESS LIFE CYCLE AND MAIN PRINCIPLES12

1.1. Public Procurement process life cycle......................................................................................................12

1.2 Public Procurement principles......................................................................................................................15

1.3 Ethics in Public Procurement........................................................................................................................17

2 THE AREA OF APPLICATION OF THE GUIDE. OBJECT AND SUBJECT OF

PUBLIC PROCUREMENT AUDIT..................................................................................21
2.1 The area of application of the Guide............................................................................................................21

2.2 Object and subject of Public Procurement Audit.....................................................................................22

3 PUBLIC PROCUREMENT AUDIT: FUNDAMENTAL PRINCIPLES AND

ASSUMPTIONS..............................................................................................................29
3.1 Principles of Public Procurement Audit......................................................................................................29

3.2 Stakeholders of Public Procurement Audit................................................................................................33

3.3 Objectives of Public Procurement Audit....................................................................................................34

4 CONTENTS OF PUBLIC PROCUREMENT AUDIT....................................................41

4.1 Types of Public Procurement Audit.............................................................................................................41

4.2 The choice of methodological and information support of audit........................................................47

4.2.1 Methodological support of Public Procurement Audit..................................................................................47
4.2.2. Information support of Public Procurement Audit.......................................................................................50

4.3 Public Procurement Audit in the context of application of modern Information Technologies..62

5 ORGANIZATION AND IMPLEMENTATION OF PUBLIC PROCUREMENT AUDIT. 64

5.1 Planning of an audit and preparation of information requests............................................................64

5.2 Audit risk in Public Procurement Audit......................................................................................................71

5.3 Methods and main procedures of Public Procurement Audit...............................................................78

5.4 Collection and analysis of audit evidence..................................................................................................90

5.5 Audit documentation........................................................................................................................................93

5.6 Types of audit conclusions............................................................................................................................97

3
5.7 Main approaches to evaluating of performance of Public Procurement Audit..............................100

6 PREPARATION OF REPORT ON THE PUBLIC PROCUREMENT AUDIT FINDINGS

.......................................................................................................................................110
6.1 Procedure of preparation and presentation of the audit conclusion................................................110

6.2 Requirements to the composition and contents of the report on the results of Public
Procurement Audit................................................................................................................................................115

6.3 Drawing up conclusions based on the Public Procurement Audit findings...................................118

ANNEXES.................................................................................................................................................................122

Annex 1. Main Types and Methods of Public Procurement.......................................................................122

Annex 2. Examples of Issues to Be Addressed in the Frame of Public Procurement Audit.............122

Annex 3. Analysis of Principal Violations Faced in the Practice of Public Procurement Audit.......122

Annex 4. Identification of the Facts of Fraud, Abuse and Corruption in Public Procurement Audit
....................................................................................................................................................................................122

Annex 5. Pilot Projects and Practical Examples...........................................................................................122

Annex 6. The summary of the EU CC Public Procurement Audit Guide................................................122

Annex 7. Glossary of Public Procurement Audit.................................................................................................122

4
MAIN TERMS AND ABBREVIATIONS

Public Procurement Purchase of goods, works or services for the

satisfaction of public needs which is made by
procurement customer or a procurement agency
authorized in accordance with national laws on public
procurement
Public Procurement A contract / agreement by Procurement Customer
Contract with Procurement Contractor in accordance with
national laws on Public Procurement in which a
Procurement Customer agrees to acquire works,
goods or services
Public Procurement SAI`s audit activity which is dealing with  Public
Audit Procurement processes and implementation of Public
Procurement Contracts
Goal-oriented Procurement process which is financed, in part or
procurement process fully, with public budget funds, whose objectives,
time limits, resource inputs, results and users of
results are predetermined and whose life cycle has a
completed nature.
Audit domains The integration of subjects of the audit whose similar
features allow using common methodological audit
techniques and tools in the framework of the audit
Stakeholders The individuals, organizations or classes thereof who
have an interest in results/outcomes of Public
Procurement
Intended users In accordance with national legislation frameworks,
individuals, organizations or classes thereof for whom
the auditor prepares the reporting document. The
intended users may be legislative or oversight bodies,
those charged with governance or the general public.
Types of audit There are two types of theoretical audit
approach approach: “audit of method” and “audit of subject”
“method’s audit” (audit by one method) may include
financial audit, compliance audit and performance
audit, since they differ both in terms of examined
aspects of the use of budgetary funds - financial rules,
regulations compliance and the correlation between
outcome and costs, and in terms of methods of

5
 accomplishment of audit engagements.
The standards of the “subject’s audits”, which have
been developed pertaining to a specific subject area,
should be based on the use of “method’s audits”,
while it is possible to use special procedures specific
for the given subject area but belonging to this or that
principal method
Information Support There are two types of information sources of Public
of Public Procurement Audit – Internal and External.
Procurement Audit Internal Information Sources - information
databases formed in the frame of Supreme Audit
Institutions to support the audit; their intended users
are employees of SAIs.
External Information Sources - information
databases of all types, both with public access and
departmental ones, which may be used to support the
audit in order to obtain various audit evidence
Big data Large and complex data beyond the traditional
capacities of IT infrastructure collated from all
imaginable sources, which leverages Information as a
vital asset.
Includes:
Structured/unstructured data
Internal/external data
Formal and informal communication
Auditee Audited entity

PP Public Procurement

PPC Public Procurement Contract

PPA Public Procurement Audit

IT Information Technology

EU European Union

UNISTRAL United Nations Commission on International Trade

Law
SAI Supreme Audit Institutions

6
INTRODUCTION

1 The role of Public Procurement Audit and the area of its application

Public procurement refers through which state and other public organizations
purchase goods, services and works to use in the accomplishment of their
objectives. In essence, it refers to choosing suppliers, contracting the resources
they provide and accepting and paying for them.
Although this concept seems quite simple, public procurement turns out to be
a very complex activity. Among other aspects, Public Procurement:
 Involves all types of public entities, including governmental and non-
governmental organizations, both commercial and non-commercial;
 Covers virtually all sectors of national economies in a meaningful scale:
it includes all sorts of goods, services and works and involves important
projects in areas such as education, health, military and infrastructure,
which can have a major impact on economic performance and
development. It represents an average of 10-20% of GDP across the
world1;
 Is financed by public funds and accounts for a substantial portion of
public spending (estimates indicate that public procurement accounts for
nearly 30% of the total amount of public expenditure in OECD countries
and can represent up to 50% of total government spending 2) and, thus,
relates with the performance of an important part of the state and public
budgetary assignments;
 As a public activity, is governed by public law and detailed procedures.
Depending on countries, the procurement law can be very
comprehensive. Officials must ensure that public procurement is
managed in accordance with the objectives, principles and procedures
defined;

1
This average is mentioned in UNCITRAL documents.
2
Idem.
7
  Is conducted on the public interest and is open to public scrutiny. Thus, it
involves specific accountability mechanisms;
 Develops through a long cycle, starting with the definition of needs,
including the choice of provider and the establishment of the contract and
going until execution, payment and assessment of results;
 Due to the size of funds involved and to the competing interests at stake,
is one of the public activities more vulnerable to waste, fraud and
corruption.
Therefore, public procurement is also a key instrument through which
governments can influence directly or indirectly all aspects of public and economic
life and through which they can pursue strategic national objectives, including
economic development.
Due to its market significance, public procurement has also gained a global
importance, leading to transnational agreements and common regulation trends.
Although public procurement systems may differ significantly from country
to country, there are nowadays a number of common characteristics and applicable
common principles. These are primarily open competition, non-discrimination,
transparency, fairness and efficiency.
Furthermore, common interests, agreements in the framework of the World
Trade Organization (WTO) and many years of shared experience and influence
explain that many of the current national legal and organizational public
procurement systems are based on typical model provisions of the relevant
international legislation. National legislation on this field generally aligns with the
principles of the UNCITRAL3 Model Law on Public Procurement. In some
economic integration spaces, there are even supranational regulations for public
procurement. This is the case for the European Union (EU) countries, where such
provisions are determined by the European Directives on Public Procurement.
Accordingly, public audit and control activities, implemented by Supreme
Audit Institutions (SAIs), often refer to public procurement cases and are affected

3
UNCITRAL: United Nations Commission on International Trade Law

8
by the mentioned complexity. Although international auditing standards for SAIs
(ISSAI) apply to different types of public financial operations, no specific
INTOSAI standard for public procurement audit is currently available. Therefore,
when auditing these cases, public sector auditors have to draw on the too general
requirements of ISSAI and on their own internal experience, often different from
the experience of other SAIs.

2 The name of Task Force in compliance with principal objectives and goals
of the Guide

In the process of document formulation, Task Force decided to correct the

name of the project. We consider that the name “Public Procurement Audit” is
more appropriate for the Guide project, than “Procurement Contract Audit”. If we
speak about Public Procurement, we mean all the stages of procurements life cycle.
Besides, the Procurement Contract Audit is the part of Public Procurement Audit
and can be used at the stage of preparation and implementation of any public
procurement contract.
Hereinafter we suppose to use the term of Public Procurement Audit and
propose an adjustment of all relevant documents.

3 Activities of the INTOSAI Task Force on Public Procurement Audit

The INTOSAI Task Force on Public Procurement Audit established in the

frame of INTOSAI has determined its Goal 2 in the Terms of Reference as:
“Preparation of practical Guide on Public Procurement Audit and development of
information exchange system pertaining to Public Procurement Audit matters”
(hereinafter referred as the Guide) with the following objectives:
1. Review and update of INTOSAI glossary regarding Public Procurement
Audit;
2. Creation of a permanent system of experience exchange with the use of
advanced IT and traditional forums, including the development and
maintenance of the permanent WEB-site;

9
 3. Preparation and implementation of Public Procurement Audit pilot
projects in INTOSAI Task Force member-states regarding Public
Procurement Audit;
4. Drafting of Public Procurement Audit Practical Guide

4 Assignment of the Guide and further work on the international standard on

Public Procurement Audit

The INTOSAI Task Force on Public Procurement Audit was established in

order to identify, summarize and exchange the existing international auditing
practices in this field and to favour the development of a unified methodological
approach. One of its objectives was to prepare a Public Procurement Audit
Practical Guide, as a step towards the preparation of an INTOSAI international
standard for Public Procurement Audit.
This Practical Guide was prepared on the basis of materials and examples
provided by members of the Task Force and other INTOSAI members and focused
on identifying:
 Common approaches of SAIs in applying the general auditing
principles to public procurement audit;
 Tools, instruments and procedures commonly used in auditing these
cases;
 Suitable and applicable recommendations for public procurement
audit, irrespective of any national jurisdiction.
The Guide considers the main stages of the goal oriented public procurement
processes.
Thereby the final goal of the work towards the improvement and
development of Public Procurement Audit is the preparation of the international
standard on Public Procurement Audit in the frame of INTOSAI.

10
5 Members of the INTOSAI Task Force on Public Procurement Audit whose
materials have been used in the Practical Guide:

Azerbaijan
Belarus
China (PRC)
Kyrgyzstan
Latvia
Portugal
Russian Federation (the author of the principal text)
Slovenia
South Africa (RSA)
Zambia

6 Also in the Practical Guide have been used being in the open access
materials SAI

Ethiopia
India
Kenya

11
1 THE PUBLIC PROCUREMENT PROCESS LIFE CYCLE AND MAIN
PRINCIPLES

1.1. Public Procurement process life cycle

As already mentioned, Public Procurement processes are targeted at acquiring

goods, services or works, financed, fully or in part, by state or other public
organizations’ budget funds. The consumers of those resources can include both
public authorities, for the support of their activities, and society, including entities
and individuals (for instance, in case of providing public services).

These processes have predetermined objectives, time limits, resource inputs,

results and users and they develop through a typical cycle.

The procurement cycle describes the entirety of the process through which
public organizations identify, prepare and execute the acquisition. There are
several possible representations of this cycle, all starting with the definition of
needs and closing with the evaluation of results.

We consider a separate Public Procurement to be a goal-oriented process

(Process), any management process which is financed, in part or fully, with public
funds, whose objectives, time limits, resource inputs, results and users of results
are predetermined and whose life cycle has a terminative nature.
Turning to the process approach makes it possible to represent each of the
selected stages (steps) as a set of processes of the second level (subprocesses).
Besides, when considering the progress of their implementation, it is necessary to
base on the following conceptual framework:
This cycle can be described according to the following conceptual framework:

12
 2.PREPARATION OF 3.IMPLEMENTATION 4.USAGE OF
PROCUREMENT OF PROCUREMENT RESULTS

1.SUBSTANTIATION 5.COMPLETION OF
OF PROCUREMENT Conclusions and PROCUREMENT
Recommendations

ENVIRONMENT
Objectives, tasks and functional characteristics of customers
Other processes
Budgets of all levels
Strategic planning documents
Plans of social-economic development
National programs

The process of each particular procurement is goal-oriented to the full

extent, since all steps should be consistent with the needs to be addressed, which
coincide with the results to be achieved.
The represented stages consist of the following content.
1. The substantiation of the procurement includes the understanding and
definition of needs, the budgeting and the planning of the whole process. It
terminates with the description of the requirements to the procurement, in terms of
amounts of the resources employed, the implementation period, anticipated
qualitative and quantitative characteristics and indicators of the planned
procurement process as well as its interaction with other ongoing and planned
procurement processes.
2. The preparation of procurement includes choosing a procurement
method, inviting to tender, evaluating and selecting offers, negotiating and
endorsing the procurement process implementation plans and awarding and
signing the contract with the supplier.
3. The implementation of procurement includes managing the contract,
monitoring work in progress, receiving and verifying deliveries and paying
suppliers. It terminates with the acceptance of the procurement results by the

13
 customer. Depending on the contract terms, the acceptance may be carried out in
accordance with contract stages, for example, development of engineering design,
detailed design documentation, etc., which is crucial, for instance, for operations
on creating information systems and in construction. In such cases, the stage of
procurement implementation could be considered as a separate project.
4. The usage of procurement results refers to the use by customers of the
good, service or work acquired and is terminated with the end of the use of the
product obtained as a result of the procurement. Conclusions and recommendations
on the results of this product use are prepared.
5. The completion of the use of procurement terminates with disposal,
evaluation, findings and recommendations on its results.
In graphic form, this goal-oriented procurement process is shown in the
figure below.
Public Procurement process life cycle

Making a contract Performance of the

contract

2. Preparation of the 3. Implementation of 4. Usage of the

procurement the procurement procurement results

Carrying out the Contract support Sphere and format of the

procurement procedure use

Preparation of Acceptance of the Results of the use

documentation procurement results

Determining the
procurement parameters
5. Completion of the
Determining goals and
1. Substantiating use
objectives of the procurement requirements to the
procurement Conclusions and
recommendations on the
results of the
procurement use

Goals Budgets Services

Implementation plans Works
Results
s National programs, Projects
Requirements Goods
Development plans
Strategies

14
 The SAI’s competence should cover the entire cycle of public procurement
considering it as a system goal-oriented process.

According to the audit objectives, the audit to be performed can cover the
entire cycle, focus in specific stages or analyze sub processes of a stage.
Basic approaches to Public Procurement are given in Annex 1.

1.2 Public Procurement principles

One can say that public procurement is globally governed by a set of common
principles that reflect international best practice.
United Nations (UN), through UNCITRAL, has been issuing guidance on
public procurement regulations, aimed at achieving value for money and avoiding
abuses in the public procurement processes. The UNCITRAL Model Law on
Public Procurement has been prepared with a view to supporting the
harmonization of international standards in public procurement and, in the 2011
version4, is said to reflect international best practice and to be designed as
appropriate for all States. The document takes account, among others, of the
provisions of the WTO Agreement on Government Procurement, the European
Union Directives on Procurement and the UN Convention Against Corruption.
The text of this model law promotes the principles of:
 Open participation and competition
 Fairness and equal treatment
 Integrity
 Transparency
 Efficiency
Nowadays, the main provisions of national laws on public procurement are
largely consistent with the principles of this UNCITRAL model law.
Principles are meant to govern action towards commonly accepted values.
In public procurement, they play an important role in providing guidance to

4
Adopted in Vienna on July 1, 2011, at the 44th session of UNCITRAL.

15
the legislator when adopting the content of regulations and in guiding the
interpretation of legal provisions, especially if they are defined inaccurately. The
correct understanding of the principles facilitates the interpretation of legal and
managerial actions carried out in the course of public procurement. They also
guide decision-making. The contracting entity launching the public procurement
needs to ensure that the fundamental principles of public procurement are
maintained throughout the entire process.
The national public procurement laws and regulations usually define and
detail the content of the principles and the provisions to achieve them. Some
common features are the following:
 Public procurement opportunities should be open to all qualified
interested economic operators
 Open competitive tenders provide equal opportunities and increase the
chances of obtaining good value
 No unjustified restrictions to competition and no discriminatory
requirements or procedures should be introduced
 The public procurement system must create fair and equal conditions for
competition among candidates to the contract
 Clear, objective and known rules and criteria should be defined, based on
the relevant requirements
 Information should be provided equally to all parties of the process
 Public procurement decision-making and actions should be unbiased
 Scope for undue influence should be limited
 The public procurement officials should act solely on the public interest
 Procurement professionals should comply with a code of conduct and
declare any conflict of interests
 General information about the procurement should be available to public
 Public procurement should be mainly driven by the objectives of
satisfying the identified needs and achieving the defined results

16
  Expenditures with public procurement should be goal oriented and cost
effective

1.3 Ethics in Public Procurement

In public procurement, ethics or integrity is twofold.

As suppliers/economic operators, bidders expect to have fair access to
markets, expect to rely on the information disseminated by the public authority and
are entitled to be treated fairly and in equal terms with others.
As payers, citizens expect that public procurement is conducted on the sole
benefit of public interest and that prices paid for goods, services and works are
acceptable and represent good value for the public funds expended on them.
However, as already mentioned, due to the size of funds involved and to the
competing interests at stake, public procurement is one of the public activities
more vulnerable to waste, fraud and corruption.
Private profits, financial gains and political benefits can motivate unethical
and/or corrupt practices of procurement actors, for example by providing
confidential advantageous information, by influencing decisions and by favoring
contractors.

The Annex 4 to this Guide provides detailed examples of the risks of

corruption that may arise during the several stages of the public procurement
process.

These situations represent extra costs (estimated at least at 20% of the total
cost of contracts), reduced quality and value and lack of trust in public
organizations.
To minimize them, some principles and policies to enhance integrity in public
procurement should be implemented. They should be mainly targeted at limiting
exceptions to the use of competitive tendering and at adopting measures against
conflicts of interests and corruption. The main ones are described below.
17
a. Using mechanisms to prevent risks to integrity.

Risks of unethical or corrupt action occur in all the stages of the procurement
cycle. The preparation of the procurement is the stage where regulations and
controls are more commonly applied, but during the substantiation (definition
of needs), implementation (delivery) and completion (evaluation) significant
risks also exist. Public organizations conducting public procurement can define
standards for integrity risk analysis in these processes, map risks to integrity
according to the stages where they are more likely to appear and design policies
to address them. Examples of these policies are: use of electronic systems and
tender templates, implementation of codes of conduct, delivering of ethics’
training and advice, rotation of staff and collecting and disclosing information
about interests and political funding. (Further detail can be found in Annexes 3
and 4 to these Guide, where concrete possible strategies and measures are
described for each identified corruption risk, according to the several stages of
the procurement process).

b. Enhancing high professional standards of knowledge, skills and integrity.

Conducting public procurement involves a lot of skills and impactful decisions.

The officials who participate in these processes can influence them, for example
by misunderstanding, disclosing or hindering information, by including
inaccurate, confusing or discriminatory procurement requirements and
specifications, by failing in detecting false information or by acting unethically
in vary many ways. Public entities should define skills and professional
certifications needed to work with public procurement, involve experts as
needed and collect and share data about procurement. They should also define,
disseminate and implement standards of conduct for public officials at risk
areas, should identify and manage conflict of interest situations, should regulate
and limit the use of confidential information and should apply sanctions where
appropriate.

18
c. Ensuring consistency with needs and intended purposes.

Unethical or corrupt actions, influenced by political, private or financial

interests may lead to decisions to purchase goods and services or make
investments that are unnecessary or inadequate to needs or to accept deliveries
that do not meet the agreed quality or intended purposes. Using fair and
transparent processes to select independent consultants that need to be involved
in the procurement, reviewing cost-benefits analysis and risk assessments,
scrutinizing discretionary powers and monitoring and evaluating public
procurement about accuracy of needs’ assessment and performance in
achieving the intended results will reduce the opportunities for these actions.

d. Maximizing transparency in the entire procurement cycle

Transparency and open and fair competition disfavor unethical and corrupt
actions and allow public and peer control over them. Opportunities for favoring
and discrimination will be reduced if direct contracting is kept to strict
exceptional circumstances and if key documents and decisions are substantiated
and published (procurement requirements, invitations to tender, evaluation and
award criteria, etc.). Good practices include documenting decisions, making
information publicly available, providing clear rules and guidance, establishing
independent committees, defining strict legal requirements to non-competitive
procedures, avoiding discretion decisions of a single individual, demanding
and publishing justification of decisions and implementing a review and remedy
system.

e. Fostering high standards of integrity also in private sector

Harmful practices often come from the private sector actors, who can, for
example, provide false or incomplete information, collude with others or deliver
lower quality than the one quoted in contracts. Public entities should define and
monitor standards of conduct for business, use pre-qualification requirements,
record feedback on experience with suppliers and punish integrity breaches.
They should be attentive to the conduct of candidates and contractors during the

19
 preparation of the procurement but, in particular, during the implementation
phase (contract management).

f. Using effective mechanisms to monitor and control public procurement,

detect misconduct and apply sanctions accordingly

Applying appropriate mechanisms to monitor and control the whole public

procurement process will reduce opportunities for integrity breaches. These
include clear chains of responsibility in the procurement process, segregation of
duties and authorization, systematic recording, internal controls, reporting tools
and checks. Progress and costs of procurement should be monitored and
published, independent oversight, evaluation and audit should be conducted
(including real-time controls) and identified irregularities should be sanctioned.
Mechanisms and protection to whistleblowers should also be established.

g. Including external scrutiny

Other potential suppliers’ and public’s scrutiny can be an effective tool to

mitigate integrity breaches in public procurement. Good practices include
handling complaints from potential suppliers in a fair and timely manner,
implementing independent appeal bodies, establishing dispute settlement
mechanisms and ensuring public access to information and to evaluation and
audit reports.

20
2 THE AREA OF APPLICATION OF THE GUIDE. OBJECT AND
SUBJECT OF PUBLIC PROCUREMENT AUDIT

2.1 The area of application of the Guide

The area of application of the Guide - Public Procurement Audit, which

is an independent field of activity of Supreme Audit Institutions (SAI). The Guide
should cover all processes and relevant subprocesses of public procurement audit

The area of application of the Guide

- variants of approaches in SAIs of countries
Belarus
Audit of state-funded procurements

Kyrgyzstan
The audit of procurement of organizations that purchase goods, works, services and
advisory services at the expense of public funds

China
The Public Procurement Audit guide can normally cover all public works, services
and supply contracts entered into by a public authority such as state organs and
non-profit organizations which use public funds.
The Public Procurement Audit guide help to explain the following questions:
Whether the annual budget review of government procurement is to follow the
statutory procedures.
Whether the government procurement and purchase have really happened, whether
there are any virtual transactions in the government procurement projects.
Whether the procurement scope, size, quantity, variety , price and the amount is
reasonable.
How to maintain a fair and competitive market, and fight against corruption, to
prevent the occurrence of various kinds of loopholes, by strengthen auditing and
supervision of government procurement.
Whether the procurement is necessary based on the actual needs of the department
or the public, whether there is a scientific argumentation, whether the procurement
project is value for money, to ensure the use of the public funds is efficient and
effective.

Latvia
The audit subject – public procurement activity. The audit may be performed at all
stages of the public procurement cycle.

Portugal
Types of audits and possible objective of audits.
According to their mandates and audit priorities, SAIs may audit public procurement
21
 in different ways and approaches.

Slovenia
Subject of the Public Procurement Auditare all contracts and their modifications
during their term of validity that were stipulated by public authorities regardless if
the public procurement procedure was initially carried out.

South Africa
SAI-SA adopted a transversal audit approach in order to ensure the risk of fraud and
non-compliance in procurement processes are appropriately responded to in a
comprehensive and consistent manner.
The audit of procurement and contract management is largely a compliance audit
based approach, with additional focus on value adding audit aspects. For this
purposes, the focus is on:
 Assessment of internal controls;
 Conflict of interest (interest in suppliers);
 Unfair and uncompetitive procurement practices; and
 Contract management on compliance audit.
The approach also links with financial audits through the audit of expenditure
classes of transaction with respect to the following objectives:
 transactions are approved by an appropriately delegated official
(compliance);
 payments are made at correct prices in accordance with the contract; and
 payments are made for goods/services that had been received at the correct
quality and quantity in accordance with the contract

2.2 Object and subject of Public Procurement Audit

SAIs may be mandated to perform many types of engagements on any
subject of relevance to the responsibilities of management and those charged with
governance and the appropriate use of public funds and assets. The extent or form
of these engagements and the reporting thereon will vary according to the
legislated mandate of the SAI concerned.
Although the audit mandates, powers and activities of SAIs differ, according
to their countries’ legislation and environment, they typically audit financial,
compliance and/or performance aspects of public management.
Auditing public procurement means focusing on a particular activity of
public management. This Guide is meant to be applied when SAIs audit public
procurement cases financed by public funds.

22
 When SAIs look into public procurement, they will mainly choose between
adopting a financial, a compliance or a performance approach, accordingly
answering the fundamental questions:
 Were public procurement records and transactions truly and fairly
reflected in the financial statements?
 Did public procurement operations comply with applicable laws and
regulations?
 Did the purchases, the processes used, the allocated funds and the
correspondent results follow the main principles of economy, efficiency
and effectiveness?
Even so, the global audit strategy and priorities of each SAI may determine
different types of audit approaches and envisaged results.
This explains why, when we compared SAI’s practices of auditing public
procurement, we found that they do it in different ways. The following table lists
some of the observed approaches.

 Studying public procurement activities in the frame of financial audit, compliance

audit or when assessing the effectiveness of a certain public entity
 Studying certain spheres and procedures in different entities or programs, comparing
practices and drawing global conclusions (ex.: construction of schools or roads,
consulting services, changing of contracts, IT investments, projects with
international budgeting).
 Assessing the effectiveness or benefits that resulted from certain procurements or
investments.
 Studying the way procurement is planned and organized at the government level or
at certain groups of entities as to conclude on improvements that can be introduced
(ex: centralized purchasing)
 Examining compliance to specific regulations and the accordance of the claims and
actions undertaken as a reaction to detected violations of those regulations.
 Studying the risks and tools used for fraud and corruption control in the sphere of
procurement.
23
 The objectives of the audit, the audit questions and the aspects to verify and
assess will depend on the chosen approach and can include a large variety of
issues. Accordingly, the audit may be performed over all the stages of the public
procurement cycle or mainly targeted at some of them.
This Guide considers all these possibilities and tries to detail them in practical
terms.

Main goals of public procurement audit

Main goals of public procurement audit - to assess how efficiently,
effectively, in compliance with requirements of laws and established ethical
standards, contracting authorities are implementing their function of public
procurement, with subsequent providing all interested users with independent,
objective and reliable information, conclusions and opinions based on sufficient
and proper evidence.

Auditee entities of Public Procurement Audit

The auditee entities (auditee) are state institutions and agencies or other
entities in compliance with applicable national laws on Public Procurement Audit

Audit domains of Public Procurement Audit

We suggest that the audit domain should be understood as a integration of

subjects of the audit whose similar features allow using common methodological
audit techniques and tools in the framework of the audit.
The audit domains of Public procurement audit can be characterized in the
following way:

1. Substantiation of Public Procurement

Subject matter of the audit: Strategic and conceptual documents regulating
the directions of development of the subject-area supported by procurement
processes; Documents substantiating and justifying the need in procurement;

24
objectives, tasks and functional characteristics of procurement customers,
anticipated results and users of the results; Documents containing requirements for
qualitative and quantitative characteristics (indicators) of a product created in the
frame of procurement implementation.
Objective of the audit: To assess the reasonability and feasibility of
procurement.

2. Matching of documents determining the necessity of procurement

and its interdependence and interrelation with other procurements
Subject matter of the audit: Strategic planning documents, socio-economic
development plans, national programs, other procurements.
Objective of the audit: To assess the balance, consistency and continuity of
the procurement system.

3. Allocation and use of resources for procurement

Subject matter of the audit: Planning documents with sections dedicated to
the use of resources allocated for organization and implementation of procurement;
Validity of financial transactions involving implementation of measures related to
procurement, budget accounting, budget and other reporting of the audited entities;
Use of the resources allocated for organization and implementation of procurement
when implementing financial and other activities of the audited entities at all stages
of the procurement life cycle.
Objective of the audit: Evaluation of resourcing of procurement and
compliance with the program-oriented principle of its implementation.

4. Legality of implementation of all stages of procurement

Subject matter of the audit: The procedure for documenting the
procurement transactions, processes and procedures for compliance with
legislative, regulatory or special rules and requirements. The procedure for
implementing current operations provided that there are applicable regulations
governing conducting thereof: Compliance with the terms of contracts; Errors and
breaches leading to overruns or misuse of public funds.

25
 Objective of the audit: Assessment of conformity of documents to laws,
regulations or special rules and requirements.

5. Organization of procurement process management

Subject matter of the audit: Documents determining the structure of
organization of procurement process management; Implemented procurement
process management technologies. Characteristics of the procurement management
systems in audited entities; Compliance with sustainable development principles
(for major projects focused on socio-economic development)
Objective of the audit: Assessment of the procurement management system.

6. Quality of procurement process planning

Subject matter of the audit: Quality of cash flow planning, designed to
ensure the stages of procurement life cycle. Quality of the network schedule of
procurements and description of interrelations between them; Promptness in
implementation of scheduled engagements and adjustments thereof; Fullness and
sufficiency of the procurement process detailing into separate subprocesses and
works and sufficiency of description of each operation; Validity of determination
of the duration of performance of each procurement given the amount of resources
allocated for its implementation; Complementary dependence of procurements,
including the consistency (non-contradiction) of the sequence and duration of
procurement operations, taking into account existing restrictions; Resource
intensity as regards validity of the amount of resources allocated for the
performance of procurement, their compliance with the contents of works and
duration of operations; Resource intensity as regards the total volume of resources
required for procurement implementation, performance of operations in each
period of procurement implementation.
Objective of the audit: Assessment of feasibility and quality of procurement
planning.

7. Efficiency and effectiveness of the organization and conduct of

procurement process

26
 Subject matter of the audit: The spending of budget funds from the
standpoint of outcomes, both achieved and potential, at all stages of procurement
life cycle, for the attainment of targets, solution of the tasks set for sustainable
development of the state and fulfillment of the functions imposed on it.
Objective of the audit: Assessment of effectiveness and efficiency of
procurement process organization.

8. Risk management organization

Subject matter of the audit: Availability and efficiency of the risk
management system during the implementation of the procurement process life
cycle. The system of internal control of the public procurement process in audited
organizations and institutions.
Objective of the audit: Assessment of the system of internal control of the
public procurement process.

9. Satisfaction of the user with the procurement results achieved

Subject matter of the audit: Qualitative and quantitative characteristics
and indicators of the product obtained as a result of the procurement in the light of
anticipated results from the position of users of the procurement results; Findings
and recommendations on the procurement results and feedback on those results,
including expert assessments and results of surveys.
Objective of the audit: Assessment of compliance of indicators to objectives,
consistence and continuity of procurement, transparency (openness) of results.

10. Documents that contain indicators characterizing objectives and

results of procurement
Subject matter of the audit: Planning documents that contain indicators
characterizing the goals of procurement; Documents that contain indicators of
procurement results, including consolidated statistical information; Documents that
contain comparison of procurement indicators of various countries.
Objective of the audit: Assessment of completeness of the system of
indicators for the measurability of the goals set.

27
 Such a variety of subject areas is not accidental and is objective in nature,
since it reflects a complicated, and sometimes controversial, practice of
procurement in public sector.

Audit subjects and objects

- variants of approaches in SAIs of countries

Latvia
The audit subject – public procurement activity. The audit may be performed at all
stages of the public procurement cycle.

Portugal
According to their mandates and audit priorities, SAIs may audit public procurement
in different ways and approaches. For example:
 Examining the procurement function as part of a financial, compliance or
performance audit of a specific public body
 Looking into specific areas or procedures in several bodies or programs,
comparing practices and extracting global conclusions (exs: construction works
in schools or roads, consulting services, modifications of contracts, IT
investments, internationally funded projects, etc.)
 Assessing compliance and action taken in response to identified irregularities
 Looking into efficiency or value added by specific or types of procurement or
investments
 Examining the risks and controls for preventing fraud and corruption in
procurement

Slovenia
Subject of the Public Procurement Audit are all contracts and their modifications
during their term of validity that were stipulated by public authorities regardless if
the public procurement procedure was initially carried out. Estimated value of the
contract placed by a public authority must meet the financial thresholds which
determine the application of the public procurement rules (Article 4 of Directive
2014/24/EU)

28
3 PUBLIC PROCUREMENT AUDIT: FUNDAMENTAL PRINCIPLES AND
ASSUMPTIONS

3.1 Principles of Public Procurement Audit

As already stated means using the SAI’s audit approach to focus on a

particular activity of public management. Therefore, the general auditing principles
are applicable to this audit.
These principles apply at two levels:

Ethical principles ISSAI 30

Principles of
transparency and ISSAI 20
accountability

Fundamental ISSAI 100

Principles of
public-sector ISSAI 200
auditing ISSAI 300
ISSAI 400

Ethical principles
Ethical values and principles, set out in the INTOSAI Code of Ethics (ISSAI
30), bind SAIs and their auditors when performing any type of audit. SAIs can
adopt their own codes, but these should be based on ISSAI 30 and be at least as
stringent as this ISSAI.
SAI’s stakeholders are entitled to expect that SAIs and their auditors conduct
themselves in each audit on the basis of the highest standards of professional ethics
and conduct.
According to ISSAI 30, ethical conduct in audit is based on five fundamental
values and the respective guiding principles5:
 Integrity: acting honestly, reliably, in good faith and in the public
interest;

5
this part is based on the endorsement version of the redrafted ISSAI 30, to be approved in the
forthcoming INCOSAI
29
  Independence and objectivity: being free from circumstances or
influences that compromise, or may be seen as compromising,
professional judgment and acting in an impartial and unbiased manner;
 Competence: acquiring and maintaining knowledge and skills
appropriate for the audit role, and acting in accordance with applicable
standards and with due care;
 Professional behavior: Complying with applicable laws, regulations
and conventions and avoiding any conduct that may discredit the SAI;
 Confidentiality and Transparency: Appropriately protecting
information, balancing this with the need for transparency and
accountability.
These principles suggest for instance that, when auditing public procurement,
auditors should:
 Have the required scope of knowledge about this complex subject
 Understand the conflicting interests that may be at stake and the
impact they may have in the procurement and its results
 Be impartial and unbiased, formulating conclusions on the sole basis
of evidence and objective criteria and refraining from political
assessments or public declarations
 Avoid any situations where they themselves may have internal
conflicts of interests associated with any issues arising from the audit
(for example relationships or interests related with the entities,
enterprises or individuals that were involved in the procurement)
 Be professional and avoid familiarities with auditees or contractors
 Protect any confidential information, for instance of commercial
nature, that they encounter during the audit
 According to the applicable national regulations, inform the competent
authorities of the cases of corruption or other crimes that they identify
or suspect during the audit.

30
 Fundamental principles of public sector auditing
Public sector auditing provides legislative and oversight bodies, those charged
with governance and the general public with information and independent and
objective assessments concerning the stewardship and performance of public
policies, programmes or operations. Thus, SAIs’ audit work is fundamental for
democratic societies, which are based on transparency, accountability, governance
and performance.
Transparency and accountability are powerful means for preventing and
combating corruption, improving quality and efficiency of public management and
enhancing responsibility for budgetary decisions.
Due to the size of public funds and risks involved in public procurement,
auditing this area is, therefore, a way of maximizing the transparency and
accountability of the public procurement processes, thus contributing to reduce
risks of corruption and misuse and promoting good governance.
ISSAI 20 explains how the principles of transparency and accountability
should be incorporated in SAI’s own governance and practices, ensuring the
accountability and control of the audit institution itself.
As regards audit work, compliance with the requirement of transparency is
also two-fold. It means that the audit procedures should follow a well-established
and well-known set of standards, the implementation of which can be verified and
controlled, and should accept in some steps the contributions of audited entities.
On the other hand, it implies unbiased full-scale disclosure of audit findings to the
public knowledge.
ISSAI 100 defines the principles of public sector auditing and the
requirements and approaches to the main stages of an audit (from the planning
procedure up to the process of ensuring the quality of the audit). These principles
refer to ethics, professional judgment, due care and skepticism, quality control,
skills, audit risk, materiality, documentation and communication. All of them apply
to public procurement audit.

31
 From them we quote the principle of professional skepticism. In public
procurement audit this principle means the following.
When planning and conducting an audit, an auditor should use professional
skepticism and understand that in public procurement there could be circumstances
that entail material misstatement and make it inefficient or ineffective. The auditor
should thoroughly study any evidence, documents or statements that are
contradictory or cast any doubt on their validity. This is important regarding the
competing interests in the process and the most vulnerable points for undue
influence or manipulation, for instance:
 When confronting cost-benefit analysis or risk assessments made by
different consultants
 When analyzing the defined requirements and technical characteristics
(to make sure they were not biased or discriminatory)
 When comparing the offer presented in the tender with the contracted
clauses
 When looking into the implementation of the evaluation criteria to the
characteristics of the several offers submitted
 When studying the agreements for the future revision of contracts, or
 When assessing if deliveries were provided and accepted according to
the contract terms.
Professional skepticism should be exercised during the audit in order to, in
particular, avoid losing sight of suspicious circumstances, making unjustified
generalizations when preparing conclusions, using erroneous assumptions in
determining the nature, time limits and scope of audit procedures, as well as in the
assessment of findings thereof.
When planning and conducting an audit, an auditor should not rely upon the
assumption that the management of the audited entity is acting dishonestly, but
he/she should also not suppose that the management is immaculate. Oral and
written statements of executives of the audited entity should not substitute the need

32
to obtain sufficient and relevant audit evidence for the preparation of reasonable
conclusions to base the auditor’s opinion.

3.2 Stakeholders of Public Procurement Audit

According to ISSAI 100, public sector audits involve at least three separate
parties:
 The auditor: the SAI or the persons to whom the task of conducting
the audit is delegated
 A responsible party: the individuals or organizations responsible for
the subject matter information, for managing the subject matter or for
addressing recommendations, and
 Intended users: the ones for whom the audit report is prepared

In the framework of public procurement audit, depending on the type of audit

objectives and on the audit approach, responsible parties and intended users could
vary a lot.
Public procurement is a process where a lot of different entities and
individuals participate:
 Entities that procure goods, services or works can be state central
entities, local authorities, public enterprises or even private entities
receiving public funds
 External consultants providing assistance and studies
 Financing institutions to fund or co-fund partnership projects: these can
include banks or international funding organizations
 All sorts of suppliers / economic operators interested in supplying the
contracting authority
 Owners and managers of electronic platforms where procurement
information is hosted

33
  Associations of public and private entities with interest in procurement
(ex: engineers or constructors)
 Legislative and executive bodies
 Global national or international oversight institutions or commissions
on public procurement
 Citizens as taxpayers, users of the services and/or as individuals
affected by the investments
 Etc.
All these may be responsible parties or users of audit reports, subject to the
audit as owners of relevant information and addressees of recommendations. For
example, the SAI can conclude on a global public procurement audit that laws and
regulations on public procurement should be amended (addressing this
recommendation to parliament) or can conclude that public works projects have
bad quality (formulating recommendations to the professional association of
engineers).

3.3 Objectives of Public Procurement Audit

In general, public sector auditing is described as a systematic process of

objectively obtaining and evaluating evidence to determine whether information or
actual conditions conform to established criteria. Accordingly, the goal of the
public procurement audit will be to provide all interested users with independent,
objective and reliable information, conclusions and opinions, based on sufficient
and proper evidence, to answer identified fundamental questions.
Public sector audits also aim at enhancing accountability and transparency,
at encouraging improvement and confidence in the appropriate use of public funds,
at reinforcing the effectiveness of management and oversight bodies and at
inducing change. These general goals will influence the approach to be used.
Public sector audits can be categorised into one or more of three main types:
audits of financial statements, audits of compliance with authorities and

34
performance audits. SAIs may carry out audits or other engagements on any
subject of relevance, which include reporting on the quantitative outputs and
outcomes of the entity’s service delivery activities, sustainability reports, future
resource requirements, adherence to internal control standards, real-time audits of
projects or other matters. SAIs may also conduct combined audits incorporating
financial, performance and/or compliance aspects.
All public sector audits start from objectives, which may differ depending on
the type of audit being conducted. The objectives, the audit questions and the
aspects to verify and assess depend on the approach chosen.
Accordingly, the audit may be performed over a single entity or a group of
entities, over a single procurement case, a sample of cases or over control systems
applicable to all cases, over all the stages of the public procurement cycle or
mainly targeted at some of them. It can also focus on all or some of the principles
governing public procurement.
The typical audit questions for a financial, compliance or performance
approach are the following:
 Were public procurement operations truly and fairly reflected in
financial statements, freely from material misstatement?
 Were they compliant with applicable laws and standards?
 Were they efficient and effective, did they provide value for money
and is there room for improvement?
Deriving from these main questions, including all or only some of them, there
is a large variety of possibilities.
The next list includes some examples of the many possible audit objectives
and audit questions that SAIs can use when auditing public procurement. They can
be applied separately or in combination.

 Whether the structure, organization, planning and monitoring of procurement

processes management and internal control are efficient and effective and
adequately address identified risks

35
 Whether the procurement and its resourcing were consistent with strategic,
investment or other global policies
 Whether the procurement scope, size, quantity, variety, price and amount were
justified and reasonable towards needs and expected results
 Whether the procurement and the related transactions, processes and procedures
complied with applicable laws, regulations and statutory requirements
 Whether procurement conducted was timely and cost-effective at all stages of the
goal-oriented process
 Whether procurement followed the principles of transparency and fair and open
competition and provided good value for money
 Whether the terms of contracts were complied with, whether the payments made
corresponded to products received and whether the works, goods or services
provided met the expected standards
 Whether measurable results’ indicators were set, how they compared with similar
procurement cases and whether they were met
 Whether the products obtained were in line with users’ expectations and/or
sustainable development principles
 Whether integrity in public procurement was adequately managed as to prevent risks
of corruption and virtual or false transactions
 Whether deviations, violations and deficiencies were frequent, which were the
causes for that and which improvements can be introduced
 Whether former recommendations were implemented and whether improvements
were achieved

So, objectives of Public Procurement Audit are:

- verification, analysis and assessment of the legality, expediency, validity,

timeliness and cost-effectiveness of procurement at all stages of the goal-oriented
procurement process;

36
 - generalization of the results of analysis and evaluation of the procurement
implementation, including the identification of causes of detected deviations,
violations and deficiencies;

- preparation of suggestions to remediate the identified deviations, abuses

and deficiencies and improve the contract system in the area of procurement;

- summary of information on the implementation of proposals to address

deviations, violations and deficiencies identified in the public procurement audit
and improvement of the contract system in the area of procurement as a whole.

The audit objectives may also be detailed according to the stages of the
goal-oriented procurement process.

Important issues to be audited at the several stages of the public procurement

cycle include:

1. At the stage of substantiation of the procurement:

• Legality and validity of the identified need for procurement, in
quantitative and qualitative terms
• Provision of the procurement with the necessary resources in accordance
with the plans
• Degree in which experience gained in previous procurements has been
taken into account in following procurement planning
• Was needs assessment performed to ensure that goods or services are
acquired in order to address the need;
• Are requirements are linked to the budget

2. At the stage of preparation of the procurement:

• Correctness of the procurement procedures’ choice
• Compliance of the procurement procedures’ implementation with
applicable standards and objectives
• If there is a certain classification of contracts, correctness of the contract
type choice

37
 • Has the supplying industry has been analysed to determine the
appropriate procurement strategy.
• Have the specifications for a particular procurement been determined

3. At the stage of implementation of the procurement:

 Evaluation of the contract implementation management process;
identification and management of implementation risks
 Control of contract deliveries: compliance to the contract and to
regulations; timeliness and quality, process of acceptance
 Compliance, timeliness and/or efficiency of the payments’ process
 Evaluation of the procurement results

4. At the stage of the usage of the procurement results:

• How the actual qualitative and quantitative characteristics of the product
obtained as a result of the procurement correspond to the planned
indicators and standards
• Procurement outcomes: assessment on whether procurement results are
used in accordance with the intended goals and on how the consumers are
satisfied with them
• Minimization of “the cost of ownership”
• Results of analysis and assessments on the procurement implementation,
including measures to eliminate identified deviations

5. At the stage of completion of the use:

• Organization and performance of the disposal procedure in terms of
minimization of expenditure and impact on environment
• Implementation of the possible reutilization of certain components of the
product used
• Results of analysis and assessment of the procurement implementation
and experience gained and how it is used for the planning of forthcoming
ones

38
 The wording of main questions is an important stage in the planning of the
audit, since they are the basis of the entire audit activity. No less important is the
fact that only with careful definition and wording of the questions will the field
audit observations meet the audit objectives. Drafting of an exhaustive list of
issues to be addressed during the Public Procurement Audit is an important
methodological objective.
Key practical issues aimed at the testing of the public procurement system at
the audited entity are determined in Annex 2. Besides, it should be remembered
that in practice there is no need to include into the analysis of problems all possible
issues of procurement, but rather determine such list thereof that would meet the
basic objectives set in the audit engagement.

Corruption risks in Public Procurement

As mentioned before, corruption risks are high in public procurement and
contracting authorities should strive towards compliance and cost effectiveness of
public procurement while upholding high standards of probity and integrity. This is
done by implementing integrity management strategies and policies.
Considering the costs involved, SAIs, apart from checking key control points
in the procurement cycle, as they usually do in their audits, could also consider
conducting audits targeted at assessing the implementation of integrity monitoring
systems and policies in the field of public procurement.
In these audits SAIs could assess:
 Whether likelihood and impact of corruption risks in public
procurement were assessed and mapped and whether policies and tools
to address them were adopted
 Whether codes of conduct were adopted and implemented, through
dissemination, training and advice
 Whether tools to identify, manage and mitigate conflicts of interests
were used

39
 Whether clear chains of responsibility were defined and whether
segregation of duties and authorization, rotation of staff , use of skilled
experts and appropriate internal controls were implemented
 Whether independent oversight, evaluation and audit were conducted
 Whether mechanisms to identify and sanction misconduct were adopted
(including protection of whistleblowers)
 Whether transparency was ensure in the entire procurement cycle
 Whether measures were adopted to verify integrity of consultants and
suppliers
 Whether measures were taken to ensure objectivity of needs and
requirements’ definition, unbiased studies and risk analysis, sound and
independent oversight and audit and objective assessment of deliveries
and results

40
4 CONTENTS OF PUBLIC PROCUREMENT AUDIT

4.1 Types of Public Procurement Audit

We think that the types of audit should be described but not judged as better
or worse. Each SAI is capable of choosing those that better fit their mandate and
objectives. Besides, realities and priorities change quite often.
Public Procurement Audit should be carried out as a related set of numerous
interrelated processes, procedures aimed both at control of the procurement process
itself and at the final appraisal of results obtained. However, at the conceptual
level, it is also expedient to turn to the project approach and consider the audit of
public procurement as a way to control the key constraints and the risk of deviation
of the results of public procurement from the established standards.
Besides, there should be a system distinction between the following
methodological approaches:
 ex-ante audit, current audit and ex-post audit;
 financial audit, compliance audit and performance audit.

Ex-ante, current and ex-post audit

The process representation of any stage of public procurement allows forming

a macro time scale of procurement contract performance including ex-ante, basic
and final stages. Accordingly, it is necessary to consider three types of audit: ex-
ante, current and ex-post.
The specified stages are set forth in INTOSAI regulatory documents and used
by national supreme audit institutions. However, the meaning given to the content
and basic procedures of each stage might differ depending on the national features
of statutory regulation of public procurement processes and control over them. The
following is a short description of key approaches to the organization of each of
the audit forms that are common for various jurisdictions.
Ex-ante audit refers to preliminary conditions and requirements, which often
have already been set by the starting point of each process. As a general matter, ex-

41
ante audit is aimed at the assessment of input parameters of the process, conditions
and supposed context of its execution, anticipated and projected values of the
results of the process performance. Thus, initial conditions, requirements and
parameters of the process execution may be subjected to audit before the beginning
of any formal procedures of the audit of execution, which makes it possible to
prevent illegal actions and eliminate negligence and mistakes at the stage of
preparation for the process execution. Besides, ex-ante audit may be implemented
both in respect of the entire procurement process and in respect of each of its
elements (subprocesses). The audit of a financial transaction on transfer of
budgetary funds before the time of execution of this transaction can be given as an
example. Documents are analyzed in terms of its correctness and compliance with
principles, laws, regulations, standards and objectives before final administrators
undertake any actions. Usually , the payment cannot be made until the auditor
confirms the correctness of the transaction after having thoroughly studied all
related documents and conditions.
Current audit concerns control over the process of execution and assessment
of preliminary results. Its principal goal is to monitor the main process, track the
administrators’ actions and supervise the progress of the fulfillment of the terms of
the public procurement contract. The outcome here is the obtainment of complete
and timely information allowing the efficient administration of resource and
financial flows, as well as the regulation of deviation from the normal flow of the
process, in order to achieve the result set forth in the public procurement contract.
Nowadays, audits over complex and long life projects, such as big
construction investments, public private partnerships and concessions, are
increasingly being conducted in early stages of those projects, either before they
are launched or during their implementation. These ex-ante or current audits (also
referred as real time audits) often include compliance and performance aspects. In
fact, they can look into adherence with prescribed applicable criteria or into, for
example, reliability of prospective studies, cost-benefit analysis and envisaged
benefits.

42
 Ex-post audit is aimed mainly at the assessment of the outcomes. The main
difference from ex-ante and current audits is that it is aimed at the output
parameters of the process and, as a rule, there is no chance to make any corrections
in the outcome. The auditor acts here as an independent monitor, while in the
frame of current and, especially, ex-ante audit he/she increasingly acts as a
participant in the process with the ability to influence the nature and course of its
flow. In a number of jurisdictions, the latter manifests itself in the fact that in the
frame of ex-ante audit, auditors are given broad powers to prevent the
implementation of operations and to veto procedures of the public contract
execution.
The main result of ex-post audit is the identification of the real situation and
the assessment of the outcome. This, as a rule, ”highlights the responsibility of
those accountable; it may lead to compensation for the damage caused and may
prevent breaches from recurring”6,.
Ex-post audit currently stands as the main approach implemented in the frame
of various jurisdictions. Specifically, in many developing countries the
introduction of flexible management schemes and enhancement of the powers of
decision-making as a result of public management reforms has led to the shift from
ex-ante to ex-post audit, which had an impact on the activities of both external and
internal systems of public financial control. While public management more and
more focuses on the assessment of financial results and efficiency of the budget
process administration, ex-post audit technologies and methods focusing on the
assessment of the outcome assume greater importance.

Financial audit, compliance audit and performance audit

Besides the distinction between the forms of audit pertaining to the time of
the process, there are other distinct types of audit, including financial audit;
compliance audit and performance/value-for-money audit.

6
С. 2.3. of the Lima Declaration

43
 Financial audit focuses on determining whether an entity’s financial
information is presented in accordance with the applicable financial reporting and
regulatory framework. This is accomplished by obtaining sufficient and
appropriate audit evidence to enable the auditor to express an opinion as to whether
the financial information is free from material misstatement due to fraud or error.
Financial audit are implementing accordance with conditions of ISSAI 1000
«General Introduction to the INTOSAI Financial Audit Guidelines» and others
conditions of ISSAI 1000-2999.

Compliance audit focuses on whether a particular subject matter complies

with authorities identified as criteria. Compliance auditing is performed by
assessing whether activities, financial transactions and information are, in all
material respects, in compliance with the authorities which govern the audited
entity. These authorities may include rules, laws and regulations, budgetary
resolutions, policy, established codes, agreed terms or the general principles
governing sound public-sector financial management and the conduct of public
officials. In the field of budget process and, in particular, public procurement,
compliance audit is carried out, first of all, in the following areas:
 at the stage of a budget commitment occurrence;
 when goods (products) have been delivered and services have been
provided;
 before making payments
 after the final payment has been made.
Compliance control confirms the legality and correctness of operations
performed, and sometimes confirms that main goals have been achieved, and
services, works or goods have been provided, performed or delivered at a high-
quality level.
Compliance audit are implementing accordance with conditions of ISSAI
4000-4999 General Auditing Guidelines on Compliance Audit, updated version
2016.

44
 Performance audit focuses on whether interventions, programmes and
institutions are performing in accordance with the principles of economy,
efficiency and effectiveness and whether there is room for improvement.
Performance is examined against suitable criteria, and the causes of deviations
from those criteria or other problems are analysed. The aim is to answer key audit
questions and to provide recommendations for improvement.
The principle of performance audit is defined through the fundamental
metodological concept of “value for money”, that is built on the correlation of
costs incurred and results achieved and is often associated with the tree component
elements (3Es):
 Economy - assessment of the economy is associated, first and
foremost, with the amount of input expenses. This means that the following
questions are important: whether expenses are made at proper (competitive) prices;
which main factors (drivers) of the change in cost are; which methods of economy
exist, etc. The principle of VfM implies not the cut in expenses, but their
reasonable undertaking. Accordingly, proper specification and substantiated
planning on input financial and material flows is very important;
 Efficiency is defined, first of all, as the ratio of costs incurred to the
result obtained. Actually, the assessment of how costs incurred are transformed
into outcomes is made through performance indicators. The most wide-spread
method to assess efficiency is to determine the indicators of returns and
profitability;
 Effectiveness7 shows how the results obtained correspond to goals set,
established criteria and desired results. To assess effectiveness, the results to be
assessed should correspond to established standards, criteria values, comparable
values, etc. That is, the assessment scale should be pre-defined and the rules of its
application should be established.

From the materials: Value for Money and Concern. – Corncern WorldWide,

7
As an extension of the above threefold scheme the extended scheme is sometimes used, which includes an
additional element: Cost-Effectiveness, Ce.

45
 November 20118
Various aspects of each of the enlisted elements are shown below:

Economy To acquire input The general principle of Purchase at the

resources at proper procurement activity right price
prices regulation, which
provides that the best
price is obtained on
competitive markets

Efficiency Cost-benefit ratios The measurement of To take required

economic efficiency actions correctly
involves comparing two
and more options for
obtaining the final result

Effectiveness Achievement of goals Incurred costs and the To take correct

set (focus on results) value of the result should actions
be assessed on the basis
of return indicators
(including social results)

The conceptual representation of the VfM principle has lately expanded due to
involvement of additional elements. For instance, the Australian Council for International
Development – ACID is considering as a forth element another “e” – equity.

Economy
Effectiveness
Efficiency

Input Process Output Outcome Integrat

ed
effect

Cost efficiency

Objectivity of representation

8
URL: https://www.concern.net/sites/default/files/resource/2012/10/5849-concern_and_value_for_money.pdf

46
 Figure - General framework of implementation of the “value for money” analysis chain
(VfM-analysis)

Performance Audit is implementing in accordance with conditions of ISSAI

3000-3999 General Auditing Guidelines on Performance Audit, updated version
2016.

4.2 The choice of methodological and information support of audit

4.2.1 Methodological support of Public Procurement Audit

Objectives of methodological support

Methodological support of Public Procurement Audit should contribute to
the overall improvement of the audit process efficiency. Planning of an audit
should involve special techniques that optimize the processes of performance of
principal audit procedures, diminish the risk of detection of material violation.
Proper planning of an audit helps ensure that the main areas of public procurement
are given due attention, and that all significant potential problems will be
identified.
Structure of methodological support
 INTOSAI methodological documents;
 methodological documents developed by SAIs and defining basic audit
principles, types of audit, both for general and special application;
 special audit tools and procedures specific for the given subject area -
public procurement.

The types of audit may be subdivided into “audit of method” and “audit
of subject”
During an audit, the method used is applied to a certain audit subject.
However, while examining the practice of application of various types of audit that
differ by audit methods applied in various subject areas, it is possible to note
whether the audit method or audit subject is the characteristic feature of the type of

47
audit under consideration. On this basis, we propose to subdivide types of audit
into the “audit of method” and “audit of subject”.
The “audit of method” in INTOSAI may include financial audit, compliance
audit and performance audit, since they, as is well known, differ both in terms of
examined aspects of the use of budgetary funds - financial rules, regulations
compliance and the correlation between outcome and costs, and in terms of
methods of accomplishment of audit engagements.
The standards of the “audit of subject”, which have been developed
pertaining to a specific subject area, should be based on the use of “method
audits”, while it is possible to use special procedures specific for the given subject
area but belonging to this or that principal method.
Public Procurement Audit refers to the “subject audit”.

Methodological support of Public Procurement Audit - variants of approaches of

the members of the INTOSAI Task Force

INTOSAI Methodological documents used in the planning of audit

ISSAI 100 - Fundamental Principles of Public-Sector Auditing

ISSAI 200 - Fundamental Principles of Financial Auditing
ISSAI 300 - Fundamental Principles of Performance Auditing
ISSAI 400 - Fundamental Principles of Compliance auditing

ISSAI 1265 – Communicating Deficiencies in Internal Control to Those Charged

with Governance and Management
ISSAI 1300 – Planning an Audit of Financial Statements$
ISSAI 1320 – Materiality in Planning and Performing an Audit

Methodological guidelines are used depending on the type of audit - financial audit,
compliance audit or performance audit

Methodological approaches should be focused on the achievement of results. The

approach focused on results should mean whether the contract has been performed
in accordance with tender documents, especially with the draft contract

Conventional risk assessment, which includes the understanding of the audited

entity, its environment and internal control system, should be conducted in order to
identify fraud risk factors and specific risk zones. The following standard working
documents are particularly important to identify and assess the risk of procurement:

48
  General understanding at the level of financial statements
 Agenda of meetings
 Understanding the internal control system of an organization associated with
financial statements
 Understanding business processes and relevant controls
 Analytical procedures - procedures of risk assessment
Working materials on procurement presented in the frame of horizontal audit
approach also include the following control charts and procedures to identify general
risks and specific decisions on awarding (contracts or quotations) where
characteristics of risks manifest themselves:
• Testing of structure and implementation of controls of SCM (Supply Chain
Management) (including the assessment of the SCM policy)
• Identification of possible fictitious suppliers and follow-up work with them
• Identification of the interest in suppliers (conflict of interest) and subsequent
work therewith
• Generalization of claims of irregularities made during the audit
These procedures of risk assessment will also identify risks that might have
influenced financial audit - procedures should be performed at the earliest possible
stage in the audit process.
Methodological approaches should be focused on the achievement of results. The
approach focused on results should mean whether the contract has been performed
in accordance with tender documents, especially with the draft contract.

4.2.2. Information support of Public Procurement Audit

The problem of information and analytical support in this sphere is

becoming increasingly important and practically significant, since information
technologies are widely used in all national contract systems. Methods capable of
detecting implicit (hidden) information, replenishing, compensating for the missing
information, generating new data, eliminating uncertainty and risk seem to be
especially relevant.
New technologies in the area of audit are largely based on modern
information technologies. In the absence of appropriate information and analytical
support, the conduct of high-quality audit, especially in procurement, turns out to

49
be challenging. The absence or lack of the use of information results in improper or
incorrect choice of audit methods, techniques and tools.

The System of Information and Analytical Support directly related to

methodological support of audit includes:
 the total of information resources and mechanisms for access to them;
 analytical techniques and measurements used in Public Procurement
Audit;
 formats of exchange and provision of information.
Analytical component gives auditors required tools that allow them to
improve the efficiency and quality of audit and address analytical tasks such as:
 analysis and identification of regularities and trends;
 enhancement of substantiation of conclusions;
 analysis of implementation of plans with due regard to the compliance of
regulations;
 determination of economic efficiency of the use of resources;
 forecast of results;
 examination and assessment of risks;
 comparative analysis of large volumes of information.

A comprehensive approach to Public Procurement Audit aimed at the

adaptation of information and analytical support of the public contract award
procedure to modern requirements allows enhancing the overall management
capacity in the public sector, thus providing necessary levers to increase efficiency
of the use of budgetary funds.
Due to the complexity, diversity and integrated nature of ongoing
procurement and the processes of its management, it is practically impossible and
inexpedient to create a full and comprehensive list of information sources used in
Public Procurement Audit, but it is rather appropriate to consider the general list of
types of information and principal, most widely used information sources:

50
 Types of information

General regulatory information Statutory acts, legislative solutions, binding

decisions of authorities, etc.

Internal regulatory information of SAIs. Internal rules, regulations, instructions,

standards, orders, ordinances, etc.

Documents of budgetary (financial) accounting Budgetary reporting, registers of budgetary

accounting, data of budgetary accounts,
primary documents: financial, receiving,
expenditure documents, etc.

Information from an official information base Registers of awarded public contracts,

of procurement system and internal accounting information on each contract, including the
system of the public contracts of the entity terms of reference, cost estimates and other
substantiations of public procurement,
certificates of completion of public contracts,
etc.

Information of the procurement participants Information on the possible affiliation of the

not included into the official register procurement participants, involvement in
dubious financial schemes

Statistical, financial, tax, customs information, Data of appropriate Government Information

the data of registration bodies Systems

Other information Any other information used in the frame of

public procurement: analytical reviews,
estimates, comparative data, results of
processing of questionnaires and working
interviews (surveys), materials of internal and
external control, direct observations of the
audit team members, etc.

Information sources

Internal Information Sources - information databases formed in the frame

of Supreme Audit Institutions to support the audit; their intended users are
employees of SAIs.
External Information Sources – information databases of all types, both
with public access and departmental ones, which may be used to support the audit
in order to obtain various audit evidence.

51
 The reference to this or that information source as part of a separate audit
engagement is performed in accordance with the requests of the audit team, which
are formed in the frame of implementation of the audit program and are determined
primarily by the internal needs in obtaining various kinds of audit evidence.
The system representation of the information support process of Public
Procurement Audit allows considering all its variety in a form of a generalized
conceptual model determining the interrelation and correlation of external and
internal information sources (see figure)

52
 Information Systems on
Contractors of Public National Information System
Procurement of Public Procurement

Other Mass Media

Procurement contractor
entity’s AUDITE
PROCUREMENT D
OTHER ENTITY
PROCUREMENTS

Public information Internal information sources of a Supreme Audit Institution

systems (financial
system, taxes, General Information resources formed in the Best audit practices
customs, statistics, Information process and in the interests of a
registration, etc.) Resources specific audit engagement

Figure - External and internal sources of information for the Public Procurement Audit

General 53
Information
Resources
 The formation of an information resource to support an audit

The information basis for the support of Public Procurement Audit should
consist of appropriate automated information databases on procurement, bases on
identified violations, suppliers and so-called “black lists”, which contain
information on unreliable suppliers and their related parties. The creation of
information resources of Public Procurement Audit should be organized in a SAI
and provide auditors with necessary information at the stages of:

 planning of an audit in the area of public procurement;

 preparation for implementation an audit in the area of public procurement
in accordance with the audit engagement;
 implementation an audit in the area of public procurement in accordance
with the audit engagement;
 documentation of the findings of the audit performed;
 control of implementation of proposals and recommendations based on
the audit results.

All specified databases should be accessible and convenient for the use by
auditors and supervising bodies, providing an automatic accomplishment of all
tasks of the audit with the use of appropriate information and analytical systems
and technologies.
Besides, depending on the method of formation and the procedure of
creation of information requests in Public Procurement Audit, it is reasonable to
classify available internal information resources of a SAI in three main groups:
general information resources, information resources created in the process and in
the interests of implementation of a specific audit engagement and best audit
practices.
 Internal Information Resources of SAIs

1. General Information Resources

In the frame of control and expert-analytical activities, auditors should have
the opportunity to use specialized information resources and services created in
accordance with national laws on procurement, as well as resources and services
containing bases of payment documents and providing access to such external
information resources as: databases of government statistical bodies, customs
bodies, financial bodies, banks, through which transactions with public funds are
effected. Legislative, planning and regulatory, and background information also
holds a prominent place in creation of an audit information database.
2. Information resources formed in the process and in the interests of
implementation of a specific audit engagement
These resources include information received as a result of inquiries of
Supreme Audit Institutions regarding the given activity, as well as information
created by the audit participants in the frame of their activities during the audit in
respect of specific control or expert-analytical activity, including economic
information characterizing production and business activities of audited entities, as
well as non-financial reporting.
Part of this information may be transferred to the general information
resource after the completion of the audit.
3. Best audit practices
It is necessary to form and constantly replenish the relevant information
resource for joint use, which is an independent subsystem of the general
knowledge base on the best practices of public procurement audit.
The project of such a resource is being implemented in the frame of
EUROSAI to implement the project of IT audit introduction (project CUBE). The
information resource “Best Practices of Public Procurement Audit” may be used
not only by the Task Group members, but also by all INTOSAI members.

55
 The need to create such resource is prompted by the fact that the problems
related to the efficiency of the use of public funds generally exist and manifest
themselves in a large variety of forms, have rather complex causal relationships.
For example, the performance audit techniques may vary depending on its
scope, specific activities of the audited entity and the objectives of a particular
performance audit.
It is proposed to include in this section the so-called best audit practices –
those results of control and expert-analytical activities that are presented in reports
and opinions of Supreme Audit Institutions of INTOSAI and that could be
examples of audit in this area.
Besides, it is necessary to elaborate the issues associated with the choice of
best practice examples, their structure and development of standardized modules to
ensure the most accurate and complete search in accordance with the requests
made.
The following is the Structure of presenting information on the practice of
Public Procurement Audit9.

A format of presenting information on the public procurement audit results

1. Conditions of audit:
1.1 name and time frame of the audit;
1.2 time period chosen to evaluate the activities of the audited entity;
1.3 time limits for putting control, expert and analytical measures in
place;
1.4 composition of those participating in the audit;
1.5 substantiation of the audit and authority of its participants;

2. The audit entities and domains:

2.1 audited entities;
2.2 domains (scope) of the audit and the authority of those participating
9
This structure was presented and approved at the 2nd meeting of the INTOSAI Task Force on Public Procurement
Audit (Moscow, June 24, 2015)

56
A format of presenting information on the public procurement audit results

in the measure to check them

3. Data used in the audit:

3.1 information used to carry out the audit;
3.2 information sources;
3.3 indicators provided by the audited entity;
3.4 indicators used while conducting the audit

4. Methodology used:
4.1 types and forms of audit used;
4.2 international and national standards and guidelines used;
4.3 methodological techniques developed for the particular audit

5. The tools used to make measurements and evaluations:

5.1 the scales and criteria selected for making measurements and
evaluations;
5.2 data processing methods used;
5.3 information technologies used while handling the information (data)

6. Audit results:
6.1 flowcharts of the activity of the audited entities;
6.2 results of data processing and evaluations for each type and form of
the audit conducted and the audit domain;
6.3 risks evaluated and deficiencies uncovered, including their costs
assessment;
6.4 visualization of results (if any)

7. Findings and recommendations:

7.1 auditor’s opinion - executive summary;
7.2 response of the audited entities;
7.3 corrective actions advised;
7.4 results of maintenance of the corrective action process;

57
A format of presenting information on the public procurement audit results

7.5 final result and recommendations on follow-up actions

As for the proposals developed in respect of the format and structure of

information presentation based on the findings of Public Procurement Audit for the
benefit of the “best audit practices” section, it was proposed to pilot them on the
basis of examples presented by SAIs, including member-countries of the INTOSAI
Task Force, when they submitted the results of pilot projects.
The results of the pilot projects of the SAIs of Portugal and Zambia
presented according to the proposed structure can be found in the Annex 5.

Information support of Public Procurement Audit - variants of approaches in

SAIs of countries
Belarus
- national laws on public procurement;
- annual plans of public procurement placed in the Internet on the official site of
the Information Republican Unitary Enterprise – “National Center of Marketing
and Price Trends” (www.icetrade.by);
- the results of electronic auctions placed on the site www.icetrade.by and
electronic trade platform - “Belarus Universal Commodity Exchange”
(http://zakupki.butb. by);
- operation data on the activities of a business entity;
- financial documentation of an audited entity;
- information on the cost of goods requested during the audit from organizations
that have earlier supplied similar goods, manufacturers, dealers, buyers of similar
goods, organizations that have carried out central procurement, the National
Center of Marketing and Price Trends.

Kyrgyzstan
National laws on public procurement
Relevant laws and regulations (Law “On Public Procurement”, decrees of the
Government of the Kyrgyz Republic, etc.)
The database of the official portal on the public procurement of Kyrgyz Republic
containing for at least the annual plans of public procurement and contracts
with suppliers on supply and execution of works, as well as law regulations and
procurement procedures manuals.
Various online resources

58
China
Material Information, which is regarded as the major component of the
information support of Public Procurement Audit, provides the convincing
evidence in the analysis part for auditors.
Published government documents: the guideline and regulation of government
procurement practice.
Certified project files: provide the basic information of government procurement
practice.
Inquiry record: evidence based on the interview with the responsible persons.
Digital database: consists of all the possible fundamental data and can be analysed
through IT technology.
Immaterial Information, which is usually ignored due to the fact that it usually
obscure useful information. However, it may provide new perspectives to
investigate and analyze government procurement.
On-line information: the unverified information which could be taken into
consideration

Latvia
The structure of the information support of Public Procurement Audit.
Data bases of informational resources in the area of procurement.
The list and characteristics of principal information sources for the audit process
support
For instance, the Procurement Monitoring Office publishes information on
announced procurements and results of the procurement procedure; auditee
information register on the signed contracts and the fulfilment of contracts.

Portugal
The main information sources for the Public Procurement Audit we perform are:
 Applicable legislation and general standards and guidance
 The written procedures and regulations on procurement approved or followed
by the concerned body
 Procurement procedure (s) file, including decision to open procurement, public
announcement, documents defining needs and rules for procurement, tenders
and bids presented, reports on evaluation and choice of proposals and award
decision
 Contract(s) file, including approval of the terms of contract, documents about
the requisites and abilities of contractors, offered guarantees and any possible
complementary documents to the contract
 Documents about the execution and review of works and related payments
 Previous internal and external audit reports
 Any internal or external documents concerning the project or contract, namely
those mentioning irregular practices or officials misconduct

59
  Media information concerning the subject or the object of the audit
 SAI’s database
Answers to surveys, interviews or SAI’s requests

South Africa

1. Overall understanding at financial statement level

This work paper is used to gather and record key information which can identify
fraud indicators but also provides an understanding of the procurement
environment of the auditee. Of particular importance are the sections on:
 Policies and procedures
 Use of service organisation
 Vacant key positions
 Key suppliers
 Regulatory environment
 External factors, reports and media coverage
 Investigations, litigations and claims,
 The auditee’s fraud risk assessment and response and information on
alleged and confirmed fraud.
2. Agendas for meetings
This working paper indicates the matters that are required to be discussed with
management, internal audit and those charged with governance – it includes
discussions on management’s assessment of fraud risk and their response thereto.
3. Understanding the entity's internal control relevant to financial reporting.
This working paper documents the auditor’s understanding of the entity’s internal
control for each of the 5 components of an effective system of internal control. The
understanding obtained will at an overall level identify whether the control
environment is strong enough to prevent and detect fraud and non-compliance.
4. Understanding of business processes and relevant controls
This working paper documents the auditor’s understanding of the auditee’s business
processes, controls in the processes and risks identified based on weaknesses in the
design and implementation of the controls. The business process and relevant controls
for procurement whether through quotations or competitive bidding should be
documented and the risks identified in order to obtain an understanding of the
weaknesses in the processes. Similarly an understanding should also be obtained of the
contract management processes.
5. Analytical procedures – risk assessment procedures
The working paper can be used very effectively to identify trends or anomalies that are
indicative of possible fraud.

Slovenia

60
 Information support of Public Procurement Auditmust contain the information
about tender dossier, the documentation of the procurement procedure and the
stipulated contract with modifications.
The list and characteristics of principal information sources for the audit process support
The main sources of information are contracting authorities, a publication of the contract
notice, which is published either by the Publications Office of the European Union or by
the contracting authorities on their buyer profile, a contract award notice, which is
published not later than 30 days after the conclusion of a contract or of a framework
agreement following the decision to award or conclude it as a result of the procurement
procedure, and the notice of the content of the contract which, according to the Slovenian
law, also has to be published in the Official Gazette of the Republic of Slovenia.

Zambia

WHAT INFORMATION SOURCES/ WHERE CAN IT BE FOUND?

IS NEEDED? DOCUMENTS
Procurement structure and Entity’s Procuring Entity/audit entity
positions Organisational
Structure
 Procurement Circulars or Procuring Entity/audit entity
thresholds pronouncements Procurement regulatory body
 Procurement issued by the
authorisation and Procurement
levels of authority regulatory body
Procurement
regulations or Acts

Procurement methods Acts or regulations  Procuring Entity/audit entity

Budget statements  Procurement Regulatory body
Entity’s
procurement policy

Procurements made during Procurement Plan

the period and nature of and procurement
procurements files

Stakeholder surveys, interviews,

Expectations on SAI’s conference reports, minutes of
reports from Parliament meetings
and key stakeholders Donor agreements
Mandatory audit Legal framework Public Audit Act and other Acts and
responsibilities of the SAI for the SAI and Regulations, for example Public
leading financial Finance Act and Regulations
Reporting timeframes for regulations
SAI and for auditees
Strategic directives for the Strategic plan Procuring Entity/audit entity
procuring entity

planned budgeted Audit entity Procuring Entity/audit entity

61
 WHAT INFORMATION SOURCES/ WHERE CAN IT BE FOUND?
IS NEEDED? DOCUMENTS
expenditure of the entity budgets
Relevant internal policies Internal policies of Procuring Entity/audit entity
may include a the audit entity
procurement manual
Actual performance Reports and Procurement performance/progress
against targets (for feedback from reports
example number of procurement
procurements finalized on department
time against what was
planned)
Problems experienced
Suggestions and ideas
Constraints
Capacity of audit staff
 Level of Summary of performance assessments
compliance with assurance reviews for
manuals, standards Performance
and guidelines of assessments Assurance unit
the Procuring or reports done by Procurement Regulatory Body
entity the procurement
 Capacity of regulatory body
procurement staff
Risk on a national level Auditor’s reports Audit business units
for the prior year Audit files

Internal auditor’s
reports

Information of the Contact Committee of the Supreme Audit Institutions (SAI) of the
European Union
PUBLIC PROCUREMENT AUDIT Lisboa 2010 (See Annex 6)
Information Sources on Public Procurement
Guidelines and Directives
EU Directive 2014/24/UE covers the procurement of public contracts. Directive
2014/25/UE covers the procurement of entities operating in the utilities sector.
Directive 2014/23/UE covers the procurement of concession contracts. These
Directives were published in OJEU of 28 March 2014 and are available on
http://eur-lex.europa.eu/en/index.htm.
General information about public procurement can be found at the following
website: http://ec.europa.eu/internal_market/publicprocurement/index_en.htm.
Official Journal of the EU
Online publication of notices is available on http:/simap.eu.int
Other relevant websites
· EU Public Procurement website : http://simap.europa.eu

62
 · General EU website: http://europa.eu
· WTO site on the 1994 Government Procurement Agreement
(GPA):http://www.wto.org/english/tratop_e/gproc_e/gp_gpa_e.htm
· Court of Justice website: http://www.curia.europa.eu

4.3 Public Procurement Audit in the context of application of

modern Information Technologies

This section is dedicated to the using of Information Technologies for the

benefit of improvement of the audit quality.
1. Big data in Information Technologies is a series of approaches, tools and
methods of processing of huge amount and large variety of structured and non-
structured data, the use of which makes it possible to obtain fundamentally new
results by taking into account multi-dimensional links.
Big data is information that is characterized by a complex structure and huge
volume in excess of the traditional capabilities of IT infrastructure, collected from
all possible sources, which allows its advantageous use as one of major assets.
It includes
 Structured / non-structured data
 Internal / external data
 Official and non-official communication channels Big data
2. Data mining - it is proposed to be used as a tool to assess fraud schemes in
procurement
3. Audit Management System (AMS) is proposed for improvement of the
audit process efficiency, employs principles and approaches of project
management.

Use of Information Technologies for the benefit of improvement of the audit

quality - variants of approaches in SAIs of countries

Belarus
Taking into account the fact that competitive procurement procedures are carried
out on electronic trade platforms, data from the Internet are used in the frame of
the audit of public procurement issues.
CNAO

63
On-line Auditing (OLA) construct a safe and efficient information -exchange
-platform for auditors .
Big data analytic such as .comparative analysis, cloud computing, Structured Query
language.

64
5 ORGANIZATION AND IMPLEMENTATION OF PUBLIC
PROCUREMENT AUDIT

5.1 Planning of an audit and preparation of information requests

When planning Public Procurement Audit, it is necessary to strictly comply

with requirements of international audit standards regarding the preparation of the
audit plan and program, risk assessment and determination of the materiality
degree. Besides, the audit should be planned in such a way that guarantees high
quality auditing, i.e. that the audit engagement would be performed on time, cost-
effectively and with professional competence.
The plan of the audit should be formed in such a way that its exact
implementation would ensure proper and efficient organization of work for each
audit engagement.
It is expedient to carry out the process of Public Procurement Audit planning
at three levels:
 strategic planning;
 macro planning;
 micro planning.
At the first, or strategic, level, a long-term strategic plan is implemented, the
contours of a common vision of goals and objectives for future engagements in
Public Procurement Audit are defined. At this stage, the general list of audited
entities for the period of about 3-5 years is shaped.
This is a long-term plan, which, due to its nature, should cover all audited
entities providing the solution of such issues as:
 long-term goals and objectives of the audit activities in public procurement;
 re-orientation of applicable audit techniques and methods in accordance
with requirements of changing external environment;
 preparation of staffing and resource support of Public Procurement Audit
 assessment of the need in training and new personnel.

65
 Strategic plans should be flexible and generalized enough in order to always
have the opportunity to review priorities and adjust previous plans.
The availability of this plan ensures system sustainability and necessary
adaptivity of the entire public-sector audit in the area of public procurement, which
is focused primarily on the most effective and complete coverage of public
procurement.
Evidently, such strategic plan should be sliding by nature, focusing on the
targets of the overall macroeconomic policy. Accordingly, it should be adjusted
and updated each two or three years to elaborate general national policy in the area
of public procurement.
Масro level is implemented in the goals of annual working plans. This is a
medium-term plan that translates strategic vision into the program of work for the
next year. It defines goals and objectives of audits throughout the year taking into
account available resources.
At this level, planning is an aggregate presentation of future activities of audit
teams and is aimed largely at building the audit engagement portfolio. The
document compiled at this level may be considered as a preliminary planning
schedule of audit throughout the specified period.
The annual audit program should be drawn up by the beginning of a new
budgetary period and should take into account the following issues:
1. The number of entities covered by the audit, the scope and specificity of
public procurement carried out by them.
2. Frequency of these or those procurement activities and the depth of their
coverage by audit procedures.
Thus, at the planning stage, main priorities should be determined and final
objectives of Public Procurement Audit should be defined. This will make it
possible to properly distribute human and material resources to carry out regular
audits and ensure necessary consistency and completeness of the coverage by audit
of the entire planned audit area.

66
 Technically, various techniques and methods may be used for this purpose,
starting from the compilation of spreadsheets to modern methods of network
planning.
The first stage of planning allows shaping up quite a full and clear picture of
future audit engagements, which makes it possible, in particular, to properly
organizing personnel management, including professional training, necessary
rotation of auditors in performing audit engagements, as well as to avoid wasting
time on overtime and unscheduled activities. This also allows determining the
budgets for the implementation of the annual plan of audit engagements, including
travel, overtime, transportation and other expenses involved in audit.
The annual plan is a document, the purpose of which is:
1. To build up the general portfolio of audit engagements for Public
Procurement Audit
2. To assess the volume of expected and actual funding necessary for the
implementation of planned audit engagements for Public Procurement Audit
3. To assess staffing and material resources necessary for the implementation
of audit engagements for Public Procurement Audit;
4. To analyze and assess the risks of non-performance or incomplete
performance of audit engagements;
5. To plan the time limits and periods of performing audit engagements for
Public Procurement Audit;
6. To plan the distribution of responsible persons throughout the entire
portfolio of audit engagements for Public Procurement Audit.
The annual plan may be adjusted, updated and even change during a year.
Therefore, it is expedient to draw up a quarterly updated plan of audits in the same
format as was used for the annual plan. Adjustments in the quarterly plan may take
place if any significant circumstances or new audit engagements occur.
Micro level suggests a detailed explanation of the plan of a particular audit
engagement with a detailed description of actions, specification of audit
procedures to be applied and assessment of the time required for the audit.

67
 Auditor should be guided by the ISSAI 1300 « Planning an Audit of Financial
Statements» and other standards in planning of effective performance of audit
engagement.
Actually, this is an operating plan for a particular audit engagement with a
detailed specification of objectives to be achieved in the frame thereof. This is
where the main technical details of the future audit are determined and the issues
of operative planning of the audit team activities are addressed.
Thus, the following considerations should be taken into account in final
planning:
 risk level, degree of materiality, priorities of efficiency assessment at each
stage of public procurement;
 degree of materiality;
 information of the problems identified in the past or expected in the future;
 available procedures of public procurement;
 information of any irregular circumstances, events, external and internal
factors able to influence the outcomes of the activities of the audited entity in the
area of public procurement;
 technologies and technical solutions applied in public procurement;
 technical, regulatory, economic, ecological and other changes able to
influence the activities of the audited entity;
 staffing requirements and distribution of working hours among the audit
team members, etc.
All plans are not static, but they should be regularly reviewed based on the
changes in the context of the audit and risk assessment.
After the risk assessment has been completed and the degree of materiality
has been determined for each objective set in the frame of the audit engagement, a
detailed audit program is drawn up taking into account all identified aspects of the
audit, including the impact of risk factors, specific features of the internal control
system operation, types and volumes of public procurement, and existing
technologies of implementation thereof.
68
 The final plan of the audit will include the following headings:
1. Name of the auditee and the period covered by the audit;
2. List of objectives and issues to be addressed during the audit (including
those that may influence the procurement);
3. Review of all possible risks (for each audit objective);
4. List of tasks for the audit team with specification of the time of beginning
and deadlines.

Planning of an audit and preparation of information requests

- variants of approaches in SAIs of countries

Belarus
Instruments used for planning of an audit
As a rule, the issues of meeting legislative requirements on public procurement are
checked along with other issues of financial and business activity of a business
entity if budgetary funds are allocated to and used by the audited entity, which is
taken into consideration at the stage of the audit planning. That is, with some
minor exceptions (specific assignments to check the legislation on procurement),
Public Procurement Audit is not the main theme of the audit.
Information requests, as a rule, are not sent to audited entities. The issues of
compliance with laws on public procurement are controlled directly during the
audit.
The exceptions are made where violations in the area of public procurement are
reported in appeals submitted to the State Control Committee from the participants
of such procurements. In such cases, depending on particular situation, requests to
provide the customer with additional information could be sent.

Kyrgyzstan
Instruments used for planning of an audit
Getting acquainted with materials, interviewing
Examples (samples) of plans and programs of Public Procurement Audit

Program
of the audit of the system of procurement of goods, works and services in the Ministry of
Education and Science (Headquarters).
I. The ground for conducting the audit: Order of the Accounts Chamber of the
Kyrgyz Republic No. __________ dated _____________. Working Plan of the Accounts
Chamber of the Kyrgyz Republic for the year ____
II. Audited entities: Ministry of Education and Science (Headquarters).
III. Audit objective: Audit of the system of procurement of goods, works and services.
IV. Audited period:
V. Audit questions:
How can you briefly characterize the public procurement system of the audited entity

69
 (organization of the procurement system in the audited entity, internal regulations,
volumes and types of procurement).
Analysis of the justification of procurements accomplished
Analysis of the regulatory framework governing the expediency of procurement of the
given type of goods, works and services.
Compliance with laws on public procurement:
availability, analysis of development and implementation of the annual plan of public
procurement;
analysis of compliance of the chosen procurement methods with requirements of efficient
use of allocated funds;
analysis of validity and quality of technical specifications for the purchased objects and
relevant requirements to potential suppliers of goods, works and services;
availability, validity and compliance of tender and documents of participants with legal
requirements.
check of documentary confirmation of the conducted procurements under the signed
contracts
Documentation of audit results.

China
Preliminary investigation: acquiring general knowledge of the business, such as
general economic factors, important characteristics of the entity, and etc.
Internal control assessment: ascertaining the validity and effectiveness of
accounting and internal control systems.
Risk assessment: making reference to inherent risk, control risk, detection risk
Flowchart: providing steps in the audit process as well as defining deadlines.
2. Examples of information requests to auditees
Annual budget and final accounts of central ministries and local government
departments
Government procurement files include government procurement plan, procurement
contracts, public bidding documents, etc.
Procurement information management system (Government procurement
information, summary list, etc.)
Related accounting voucher, electronic data, etc.
Government procurement laws, regulations and other provisions.
3. Samples of plans and programs of Public Procurement Audit
Ex ante survey: understand audit requirements, choose subject matter
Evaluate the audit resources available in audit institutions
Design Audit plan (for an audit engagement by the audit team)

Latvia
How does the identification of necessity of a service for a certain branch takes place?
Are the goods purchased through utilization of procurement procedure?
Does the procurement take place in compliance with the legal regulation?
In the year (xxxx) has there been any case when goods are delivered from a supplier
without signing of a contract and without announcement of procurement in accordance
with the Law on public procurement?
What is the total value of the delivery from suppliers with whom no contract has been
signed as a result of the procurement procedure?
Does the quantity and price of goods/services match the ones mentioned in the signed

70
contract (as well the price per unit)?
Are the expenditures for repair materials properly accounted including the proper
value and accounts?

Portugal
Example of objectives defined in audit plans:
 Assess whether the process of estimating the amount/value of the contract works
was accurate and in line with the market conditions;
 Check whether the works performed and the materials applied complied with the
requirements set forth in the specifications of the contract
 Verify whether the additional works authorised and conducted complied with the
applicable legal provisions, namely:
 If they were strictly necessary for the conclusion of the contracted work
 If they resulted from an acceptable error or from unforeseeable circumstances
 If they respect mandatory quantitative limits

 Check whether the execution of additional works to the initial contract,

conducted by the original contractor, resulted in circumventing a new and due
competitive awarding procedure

 Verify if public financial regulations regarding expenditure were complied with

assess if modifications to the original contract were authorised by the staff
member with the necessary authority

 Find cases where the cost of additional works becomes excessive when
compared with similar works and circumstances occurred in other construction
projects

 Confirm that companies who act as site inspectors dully exercise their duties
and contractual responsibilities as to ensure that construction works are
executed according to the plans and specifications set by the public owner

 Highlight situations where the execution of additional works revealed

disproportionate and overpriced technical and construction solutions in view of
the scope of the project
 When justified, assess whether responsibilities were asked from those
responsible for projects` errors and deficiencies originating additional works

 Identify infringements of legal financial regulations and, in those cases, submit

all the necessary information to the jurisdictional chamber for judgement

Issue recommendations where applicable

Examples of documents requested to auditees to meet these objectives:

 Audit reports of the internal audit or other public and private audit entities
 The procurement process file, including the program and specification of works
and the implementation project

71
  Lists of errors and omissions identified by the competitors during the awarding
procedure
 The contractor’s bid and the enclosed documents
 Contract and documents of the work assignment
 Work plan and financial schedule and any respective changes
 The work books(according to legislation all the relevant events during the
works have to be recorded in these books), including the site diary, construction
progress meetings minutes or other written reports
 Information and any technical consultancy produced during the course of the
works
 Any notices of suspension and resumption of works as well as requests and
authorisations of time extensions for the completion of works
 Proposals and claims submitted by the contractor

Documents recording the measurement of works , invoices, orders, payment

authorisations and receipts checking/current account of the contractor

Slovenia and Zambia are in Annex 5

5.2 Audit risk in Public Procurement Audit

One of the paramount objectives of Public Procurement Audit is to achieve

required balance between expenses on risk management in public procurement and
possible consequences of occurrence of risk events. Successful solution of the said
problem implies that principal risks are controllable and are at an acceptable level.
Thus, the assessment of sustainability of procurement process should
obligatorily imply the analysis of risk management policy and processes in place in
the auditee. If the assessment shows a high risk level, there is a need for more
detailed audit procedures.
When planning an audit, a public-sector auditor is obliged to carry out an
assessment of audit risk. Besides, it is to be understood that high risk zones are
subject to audit on a mandatory basis, while the areas with medium and low risk
may become a subject of the audit on a sample basis. Among the factors
determining the increased risk level in the area of public procurement, the
following are the most important:

72
  volume and number of contracts awarded – the more the volume, the higher
the risk;
 amounts of financial resources allocated for public procurement – the
higher the figure, the higher the risk;
 relative complexity (and/or stiffness) of the procurement system, internal
controls and the degree of separation of duties: complex, too stiff or, vice versa,
too soft systems are exposed to high risk;
 availability of the conflict of interests or factors evidencing a high
probability or possibility of staff misconduct (availability of high level of interest,
probability of collusion, increased staff turnover, etc.)
Main activities that can be implemented in the frame of implementation of the
risk-oriented approach in the development of the audit plan involve:
 stock-taking of the procurement system in the audited entity;
 identification of critical components of the system, such as special funds,
end users, sole suppliers, etc.
 assessment of main risks affecting these components and analysis of the
gravity of such impact.
Risks affecting the procurement system should be taken into account at the
time of planning of the audit and may be considered as elements of inherited risk,
risk of controls and detection risk. Each of these elements directly influences the
scope of general audit risk, which may be defined as the risk that information on
public procurement might contain material errors that might remain unnoticed
during the audit.
Inherent risk is the risk that resources controlled by the procurement system
are compromised as a result of misappropriation of material values, destruction,
disclosure, unauthorized changes, etc., subject to the availability of appropriate
internal controls.
Control risk is the risk that an error which might occur in the audit and which
might be material alone or in combination with other errors will not be timely
prevented or detected and corrected by the internal control system.
73
 Detection risk is the risk that even in the frame of main audit procedures the
auditor will not identify errors and violations evaluated as material ones as taken
alone or in combination with other errors.
In assessment audit risks it should be guided by the ISSAI 1315 (Revised)
«Identifying and Assessing the Risks of Material Misstatement through
Understanding the Entity and Its Environment» and ISSAI 1330 «The Auditor’s
Responses to Assessed Risks»
Auditor should be counted for much the preliminary assessment of the
reliability of the internal control system. It might be implemented on the basis of
preliminary examination of available documentation or discussions with the
management of the audited entity. Auditors assess the level of control and
awareness of the management regarding possible violations and errors in the area
of procurement. This assessment also helps concretize and specify the objectives of
the ongoing audit, which should be determined prior to the beginning of the main
audit.
In order to mitigate the overall risk, key policies, procedures, practices and
organizational structures in the area of public procurement should be clearly
determined in the entity. The level of internal control determines the overall risk
level in the entity, since only the availability of an adequate internal control system
would minimize the risk of errors and abuse. If the internal control system is weak
and operates ineffectively, the number and scale of audit procedures virtually
increase and testing is implemented in more detail.
Elements of management that should be taken into account when assessing
the reliability of the internal control system may be classified as follows:

74
Preventive Aimed at the implementation of preventive measures and
identification of possible problems before they occur.

Their main purpose:

 to control operation and data input;
 to try to predict potential problems before they occur and make
adjustments;
 to exclude the possibility of errors (unconscious actions),
omissions (negligence) or conscious unscrupulous practices of
the staff

Identifying Aimed at identification of errors, omissions or fraud and reporting

about their occurrence should any signs of the their availability be
identified

Corrective Their main objective is to minimize potential threats, risks and

consequences of violations. They also exercise some more functions,
such as:
 elimination of problems identified in the frame of audit
procedures;
 identification of the causes of problems;
 correction of errors where problems occur;
 introduction of changes in data processing systems to minimize
possible threats.

On the basis of a preliminary assessment of control risk, the auditor should

determine principal approaches, techniques and methods that help the internal
control system operate efficiently. As a result of the ongoing testing, the auditor
obtains a substantiated confirmation of operability and reliability of the internal
control system.
It is reasonable to use materials of internal control, inspections or other
information regarding procurements completed as additional information during
the audit. A well prepared internal report allows for brief but meaningful comment
on the adequacy of the management and procurement system of the audited entity.
Besides, it may contain recommendations and remarks regarding the effectiveness,
efficiency and economy of procurement. In this case, the auditor should analyze
the correctness and justification of recommendations and proposals made and
include the clause on their implementation into the plan of the audit.
The main risk areas associated with low efficiency of the internal control
function could be characterized as follows:
75
  internal control reports are not submitted to top management or are
submitted with a long delay;
 recommendations and proposals of the internal control are ignored by the
management of the audited entity;
 internal control operates within the constraints of its powers in terms of
conducted inspections and audit procedures;
 no audits of procurement activities are implemented or they are carried out
in a formal way, etc.
When determining the level of materiality it should be guided by the ISSAI
1320 «Materiality in Planning and Performing an Audit».
The auditor should obligatorily assess the inherited risk and form an
informed opinion on the magnitude of the risk of internal controls. In this case, the
following rule should be taken as a basis: the higher the level of internal control
risk is the more audit evidence should the auditor collect, turning mainly to the
procedures on the merits.

Material was based on the Public Procurement Audit Manual. The Federal Democratic
Republic of Ethiopia. Public Procurement and Property Administration Agency

A common audit checklist to be used when auditing public procurement processes has been
prepared on the basis of common principles and procedures having regard to:
 Focusing on the robustness of the procurement function, meeting public needs,
ensuring value for money in using public funds, competition objectives and transparent
procedures;
 The Federal Government of Ethiopia's public bodies are bound to process
procurements in accordance with the Federal Government Procurement and Property
Administration Proclamation No. 649/2009 and Public Procurement Directive dated
June 8, 2010;
 Public bodies must respect the requirements of a competitive process and make their
decisions in a transparent way which respects all participants equally.
 Procurement is a risk area for fraud and corruption and they usually result in the
misuse of public resources.
While the checklist is general in nature it follows the requirements of the Procurement
Proclamation and Directive.
Checklist also addresses some relevant questions not included in the Procurement
Proclamation, e.g. organizational issues.
In addition, the emphasis has been placed on aspects that are prone to failure and irregular

76
influence.
When using these checklists, the auditor should keep in mind that:
 The evaluation of public procurement processes may be only a part of the audit (as in
the case of a financial audit), and, thus, the proposed questions may have to be
integrated within the broad methodology of that audit;
 Depending on assessed risks, not all questions will be applicable to each audit;
 According to audit mandates and national Federal Auditor General's systems and
procedures, some items may have to be modified or questions added. For instance,
financing through Federal Government budget, local budgets or international donors
grants or loans will put the public body under the obligation of following the relevant
Federal, local or international financial and procurement regulations;
The checklist is organized according to the main stages of the procurement process such as
pre-biding stage, selection of procurement method, publicity and notifications,
identification of potential bidders, evaluation of bids, and award procedure.
A specific attention is given to activities related to audit preparation.
Since public procurement is one of the activities creating more opportunities for corruption
a fraud and corruption perspective have been included in this checklist. Where the audit
emphasis is on fraud and corrupt practices, then the auditor should take special note of
those questions highlighted with the following warning sign : . If the answer to those
questions is “No” increased risks of fraud and corruption are probable and further analysis
is needed.
The following table provides guidance on analyzing risks‟ level and possible
consequences:

Extreme risk Detailed action/plan required

High risk Needs senior management attention
Risk Level
Moderate risk Specify management responsibility
Low risk Managed by routine procedures
Would stop achievement of functional goals /
5 Severe
objectives
4 Major Would threaten or functional goals / objectives
Consequence
Necessitating significant adjustment to overall
3 Moderate
function
2 Minor Would threaten an element of the function

Audit risk in Public Procurement Audit - variants of approaches in SAIs of

countries

77
Kyrgyzstan
Audit risks that are considered and taken into account
 Detection risk
 Misstatement risk
 The risk of intentional violation of procurement procedures by procuring public
entity in the last quarter of the year. After termination of this period unused
allocated budgetary funds shall be returned to the state budget in accordance
with the provisions of national legislation on the budget law, and their re-
allocation for the next year is largely undefined due to strict policy
constraints in public spending. The higher is the budget for procurement, the
higher is risk of this kind.

China
Audit risks to be considered and taken into account
Government procurement monitory system is incomplete.
Government procurement management internal control system is not appropriate.
(Procurement management supporting rules and regulations is not sound. The
procurement management, internal audit oversight mechanism did not form)
Government procurement procedures are not standard. In the Local government
procurement activities, splitting the market, discrimination against suppliers, black-box
operations, and other problems still exist.

Latvia
Audit risks to be considered and taken into account
A risk assessment should be conducted during the planning phase of the audit to help
ensure that the audit focuses on areas of most significance. For instance:
- Risk that procurement activities are not conducted in compliance with legislation;
- Risk that procurement activities are not completed in a timely manner to achieve
objectives set in legislation and in internal documents of institution;
- Risk that oversight and internal control mechanisms are not effective enough in
identifying, analysing, and monitoring procurement activities across the
institution;
Risk that the procurement process is not transparent and/or equitable, leading to a
damaged reputation and negative publicity.

Portugal
There are risks that:
 Used criteria are not adequate and relevant for the audit object and objectives
 Information to be considered is not complete
 Complex information is not fully understood
 Irregularities are not detected (example: splitting procurement in parts to avoid

78
 competitive procedures)
Strict defined procedures and compliance audit observations increase bureaucratic and
formal procedures by contracting authorities.

South Africa
SAI-SA adopted the ISSAI 1240 approach and it is assumed that there is a
significant risk of fraud in public sector procurement processes. The fraud risk will
translate into a significant risk that requires an overall response which affects how
the audit is planned and executed. It results in a significant risk for certain assertions
for affected classes of transactions and balances and also translates into a significant
risk that supply chain management related legislation is not complied with.

Slovenia
Not obtaining the reliable information about the circumstances of a concluded
contract, especially in the cases where the public contract was concluded without
following public procurement procedures.

Zambia
Audit risks to be considered and taken into account:
Risk of making a negative conclusion based on inadequate documentation
Risk of using inappropriate indicators not agreed upon by management thus
rendering the audit wasteful.

5.3 Methods and main procedures of Public Procurement Audit

One of the most important objectives of the audit is to collect, accumulate and
process a variety of information about the audited entity and the subject of the
audit. Classified, properly processed and well-documented information forms a
complex, systematically organized array of audit evidence.
The information can be both quantitative and qualitative. Quantitative
information is readily measured and is generally used for illustration of statements
and conclusions made during the audit. Qualitative information refers largely to the
formation of general audit context (rules, instructions, regulations, etc.) or plays
the role of addition and extension in respect to quantitative data, making it possible
to make more profound and valid conclusions.
Data collection may be implemented on the single basis, on a regular basis or
directly during the measurements. Methods and techniques for collecting

79
information for procurement are quite diverse and are determined primarily by the
nature and features of the audit engagement. They include the physical proof
availability, analysis of various documents, oral interviews, etc.
The information obtained should be easy to explain and respond to the criteria
of sufficiency, relevance, credibility, completeness and timeliness. Primary
information is gained by technologies of questionnaire survey, surveillance and
direct observation. However, in a number of cases, it becomes necessary to turn to
sources of secondary information, which is contained in official statements,
statistical data, internal reviews, etc.
Main tools for data collection and processing used in the frame of Public
Procurement Audit may be characterized as follows:
 Tools of audit surveillance. Surveillance is a systematic information
collection from certain sources carried out through surveys and questionnaires. The
tools of surveillance are aimed at formation of detailed and specific information
regarding the procurement process. They are especially useful when it is necessary
to obtain information from a large number of sources related to a narrow theme or
problem (for instance, observance of the established procedure for the conclusion
of public contracts for the supply of a specific nomenclature).
 A questionnaire survey is implemented out to obtain audit evidence,
which is best of all to obtain directly from responsible employees or, if necessary,
from other persons. Answering the questionnaire may be implemented in different
ways. In particular, electronic Internet communication channels should be actively
used in modern conditions. Among other methods, there can be sending
questionnaires by mail, telephone surveys and personal interviews. Questionnaire
survey is currently the most widespread, inexpensive and efficient method of
surveillance despite technical difficulties of processing a large number of
questionnaires.
One of the forms of surveys is a questionnaire survey of key personnel and
executives involved in the procurement process. Questionnaire surveys are a
traditional tool to assess the management system aimed at the assessment of means

80
and methods of management and control of the audited system. Questionnaire
surveys can also be used to identify weak and narrow points in audit evidence
collected.
For instance, it is possible to draw up special questionnaire forms to assess
the completeness of compliance with applicable regulations and internal
regulations defining the procedure of procurement process and to assess its
efficiency. Special care is required to develop questionnaires intended to assess the
correctness and lawfulness of substantiation of future procurement, effectiveness
of the internal control system in the area of public procurement.
Questionnaires should be developed by an auditor so as to cover all key
conditions and circumstances of the procurement process, while the instruction to
carry out the questionnaire survey should be clear and easily understandable.
Besides, the following requirements should be met:
 the questions of the questionnaire should reflect the subject of the survey
and prevent ambiguous interpretations or misunderstandings on the part of those
who answer them;
 the language and content of the questions should correspond to the
specific nature of the examined subject or area of the audit;
 the following types of questions should be avoided as far as possible:
suggestive, hypothetical questions, questions containing self-evident answers.
Questionnaires are prepared by the audit team members and, depending on
objectives set and specific features of the audit, may be filled in by the auditor
independently or be given for filling in to appropriate personnel or to a third party.
A special case of the survey is the preparation and filling of the so-called
checklists completed for each area of the audit. The analysis of materials received
by the working group from other participants of this project shows that this tool is
widespread in the international practice of Public Procurement Audit. Its
advantages include relative simplicity, accessibility, and clarity of filling. Besides,
the data reflected in it fully meet the criteria of independence and objectivity, since
checklists are filled in directly by a member of the audit team. Checklists often

81
serve as a supplementing or clarifying element when performing other audit
procedures in order to obtain the necessary audit evidence.
Implementing an interview could be considered as an expanded form of
survey, the main purpose of which is to deepen the understanding of some
particular issue. An interview may be implemented at various stages of the audit
and it can have both structured and unstructured nature. Depending on the nature
and specific features of their implementation, interviews may take on the character
of:
 preliminary interviews, which are primarily used for planning or examining
problem domains
 interviews to collect materials and proof on a certain theme – they are
intended mainly to obtain necessary evidence and arguments for developing the
auditor’s opinion;
 interviews for the purpose of discussing problems and violations identified
– they are used, as a rule, to develop recommendations and search for ways to
address risks;
 analytical interviews aimed at identification of new ideas or proposals.
To obtain a volume representation of the entire picture during the audit, it is
appropriate to take independent cross-interviews of persons with different visions
in the examined area.
The results of an interview should be obligatorily documented and
generalized on various bases: types of problems identified, areas of ongoing
procurements, types of public contracts, etc.
It should be noted that competent conduct of surveys and interviews can be
considered as one of the key skills required from each member of the audit team.
Special care should be exercised in respect of surveys and interviews in the event
of identification of the signs of fraud and substantial violation of the rules of public
procurement.
This procedure should be implemented in several stages:
 preparation stage,

82
  initial stage,
 survey or interview by itself,
 final stage,
 processing and analysis of results.
Each of the above stages should be described separately in the Guide on
Public Procurement Audit.
 Direct observations - they are not widely used in the audit, however,
they might be effective enough. This tool makes it possible to understand how
operations are carried out, study the opinion of those directly responsible for them,
as well as to obtain direct understanding of audited processes.
The observation procedure may take on two forms: passive and active.
Passive observation suggests that the member of the audit team stays at the
same place in order to directly observe the action of a certain individual, a group of
individuals, the flow of the audited process, and specific operations. Observation
may be performed either on a target-focused or anonymous basis. Besides, third
parties (experts) can be involved in the observation process, if necessary. The latter
can take place when the member of the audit team is unable to carry out
observation by himself/herself. Active observation implies following after the
responsible employee (employees) along the entire chain of the audited procedure.
In order to efficiently carry out the observation procedure, the following
general rules should be observed:
 if possible, the observation should be implemented in pairs;
 continuity and consistency of the observance procedure within the
monitored period should be provided for, if possible;
 it is necessary to provide for immediate and complete registration of all
assessed positions: time limits, control points, regulatory activities, compilation of
documents, approval procedures, etc. No gaps and omissions should occur in the
frame of recording the process flow.
If the management or employees of the audited entity refuse to take part in the
observance procedure or avoid explaining its results, auditors should pass on to
83
alternative procedures of audit evidence collection, including questionnaires and
surveys of employees.
 Secondary analysis and search in external sources. Secondary analysis is
used more in assessment of efficiency and is implemented through the review of
any sources not directly related to the audited entity. These may be scientific
developments and reports, the results of research and development and regulatory
documents defining the specifics of public procurement at the audited entity,
external statistical and analytical reviews, expert opinions, etc. Internal documents
of the audited entity that do not directly relate to the organization of the public
procurement process, the materials of previous research, control activities and
audits might be referred to the secondary analysis. Such information can give
additional information or communication connections, form an objective picture of
the management system and internal control of the audited entity and allows
enhancing principal conclusions and arguments obtained during the audit.
Process-oriented approach suggests that the issues of proper organization of
the procurement process, justification of the outcomes and search for the ways of
their improvement become the key issues of the Public Procurement Audit. The
main tools of the audit, accordingly, should be focused on the search for and
confirmation of the following aspects:
What steps and processes are involved and what is the degree of efficiency
and effectiveness of this involvement?
What are system constraints of the operation of the audited process?
What are main risks of the project implementation?
Such approach is especially effective in the audit of major, complicated
procurement programs, which are long-term in their nature. However, it might be
successfully applied in other cases as well.
The most widespread types of analysis are the following: cost-benefit
analysis, assessment of the effect according to the “what if” principle and
application of benchmarking technologies (comparative analysis).
Cost-benefit analysis is based on the comparison of expenses and usefulness

84
(benefits), both direct and indirect, provided by public procurement. Comparison is
usually performed in terms of value. This approach is often used in assessment of
investment projects, which might be considered as a certain case of infrastructure
procurement (for instance, construction of roads or facilities for a football
championship). The main questions asked by the auditor in the implementation of
the said approach is whether budgetary expenses incurred meet the criteria of pay-
back or economic efficiency.
A version of this approach is the choice of the analysis methodology
according to the cost-benefit criterion, which examines the relations between the
costs of designing and results expressed per unit of the outcomes obtained. In the
frame of the second approach, the outcomes might be expressed not in value
indicators, which allows expanding the number of criteria and methods of
evaluation.
“What if” assessment makes it possible to carry out evaluation and analysis
of various scenarios and alternatives showing possible gains and losses from
changing procurement parameters. The sense of this approach is to identify the
factors that can affect the outcomes of the public procurement procedure. A special
case is the consideration of situation where the given procurement or program is
not implemented at all.
Comparative analysis (benchmarking) is aimed mainly at assessing trends and
identifying alternatives. Comparisons may be implemented on various bases:
between those working well and working badly; between standard and actual
result; between fact and plan; between similar procurement contracts or between
entities or units that are similar by performance indicators, etc. One of the main
questions of comparative analysis is whether there was the opportunity to obtain
better result in comparison with the achieved one.
Various methods may be used to carry out such examination: estimation of
coefficients, trend analysis, construction of time schedules, alternative calculations,
etc.

85
 Methods of qualitative analysis are mainly used to analyze the results of
surveys and interviews. In the frame of studying documents, orders, contracts,
regulations and other documents such qualitative analysis always implies building
thematic information blocks and their classification in accordance with pre-
determined criteria.
At the same time, it is necessary to remember that the analysis of information
is always a complicated intellectual and creative process, where both rational and
irrational elements could take place. As alternative versions of analysis, it is
possible to use group discussions, brain storm, content analysis, comparison,
expert analysis, etc.
Quantitative analysis technologies used in Public Procurement Audit are
aimed at the search and processing of information distributed by time and other
quantitative parameters. This may be the data on procurements completed within
the period, arrival of purchased nomenclature inflow, progress of construction,
release of material values within the period, performance of the schedule of
funding, etc. The variety of methods and approaches of quantitative analysis
allows considering tools used therein separately. Their full listing is beyond the
scope of this section; however, for methodological purposes it is advisable to
describe briefly at least two types of such tools:
 Methods and tools of descriptive statistics aimed at the analysis and
processing of data on the distribution of assessed values. Such distribution, as a
rule, may be expressed in a form of a diagram based on estimates of the analyzed
variables. Principal areas of such analysis, which might be used in the frame of
Public Procurement Audit, are:
 assessment of the main trend in distribution with the definition of its key
features (mode, median, mean, etc.);
 the nature of information dispersion and volatility (assessment of
dispersion, minimum and maximum level, etc.)
 main characteristics of data (standard deviation, Gaussian distribution,
evenness, etc.)

86
 Information on parameters and characteristics of information arrays is very
important for Public Procurement Audit and may be used in a very effective
manner. Thus, conclusions are made on the basis thereof on the need to carry out
these or those audit procedures, assess audit risk and reliability of internal control
systems, analyze the relevance of data obtained for the achievement of objectives
set for the auditors, etc.
 Regression and correlation analysis may be used to establish interrelations
and determine the degree of influence between variables, as well as to identify
unusual or undesirable relations between expected elements.
The independent areas of the use of quantitative information processing tools
are the methods for evaluation of the quality of nomenclature purchased in the
frame of performance of public procurement contracts and analysis carried out in
the frame of technologies of the so-called continuous audit, which is based on
constant automated monitoring of data flows in computer systems. Under the
conditions of the explosive growth of information technologies working in the real
time mode and creation of trading platforms with a maximum level of
computerization, the latest trend seems to be especially promising.
The main purpose of control testing is to provide for the necessary objective
level of the auditor’s confidence that the audited process, procedure and outcomes
have been carried out or achieved exactly in accordance with the evaluated criteria
or standards. The main idea of control testing is based on the fact that the
assessment of part of ongoing operations or processes will demonstrate the same
results as the audit of the entire system. This technology allows optimizing the
time and efforts to carry out the audit.
Besides, it is possible to distinguish three types of tests widely used in Public
Procurement Audit:
 compliance test
 independent confirmation on the basis of an audit sample
 analytical reviews.
Below are the features of each of the methods listed above:

87
 Compliance test is aimed, first of all, at the confirmation of compliance with
applicable laws and internal regulations on procurement. Its principal goal is to
assess the compliance of procurement processes and procedures performed with
established standards and assessment criteria. The results of testing may serve as
audit evidence of efficiency and quality of the internal control system. Such tests
may also be used to check the quality of this or that process and assess the
outcomes thereof (for instance, documentation of procurement, meeting the
delivery deadlines and quality requirements for the nomenclature delivered, etc.).
The confirmation of the compliance test may be carried out both on the basis of
standard documentary evidence and using automation means and electronic
documents.
The choice of the number of objects subjected to the compliance test
procedure should provide assurance that an unambiguous conclusion can be made,
based on the results of the test, that all the prescribed rules and regulations have
been observed (have not been observed), and the internal control system worked
efficiently (inefficiently). The selection of the objects of testing may be made by
means of systematic, random (statistical) or stratified sampling.
General conclusions made on the basis of the compliance test should confirm
or fail to confirm the reliability of control systems, which, in its turn, should
influence the choice of the set of independent audit tests. The results of compliance
test largely determine the degree and scale of subsequent independent testing.
Efficient controls, the effectiveness of which has been confirmed by compliance
testing, allow making a conclusion on the possibility to carry out a lower volume
of independent tests. The reverse situation is also possible.
Independent confirmation based on the audit sampling (independent testing)
is aimed at obtainment of objective validations and evidence regarding correctness,
reliability and completeness of quantitative and qualitative data on the results of
public procurement, public contracts awarded, legality and substantiation of
budgetary funds spending, as well as appropriate data of budgetary accounting.
Besides, independent testing may be aimed at the identification of errors,

88
violations and failures in public procurement system, assessment of the nature and
scale of their consequences. Independent sample tests, as a separate area, may
provide for analysis and assessment of efficiency of unusual, complicated or major
transactions related to public contracts, as well as transactions in the frame of a
certain national program.
An audit sampling is an independent testing of selected elements within the
pre-determined population of audit objects for the purpose of analysis and
assessment of obtained evidence regarding the distinctive features of the specified
population and the formation of opinion on the entire pre-determined population.
An important condition of a properly selected sampling is its representative
character. That means that correct and objective conclusions should be made on
the basis of the selected population.
As it was mentioned above, there are various approaches to the formation of
an audit sampling. The approach aimed at obtaining a random sample is the most
widespread.
Random sampling implies that all selected elements are taken at random from
any pre-determined data array. The random nature of the sample suggests the
equality of the choice of any element. Such approach works well in situations
where documents or operations with a unique registration number are subjected to
audit (for instance, uniform public contracts, invoices, delivery documents,
payment orders, etc.). To ensure the randomness of selection, a table of random
numbers or computer random number generator may be used.
After the formation of the sample, pre-determined audit tests are applied
thereto. It should be noted that a random sampling can also be used in audit
procedure on the merit, for instance, when assessing the quality of delivery in the
frame of performance of a public contract for the delivery of a large homogeneous
nomenclature with pre-determined quality and quantity.
Stratified sampling - is carried out with respect to an examined population,
the constituent elements of which can be grouped (stratified) on certain, generally
quantitative grounds. The established level of materiality is very important for the

89
choice of a group (strata). Thus, in the frame of the entire population of
procurement all public contracts may be grouped, for instance, on the cost basis:
 minimum - up to EURO 100,000
 medium - from EURO 100,001 to EURO 1,000,000;
 upper-medium level - from EURO 1,000,001 to EURO 10,000,000;
 large - above EURO 10,000,000.
Statistical sampling is built upon the use of various statistical criteria, where
there is a large amount of heterogeneous procurement operations or procurement
operations with special characteristics. Well-known heuristic methods, for
instance, Pareto rule 80/20, could also be used for taking decisions on the amount
and nature of the chosen population. In the latter case, the sampling is based on
operations or contracts constituting 80% of expenses. The Pareto principle in this
case suggests that about 20% of procurement contracts will form 80% of expenses
of the audited entity on public procurement.
Analytical reviews are prepared based on the results of a random testing and
special examination of audit evidence obtained on the basis of study and
comparison of financial, legal and other information regarding procurement
processes and procedures of the audited entity. The most important coefficients,
trends, standards, results of the procurement system operation are studied and
unusual deviations and fluctuations of values are examined during an analytical
review. Analytical reviews are often considered as an element of independent
testing. But for the purpose of development of the Guide on Public Procurement
Audit this element should be considered separately.
5.4 Collection and analysis of audit evidence
During the audit, a number of actions should be implemented to collect and
prepare required audit evidence, which implies the availability of thoroughly
prepared procedures of data collection and formation of samples, taking into
account the level of materiality. Besides, auditors should have necessary
competence and skills in using such methods and procedures as inspection,
observation, inquiry and confirmation. Audit evidence collected should support the
90
auditor’s opinion on the audited procurement system as accurately and completely
as possible.
Collected audit evidence may refer both to general and to specific objectives
of the audit. Their full array should contain information on the principal audit
issues. Key issues linked to the standard process of public procurement were
shown above.
The main sources of audit evidence include the data of accounting and
internal control system, electronic procurement system, other primary or
supporting documents of the audited entity, stock-taking materials, information
obtained from the management and employees, clients, suppliers and other third
parties, if necessary.
In general in searching of audit evidence it should be guided by the ISSAI
1500 «Audit Evidence» and 1501 «Audit Evidence—Specific Considerations for
Selected Items».
An auditor is obliged to collect enough objective and reliable evidence to
form a substantiated judgment on the audited entity. Besides, he/she should rely on
his/her understanding of activities of the audited entity and specificity of its
procurement system, assessment of the level of materiality and audit risk,
reliability of the control system and organization of accounting of ongoing
procurement, credibility and relevance of obtained evidence.
The main sources of audit evidence are:
1. Accumulated experience, competence and personal knowledge of an
auditor sometimes serve, since the auditor acts independently, without any interest,
relying on his/her own judgments and qualification;
2. Material and documentary confirmation of existence and assessment of the
objects received as a result of public procurement include materials of stock-
taking, accounting and internal administrative records, primary and financial
documents, public contracts and annexes thereto, etc.;
3. Carrying out test procurements and accounting their results contributes to
independent and objective confirmation of the public procurement procedure,

91
correctness of its documentation and recognition in the accounting of the audited
entity. Thus, it is possible to carry out a control procedure to check compliance in
terms of quantity and price as recorded in invoices, certificates and delivery
documents. Relevant control activities can also be carried out while testing
controls of procurement process to check their compliance with established
regulations and instructions;
4. Reception of external and internal confirmations. External evidence is
formed outside the audited entity and includes allegations, documents, oral
confirmations made by third parties to the audit team members. Internal evidence
is formed inside the audited entity itself and is considered as less reliable.
However, they are, as a rule, the most accessible and full evidence, and their
obtainment involves low costs and short time to receive.
The main methods of collection of external and internal evidence are the
preparation of requests, receipt of confirmations and application of analytical
procedures.
The preparation of requests and receipt of confirmations are considered as
principal methods of audit evidence obtainment. Such requests can be direct:
requests for information on procurement operations, availability of delivered
assets, sums paid according to invoices. Evidence can also be received as written
and internal allegations and provision of confirmations directly from the personnel
of the audited entity. Analytical procedures are intended for the review of
information on the procurement system, financial and other accounting data in
order to find operations or actions that are unusual or deviate from established
rules. Carrying out analytical procedures involves, inter alia, the examination of
specifications, terms of reference and estimates attached to public contracts,
accounting registers and accounting statements, analysis of certificates, delivery
documents, bills, invoices, inventory control data, etc.
The receipt of external inquiries, among other matters, can also suggest
involving external experts, which is provided for by INTOSAI standards. In
modern international audit practice, turning to experts’ opinion, especially on

92
special matters, is a widespread audit technique and is considered as an integral
part of an audit. In using experts it should be guided by the ISSAI 1620 «Using the
Work of an Auditor’s Expert».
An expert is a natural or legal person having certain professional knowledge
and experience in some special field of expertise. An expert’s work is used by an
auditor as an auxiliary aid to obtain sufficient and proper audit evidence. An expert
can be either an internal expert, whom the auditor chooses from among employees
of the public-sector financial control body not involved in the audit, or an external,
i.e. outsourced, expert.
Turning to external experts has become a widespread practice in financial
audit. However, it is rather widely used in performance audit as well, in particular,
in the so-called thematic reviews on special issues requiring knowledge and
experience in a certain narrow field (for instance, the quality and reliability of
engineering structures, procurement of special equipment with no analogues
available, environment protection, etc.).
Thematic reviews in performance audit are intended to demonstrate the
availability of these or those problems or to evaluate specific characteristics of the
purchased nomenclature. They are aimed, as a rule, at analysis and comparison
and, in combination with other methods, may provide some general conclusions on
the subject of the audit.
The role of experts, as a rule, is to accomplish the following tasks:
- to carry out a comprehensive and profound analysis of existing or potential
key problems;
- to obtain spectacular examples;
- to assess and confirm earlier estimates, measurements, conclusions,
including those made as a result of other studies.
Besides, apart from requirements of professional competence and experience
in the field of expert evaluation, the expert must comply with provisions of
confidentiality and professional secrecy provided for by appropriate ethical
requirements applied to the audit team members.

93
 5.5 Audit documentation

An auditor should be able to present all necessary audit evidence confirming

the audit findings presented in the audit report. Besides, a necessary condition of a
high-quality and efficient audit is the availability of duly prepared and executed
working documents.

The following set of audit files should be made up for the storage of audit
documentation:

 permanent file;
 file of planning and quality control;
 working files (working documentation).

The permanent file should include general information on the audited entities
and, as a rule, contains the following documents:

 information on the audited entity, copies of foundation documents,

registration and approval documents, etc.;
 characteristics of the organizational structure, names of key employees, etc.
(such data should be updated as and when necessary);

 characteristics of the procurement system of the audited entity;

 all correspondence with the audited entity regarding general issues and the
issues of the audit organization;

 copies of previous reports of the auditor, etc.

The file of planning and quality control should contain information on the
audit engagements, the plan and program of the audit, documents on the
organization of the audit process, information on the audit team, etc. It is used as a
depository of organizational documents and contains the information on the
progress of the audit, as well as on various issues related to various stages of the

94
audit. In particular, as for the ex-ante stage, it contains the information on which
engagements have been planned and in what sequence. During the audit, it may
accept the information on any changes in objectives, adjustments on the audit plan,
and changes in the time and deadline of the audit, etc. At the final stage of the
audit, this file accumulates the information on the actual completion of the work,
responsible persons and the time of completion.

The working file (working documentation) is the main audit file containing a
detailed disclosure of the process of the audit program implementation and all
working documents prepared by the audit team. Upon the completion of the audit,
this file is reviewed and finally compiled by the head of the audit team and
submitted for the review by the auditor who is responsible for the audit and
approves all its findings.

The final summary of the audit findings is prepared and the auditor’s report is
compiled on the basis of the documents accumulated in the working file. The
contents of the working file reflects the volume of the work performed and gives
an idea of the array of audit evidence gathered during the audit on each item of the
audit engagement.

The preparation and execution of working documents should be carried out

based on the procedure developed with due regard for international audit standards
and using standard forms.

All documents of the auditor should be analyzed, properly arranged,

numbered and indexed. The index is put in the upper right corner in accordance
with the established procedure.

The working documents of one audit engagement should contain cross-

references to other working documents in the audit file related to them. Such cross-
indexation ensures the necessary linkage between the various stages and audit
procedures by putting in order the array of prepared audit working documents and

95
audit evidence collected at each stage of the audit. Particular attention should be
paid to links in the documents containing interrelated amounts and figures.

The audit team members should ensure the safety and confidentiality of audit
documentation. Upon the completion of audit, the head of the audit team should
make sure that all audit documents have been copied, duly executed and certified
and placed into the appropriate file. The audit team members must comply with the
requirements of confidentiality and professional secrecy. The requirements of
confidentiality or secrecy must also be complied with in respect to audit
documents, irrespective of the fact whether such documents have been created
within or outside the audited entity. The documents collected should only be
discussed by the audit team members, except for specially defined conditions. No
information about the audited entity that became known in the course of the audit
can be disclosed or published without an appropriate permission. If necessary, a
request to that end may be made directly to this audited entity.

In accordance with the requirements of international audit standards, auditors

are obliged to record all actions carried out during the audit, audit evidence
obtained, as well as the results of analytical procedures and estimates in the
relevant working documents.

The full package of working documents (final audit file) should contain
information that is sufficient, reliable, valid, may be corroborated and allows,
based on the audit evidence collected, preparing the auditor’s opinion and
compiling the detailed audit report.

Besides, working audit documentation allows evaluating the quality of the

audit in the following areas:

 compliance by the audit team with audit standards and other management
documents associated with the audit;

96
  confirmation of performance of the audit engagement, introduction of
amendments in the plan and program of the audit, review of the audit progress;

 use of experts’ services in the work;

 review and analysis of situations associated with the identification of cases

of abuse, fraud and gross errors in the procurement management system of the
audited entity.

The final audit file should also contain the documents reflecting:

 discussions of the results of interviews and surveys specifying the

responsible persons, time and place of such audit procedures;
 description of procedures of observation over the procurement procedure,
including the place and time of and reason for observation and the personnel
engaged in the observation;

 reports and data obtained directly from the electronic procurement system
directly by the auditor, specifying the date, time and circumstances of receipt of
such information;

 comments and explanation regarding the problems, ambiguous situations,

requests for additional information.

The final auditor’s report based on the audit findings should also be
considered as an integral element of the audit documentation. Time limits, scope
and nature of main audit procedures should be included into this report on a
mandatory basis. The membership of the audit team and the timing of the audit, as
well as the overall description of the quality control of the audit specifying the
timing thereof should be documented in the auditor’s report.

The timing and contents of principal audit procedures, as well as the deadlines
of the audit of each audit area according to the approved audit plan should be

97
included into the file of the audit in the form of separate, specially executed
documents.

The content of the file of working documents should be adjusted depending

on the object, subject and applied control procedures. Thus, in the event of
implementing procedures requiring the sampling or review of elements of the
population selected depending on a certain sum (for instance, all contracts not
exceeding EURO 100,000), a document containing the records of the volumes of
the public contracts included into the sampling and the amount of the sampling
will be generated.

For procedures implying information requests to the personnel involved in

procurement, the audit documentation may contain the numbers, dates, deadlines
for the answer, the content of requests, as well as the name of the person who
replied to the request. During the observation procedure, the documents
determining the timing and flow of the observation process, the object of the
observation, responsible personnel, etc. should be included into the documentation
file.

5.6 Types of audit conclusions

In accordance with ISSAI 100, Auditors should prepare a reporting

document based on the conclusions reached.
The audit process involves preparing a report to communicate the results of
the audit to stakeholders, those charged with governance and the general public.
The form and content of a report will depend on the nature of the audit, the
intended users, the applicable standards and legal requirements. The SAI’s
mandate and other relevant laws and regulations may specify the layout or wording
of reports, which can appear in a short form or in full.
Expanded reports describe in detail the audit scope, audit findings and
conclusions, including potential consequences and constructive recommendations
to enable remedial action.

98
 Short-form reports are presented in more condensed and generally a more
standardized format.
In accordance with the INTOSAI standards, two types of implementation of
audit checks/engagements are used:
In the attestation engagement the person in charge conducts measurements
related to the audit scope in accordance with the defined criteria and presents
information regarding the scope on which the auditor then collects sufficient and
proper audit evidence ensuring a sound foundation for forming a conclusion.
In engagements on direct reports measurements or evaluations of the scope
are carried out by the auditor. The auditor chooses the scope and criteria with due
account taken of risks and materiality. The result of measurement by the scope
against criteria is submitted in an audit report in the form of findings, conclusions,
recommendations or opinion. An audit may also result in new information,
analysis findings or the understanding of the scope.
The audit report should explain how the evidence obtained was used and
why the resulting conclusions were drawn. As a result, the required degree of
target users’ assurance in the data presented therein will be achieved.
One of the types of audit conclusions is an opinion
Types of opinions
 Qualified (except for)
 Adverse
 Disclaimed.

Versions of the types of audit conclusions

- variants of approaches in SAIs of countries
Kyrgyzstan
Brief report, expanded report, auditor’s opinion specifying detected violations and
shortcomings, if they have a place, appropriate recommendations / instructions for
their removal and correction

99
China
We did not publish formal and structured audit opinions.
Latvia
The opinions are to be applied depending on the type of audit – financial,
compliance or performance.

Portugal
 In financial and compliance audits the opinion may be:
 Favourable: the contracting entity internal control, procedures, and
procurement activity, in general, comply with applicable laws, regulations and
international standards, as well as, with the principles of economy, efficiency
and effectiveness
 Favourable with a qualification: except for specific deficiencies, internal
control inadequacies, or minor cases of non-compliance, the contracting entity
procurement activity is compliant with applicable laws, regulations and, as well
as, with the principles of economy, efficiency and effectiveness
 Unfavourable: due to relevant errors, deficiencies, inadequate internal control,
and cases of non-compliance, the contracting entity procurement activity is not
accordance with applicable laws, regulations and international standards
 Neither favourable nor unfavourable: the auditors were unable to express an
opinion due to significant limitations on audit scope, such as, the absence of
relevant procurement documentation, lack of cooperation of the audited entity,
or inaccurate or incomplete related financial information
 In mixed or performance audits answers are given to audit questions, against defined
audit criteria and recommendations are given accordingly.
 The SAI also approves studies and reports without formal opinion, identifying
recurrent weaknesses and irregularities and formulating recommendations for
improvement
 The SAI also issues approval or refusal of approval on signed contracts above certain
thresholds, as required by law
The SAI includes in the audit reports information and details on concrete
irregularities found in order to proceed with judicial processes

South Africa
Currently, SAI-SA does not issue an opinion or conclusion in compliance. We only
list the material findings in the audit report. We are currently testing a new audit
methodology that will enable us to issue an opinion in compliance.

Slovenia
In accordance with the audit criteria, which serve as the basis for the evaluation of
the audit results, an opinion is expressed through the audit report. In case of
financial audit and compliance audit, an opinion is expressed in one of standard
forms as non-qualified opinion, qualified opinion or negative opinion. The
expression of opinion can also be rejected.
In case of performance audit, opinions expressed are always descriptive. An
100
 auditor’s opinion should represent clear answers to sub-questions of the audit,
which, at the same time, serve as the answers to the main question of the audit.
Standard audit criteria are usually not adopted for such audits. Thus, it is even more
important that an auditor should clearly determine criteria that are actually applied.

5.7 Main approaches to evaluating of performance of Public

Procurement Audit

Efficient functioning of any procurement system implies that the

procurement process is carried out properly, on a regular basis and in accordance
with established rules. The procurement process should be carried out without
delays, failures, uncontrolled financial and material losses.
In the frame of a project, as a rule, the following types of efficiency are to be
assessed:
 the project efficiency under the market conditions is assessed in order to
determine potential attractiveness of the project for possible participants and to
search for sources of funding, while in the budget sphere, it can be considered as
the ultimate effectiveness of the use of budget money;
 the local efficiency of certain elements of the project.
Besides, the following three types of efficiency could be distinguished in the
assessment of the project efficiency as a whole:
 investment (financial) efficiency takes into account financial
consequences for participants implementing the investment project, with the
assumption that they have incurred all costs necessary to implement the project and
they can take advantage of all its results;
 social (public) efficiency takes into account socio-economic
consequences of the investment project for society in general, including both direct
results and expenses of the project and “external” results and expenses in related
sectors of the economy, environmental, social and other non-economic effects;
 budgetary efficiency is the efficiency of participation of the state in the
project; it may be considered in terms of cost cutting and increase in income.

101
 Standard indicators of investment (financial) efficiency are net present value
(NPV) of the project and internal rate of return (IRR). NPV characterizes the
excess of the total amount of money received from the project implementation over
the total costs, taking into account the discount, and is defined as the accumulated
discounted effect of the project implementation for the accounting period. IRR is
characterizes as discount rate at which the net present value of the project is equal
to zero.
The basic indicator of budgetary efficiency is the ratio of discounted tax
revenue and (or) saving of budget costs due to implementation of the procurement
contract, to the total volume of budgetary funding for its implementation.
The process idea about efficiency is formed as a result of evaluation of the
availability and quality of operation of main elements of the implemented process.
A competent building of each element acts as sort of internal warranty of the
overall efficiency of the entire procurement process. Besides, in order to recognize
that all basic components are available and operate properly, it is necessary to
confirm not only their availability, but also the absence of significant deficiencies
in their functioning.
In general, the following two components of process assessment should be
distinguished:
 procedural efficiency, i.e. correctness, legality, lawfulness, accuracy of
performance of individual procedures of the procurement process. Each of them
can be implemented inefficiently or with violations, which entails failures, errors
and violation of the entire sequence of the procurement contract implementation;
 efficiency of result is assessed, first of all, in terms of correspondence to
initial indicators pre-determined in the specification. Classical estimation of
investment (financial) efficiency is just the appraisal of result.
Both components can be supplemented by the introduction of the third
component, subsequent or operational efficiency, which is assessed on the basis of
the outcome at the expiration of the delivery warranty period through the indicators

102
of correspondence of the overall result, reliability or productivity of purchased
goods, equipment, etc. to the parameters specified by the supplier.
Efficiency assessment implies the choice of appropriate control points or
assessed positions – these are those issues or elements of public procurement that
are to be considered during the audit.
Besides, it suggests determining criteria (indicators) of efficiency assessment.
They are often based on a simple price comparison of the final price with the
prices of similar purchases in the previous period, or with the prices of other
suppliers participating in the procurement procedure, or with the price prevailing in
the market. However, such approach seems to be one-sided and, therefore, in order
to carry out a high-quality Public Procurement Audit, there should be a
comprehensive set of criteria, each of which would assess any single, but fairly
significant aspect of the public procurement results. A possible, although
incomplete, set of practicable criteria might be presented as follows:

Name of the group of criteria Content of the group

1. Criteria of assessment of documentation criteria of documentation completeness

associated with the substantiation, organization criteria of alignment and non-contradiction of
and performance of public procurement documentation
criteria of legality and validity of the
documentation

2. Criteria of allocation and use of resources sufficiency of the resources allocated

in public procurement validity of the resources allocated
completeness of the resources allocated
timeliness of the resources allocated
complete use of the resources
timely use of the resources

3. Criteria of efficiency of organizing the Criteria applicable to the evaluation of project

management of the process of public management performance may be used here
procurement planning and implementation

4. Criteria for assessing the achievement of

specific characteristics of goods, works and
services reached at public procurement,

103
 Name of the group of criteria Content of the group

including

4.1. criterion of sufficiency (soundness, absence of excessively high expectations and

optimality) requirements (taking into account
development prospects) in respect of
characteristics of purchased goods, works and
services. Failure to comply with this criterion
entails an increase in the cost of procurement

4.2. criterion of compliance of the defined extent of compliance of procurement

characteristics characteristics at all stages of the life cycle
with the requirements contained in relevant
regulatory documents

4.3. criterion of advantage (superiority) of advantage of the procurement characteristics

the decisions made proposed for fulfillment of the functional tasks
of the customer in relation to the existing
methods, procedures and technologies of their
solution in terms of feasibility, cost and time
inputs into the function fulfillment

4.4. criterion of compliance of the quality of It is determined for each type of procurement
purchased goods, works and services with
modern requirements

The availability of criteria alone does not solve the issue of efficiency
assessment, since a combination of appropriate indicators is required for this
purpose as well.
When choosing the indicators of evaluation of the efficiency of public
procurement implementation, it is necessary to take account of the fact that they
should quantify the implementation progress; reflect the extent of solution of main
tasks and fulfillment of objectives; enable evaluation of the footprint of creation
(introduction, development) of procurement results with due account taken of their
specific domain and have a corresponding quantitative value calculated with
approved methods. In addition, indicators of evaluation of the efficient use of
budget funds should reflect the effects of procurement implementation.
The following indicators may be used as the main indicators of evaluation of
procurement implementation quality and efficiency:
 indicators of fulfillment - are calculated by stages and key points;

104
  planned budgetary assignment - completeness and timeliness of
achievement of the public budget values;
 indicators of the legality of budget spending - confirmation of compliance
of actions aimed at using budgetary funds for procurement implementation with
the requirements of the national budget legislation.
Despite the fact that it seems impossible to create an exhaustive set of
indicators, yet it is advisable to define a minimum set thereof to encompass most
of the questions of assessing the efficiency of public procurement. This set might
include, in particular, the following indicators:

Planned value of expenses part of the planned cost of procurement

on procurement implementation, which should have been spent by the
time under consideration in accordance with the initial
plan

Actual value of expenses actual expenses on the actions in the frame of

on procurement procurement, which have been completed by the time
under consideration

Planned cost of planned cost of actually performed procurement

procurement

Feasibility the possibility to attain the objectives of procurement

implementation in the light of planned and allocated
resources, time and other restrictions; characteristic of
procurement implementation progress at a given stage;

the value characterizing the degree of achievement in

Effectiveness procurement implementation of the desired results
according to the ratio of the planned and actual time
spent on the implementation of activity

efficiency the ratio between the results achieved in procurement

implementation, and material, financial, labor and
other types of resources spent to obtain these results

Economy the value, characterizing the amount of inputs into

procurement implementation compared to other
similar activities based on a certain comparison
baseline

105
Excessive spending the difference between the actual amount of resources
used in procurement implementation and the amount
of minimum resources required for the implementation
thereof

Resource intensity the amount of all types of resources necessary for

procurement implementation with pre-determined
results within prescribed time limits

Resource availability the value characterizing the ratio of the volume of

resources allocated to procurement implementation, to
its resource intensity

Expediency the usefulness of procurement from the standpoint of

the end user of its results

A sample set of efficiency assessment indicators for a particular contract may

be determined by the auditor at the audit planning stage, taking into account the
characteristics of the audit engagement.

Example of performance audit - variants of approaches of Slovenia

“The implementation of a public-private partnership project was compared with the
opportunity provided to a municipality to take a decision to finance investments
with borrowed funds.
On the basis of the concession contract the municipality undertook to pay to the
private partner a monthly fee in the total amount of EURO 10,104 over a period of
15 years (This amount includes the payment for the use of the gym in the amount of
EURO 9,548 and the payment after the termination of the ownership right to the
building, broken down into monthly payments in the amount of EURO 556)
The analysis of the competence of the public-private partnership proves that if the
municipality had taken out a loan, implemented a new procedure of awarding a
public contract, built and operated the gym independently, the investment would
have amounted in total to EURO 1,666,496, which is by EURO 152,144 or 9.1 per
cent less than the total sum that the municipality is obliged to pay for the gym to the
private partner within the given concession period, as well as less than the
compensation associated with the increase in the economic importance of the gym
after the termination of the rights to the building. Besides, the Court of Accounts of
the Republic of Slovenia also assessed that if the municipality had undertaken a new
procedure of the public contract awarding and taken a decision on borrowed funds,
it would have been more advantageous for it from the financial point of view, since
a monthly installment in the event of investment funding in the amount of EURO
840,000 by means of borrowed funds for the same period would have amounted to a

106
 sum between EURO 5,787 and EURO 6,291.
Actual costs of maintenance, operation, management and administration over the life
time of the object were unknown. However, it is possible to suppose that, taking into
account the monthly sum of repayment in the amount of EURO 5,787, such
expenses could amount to EURO 4,317 per month or EURO 3,813, taking into
account the monthly sum of repayment in the amount of EURO 6,291. Yet, taking
out a loan in order to build and maintain the gym independently remains a decision
that is financially more advantageous for the municipality.”

Annex 5 contains an example of an audit program of the SAI of Zambia,

which could be used to assess the adequacy of the structure and efficiency of an
entity that carrying out procurement activities in comparison with other similar
entities.

Main approaches to evaluating the efficiency of performance of public

procurement contracts
- variants of approaches in SAIs of countries

Belarus
The main criterion of efficiency of performance of public procurement contracts is
the timely involvement of purchased goods into business turnover and the general
effect from their use.

China
1. Indicators and criteria used to assess efficiency of performance of public procurement
contracts
The efficiency can be assessed based on a benchmark weighing the average scores
that projects are evaluated mainly through the following aspects,
Whether the procurements are conducted in a fair and impartial manner.
Whether all qualified vendors have access to public business.
Whether no offer or is arbitrarily or capriciously excluded.
Whether competition is sought to the maximum feasible degree.
Whether procurement procedures involve openness and administrative efficiency.
Whether the rules governing contract awards is made clear in advance of the
competition.
Whether specifications reflect the needs of the purchasing body rather than favor a
particular vendor.
Whether the purchaser and vendor freely exchange information concerning what is
sought and offered.

Portugal

107
Examples of criteria/indicators used by our SAI to assess efficiency and effectiveness of
performance of public procurement contracts:
 Standardization and adequacy of the procurement management process and skills
of staff to manage those processes
 Skills and practice of procurement staff to look for advantageous conditions in
procurement
 Timeliness, duration and time waste of procedures
 Justification, accuracy and adequacy of needs
 Reasonability of purchases towards the actual needs
 Needed funding available
 Division of risks in PPPs
 Cost/benefit analysis
 Comparison with alternatives
 Use of centralised purchasing to obtain savings
 Duplication of contracts for the same purpose
 Accuracy and justification for contract description
 Timeliness, adequacy and consistency of preliminary studies and advices with the
concrete contract description
 Ground for qualification criteria and defined requirements
 Justification and adequacy of the award criteria
 Verification of the application of the award criteria, in terms of the lowest price or
more advantageous offer
 Use of negotiations in the procurement procedure and value obtained from them
 Justification of estimations and prices
 Final costs compared to initial estimation
 Final costs compared to market prices/indexes for the same type of works
 Costs compared to previous contracts for the same purpose
 Benchmarking with similar purchases
 Required and offered guaranties
 Scope for dispute in execution of contract
 Time limit for contracts
 Control, review and evaluation of contract performance
 Match of goods delivered and works done with specifications of the contract
 Timely delivery

108
  Consistency between budget allocations and programmed payments
 Application of controls before payment
 Publicity of awards
 Actual need for the modifications introduced to the contract
 Ground and proportionality of the price of modifications
 Need and causes for compensations and payment of interests
 Assessment of savings’ criteria
 Satisfaction of users

Further efficiency and effectiveness criteria/indicators can be built using the audit
questions included in the document “Public Procurement Audit” in the EU, which
was published by our SAI.

Slovenia
Example: The audit of efficiency and effectiveness of investments in the buildings
of a pre-school institution at the stage of construction
Criteria used to assess the efficiency and effectiveness
The assessment of efficiency and effectiveness of the municipality’s operations
associated with investments in the buildings of a pre-school institution is based on
the answer to the main question: is it true that the municipality has efficiently and
effectively implemented investments in the buildings of the pre-school institution?
The answer to the main question is based on the answers to the following sub-
questions: Is it true that the municipality efficiently planned and implemented
investments in the buildings of the pre-school institution? Has the municipality
reached the desired goal through investments? The answer is confirmed by
operative questions represented in the Matrix presented in Annex 5.
If most operative questions are answered in a positive way, the auditor’s opinion
should be expressed in a descriptive form, namely: within the period covered by
the audit the municipality was efficiently and effectively investing in the pre-
school institution. If most operative questions are answered in a negative way, the
auditor’s opinion should be expressed in a descriptive form, namely: within the
period covered by the audit the municipality was inefficiently and ineffectively
investing in the pre-school institution. In all other cases the auditor’s opinion
should be expressed in a descriptive form, namely: within the period covered by
the audit the municipality was investing in the pre-school institution in a partly
efficient and partly effective way.

Zambia are in Annex 5

109
6 PREPARATION OF REPORT ON THE PUBLIC PROCUREMENT
AUDIT FINDINGS

6.1 Procedure of preparation and presentation of the audit

conclusion

International audit standards imply that upon completion of the main stage, it
is allowed to discuss the principal preliminary audit findings with the top
management of the Supreme Audit Institution. Such discussion guarantees a better
understanding of results achieved, the focus of conclusions made and the content
of future recommendations. During discussion, the audit team members may
express their judgment or specify problem situations that arose during the audit.
The working meeting held to enable the final discussion makes it possible to
reach an agreement regarding the problems and complicated issues identified
during the audit, which will subsequently improve the quality of the content of the
final audit report, as well as its conclusions and recommendations.
At the final stage of the Public Procurement Audit the special audit report is
prepared. in accordance to the established format of audit reports, examples of the
form and structure of which should be given in the appropriate section of the
guide.
In order to make prompt decisions on critical issues during the audit, it is also
possible to compile intermediate messages or even a brief report on specific
aspects covered by audit procedures.
Intermediate information can be presented in both oral and written form and
should not be considered as a substitute for the final audit report. Its main aim is to
attract responsible persons’ attention to essential problems and issues requiring
immediate reaction of the management. All decisions made on the basis of
intermediate information should be presented in the final audit report.
The final stage of Public Procurement Audit is post-contract analysis and
follow-up control of implementation of recommendations made on the basis of the
audit findings. Its main aim is to establish feedback, continuously monitor the

110
implementation of recommendations in the practice of the audited entity and assess
the final effect of such implementation.
Besides, special attention should be paid not so much to the process of
implementation of recommendations, but to the outcomes, which should manifest
themselves in the improvement of the internal control system efficiency,
elimination of weaknesses in the procurement process management and,
ultimately, reduction in the number of violations and errors, minimization of
unjustified budget spending and overall growth of the efficiency of procurement
activities. The choice of priorities of the post-contract analysis should be ultimately
determined by the overall strategic objective of Public Procurement Audit at the
level of the annual plan of the audit.
It is also necessary to understand that during the final stage of Public
Procurement Audit, a supreme audit institution should ultimately accomplish
internal objectives of the following three types:
 to increase the audit efficiency - recommendations made in the frame
of the audit should be practical and yield a tangible economic effect;
 to ensure the required quality control of the audit – the results of
implementation of recommendations make it possible to characterize the overall
quality of the audit and subsequently assess the activities of the entire audit team;
 to create incentives for the development and improvement of audit
methods and tools – in the process of the implementation of recommendations the
best and most efficient proposals may be monitored, which becomes the basis for
the development of the “best practice” of Public Procurement Audit.
Feedback is implemented in this case through the assessment of the effects of
a corrective action. Besides, it should be taken into account that there is an
objective time lag between the beginning of the implementation of
recommendations and the intended effect. Therefore, the assessment should be
carried out not earlier than after a year, and in most cases, after two years of the
practical implementation of recommendations. If the intended effect is expected to
be achieved earlier, such assessment may be carried out in accordance with the

111
planned schedule. To carry out the follow-up analysis and assessment of the
implementation of recommendations, special audit procedures may be developed,
the auditor’s visit to the audited entity may be planned.
In this case, the public-sector auditor carrying out such follow-up assessment
should have an adequate and fairly well-developed analytical toolkit to assess
results reached after the implementation of recommendations. The development of
such tools is a separate methodological problem, which may be solved in the frame
of specific research and practical elaboration.
The qualitative estimation scale below may be used as a basis for the follow-
up assessment of the implementation of recommendations following the outcomes
of Public Procurement Audit:

Weak process, failure to Recommendations have not been implemented or

achieve the planned results were implemented slowly, including due to
resistance from the part of the management of the
audited entity. Another scenario: from the very
beginning, the recommendations were declarative,
formal, did not take into account the specific
features of the procurement carried out by the
audited entities, and their implementation could not
give the expected result from the outset

Certain positive changes Changes have been made in the structures of

have occurred, partial effect management, internal control, internal regulation of
has been achieved procurement processes, which gave a certain result.
However, the final result significantly differs from
the targeted result

Generally, the The implementation of recommendations was

implementation was carried carried out according to the plan, all structures and
out in accordance with the processes work properly. The achieved effect is
agreed recommendations and generally consistent with expected values
considerable effect has been
achieved

The recommendation has Recommendations have become outdated and are

lost its relevance unable to give the desired effect due to changes in
external and internal environment of the audited
entity, principal changes in the legislation on public

112
 procurement, or after the expiration of deadlines

The implementation has As a rule it takes place at the initial stage of

begun, however the results implementation. The formal plans of future
have not been determined organizational and staffing changes have been
developed and approved by the management. The
timing and budget have been determined. However,
due to the fact that the implementation of
recommendations by itself have just begun, it is
impossible to assess any effect

The represented assessment scale is a framework and requires revision.

Nevertheless, after appropriate improvement, adjustment and expansion it may
serve as a basis for creation of a more functional working tool for the follow-up
assessment and analysis of the implementation of recommendations based on the
audit findings. The qualitative scale may be adjusted through adding a quantitative
assessment scale for most items.
The final data of the implementation results should be summarized, discussed
in the presence of representatives of the audited entity and represented as a special
report within the specified time. All results achieved, conclusions made and best
practices confirmed should be duly documented. Provisions of international audit
standards related to the preparation of audit documentation should be used to the
full extent for the execution of working documents and materials of the working
group engaged in the follow-up assessment of the implementation of
recommendations made on the basis of the audit findings.
In the frame of implementation of all above-listed stages of the audit, it is
possible to determine the main parameters of the quality of the audit. All necessary
steps should be taken within the audit process in accordance with the plan of the
audit and all required assessments and analytical procedures should be carried out
to allow the auditor to make a confident judgment about the fact that all the
planned activities included in the plan of the audit have been carried out.

113
 Besides, all actions in the frame of quality control, in their turn, should also
be documented and executed in accordance with the requirements of relevant audit
standards.
The review of the audit materials should be carried out before the submission
of the final auditor’s report and contain necessary comments describing both the
particularities of the audit itself and the report on the activities of the persons who
carried out the quality control.
In the event of a large-scale audit engagement the quality appraisal may be
carried out as a continuous process or in the frame of scheduled stages of the audit
at each critical point of the audit. This is necessary in order to maximize the
efficiency and effectiveness of the audit.
It should also be understood that the nature and depth of the ongoing quality
assessment is largely determined by experience and length of service of the
persons carrying out the quality control.
The audit quality is also ensured at the level of the audit team.
Quality control at the level of an audit team requires mandatory evaluation of
the work conducted, working files collected and conclusions reached at each stage
of the audit.
If necessary, the auditor has the right, depending on the circumstances, in its
sole discretion, to conduct more detailed analysis and evaluation of the quality of
the ongoing audit.
When carrying out tests and preparing requests, it is expedient to draw up a
schedule of sending requests and receipt of answers. This schedule should be
reconciled with the plan of the audit. The schedule performance should be
regularly monitored during the audit. Besides, the following requirements to the
schedule performance should be met:
 answers to requests should be exhaustive and provided on a timely basis;
 answers to requests or testing results should be reflected in the audit file
without any gaps and omissions in information.

114
 A request should be clear and unambiguous, provide insight into the problem
or question posed by the auditor and, if necessary, contain references to other
similar documents. Commentaries and answers of responsible persons of the
audited entity given as a reply to the requests should also be duly documented and
executed.

6.2 Requirements to the composition and contents of the report on

the results of Public Procurement Audit

Main requirements to an audit report are determined on the basis of general

requirements of international audit standards:
- a report should be timely, complete, accurate, objective and should contain
compelling evidence to support the validity of the conclusions made by the auditor;
- a report should contain clear and well-defined wordings, be well-structured
and contain only the information and data that meet the established criterion of
materiality.
The final report should present main data and information about the course of
the audit, main facts identified by the auditors regarding the condition of the
procurement system of the audited entity, the status of internal control, as well as
an objective assessment of the efficiency of procurements completed.
To ensure methodological unity of the process of compilation and preparation
of the final audit report, the following approximate structure thereof can be
recommended:
1 Introduction
This is the starting point of the audit report describing the main parameters
and nature of the audit engagement. Such elements as the audit goals and
objectives, the audited period, the membership of the audit team should be
reflected in this section on a mandatory basis.
In order to explain objectives and goals of the audit, it is possible to give a
brief plan thereof, characterize its main areas of the audit and determine principal
constraints associated therewith. The degree of information disclosure on the audit

115
methodology applied is determined independently by the authors of the audit
report. But anyway, such information should ensure clear and distinct
understanding of the choice of main approaches and methods of audit evidence
gathering, assessment criteria applied, including performance indicators, audit
sampling volumes. Besides, it may be explained here which type of the audit (ex-
ante, current, ex-post) was applied in each audited area, and the choice of the used
set of indicators of public procurement efficiency assessment may be substantiated.
The information on the audited entity should be separately disclosed in the
introductory part; a brief characteristic of the existing procurement system should
also be given and main parameters of the latter should be characterized here.
Besides, it is expedient to link such description with the substantiation of the audit
sampling volumes and assessment of the complexity of audit procedures pertaining
to the assessment of the internal control system efficiency at the area of
procurement.
2 Description of main results
The report should contain all necessary information about the activities of the
audit team and results obtained with respect to each of the goals and objectives of
the audit. The message should contain sufficient, adequately formulated, relevant
information, the analysis of which would allow forming an objective opinion on
each of the issues presented in the report. If necessary, it is possible to introduce
additional data and appropriate reference information enabling the reader to duly
understand and properly interpret the results of audit procedures performed. It is
possible not to include into the report a number of control results not meeting the
criterion of materiality.
3 Conclusions and findings
All conclusions and findings made by the audit team in the course of the audit
must be presented in the report. The quality and justification of the wording of
conclusions and findings are determined by the persuasiveness of the evidence
collected, the accuracy and reliability of the estimates made, as well as the results
of tests on the merits. The internal logic linking between conclusions and findings

116
and their reconciliation with goals and objectives set in the audit engagement are
of paramount importance.
4 Recomendations
On the basis of conclusions and findings achieved, the auditors must
obligatorily draw up recommendations; the existing capacity of improvement of
the management and organization system of internal control over public
procurement at the audited entity should be implemented through these
recommendations.
Recommendations may be of various nature, but anyway, they are to be
logically consistent and economically sound and should strictly comply with
applicable laws and intradepartmental regulations in the area of public
procurement.
The contents of recommendations should reflect main problems and method
of addressing them at least in the following areas of the public procurement
process:
elimination of violations identified, elimination of weaknesses in the
procurement system management;
strengthening and improvement of the internal control system;
development and refinement of procurement efficiency;
improvement of the internal regulatory framework governing the procurement
process.
Special attention should be paid to thorough elaboration of constructive
recommendations, i.e. those that are aimed at the procurement system
improvement and modernizing, addressing internal organizational and
communication processes. Such recommendations are aimed at finding the causes
of certain problems and elaboration of such mechanisms and tools of addressing
these causes that ultimately allow improving the indicators of the existing
procurement system, enhance its transparency, reliability and effectiveness.

117
 6.3 Drawing up conclusions based on the Public Procurement
Audit findings

Before the final compilation of the audit report, the audit team members
should determine the fundamental ultimate results of the audit, draw up necessary
conclusions and prepare recommendations.
Despite the fact that the sector of the audit report dedicated to conclusions and
recommendations based on the audit results is in the very end of the report, it is of
paramount importance. The information contained therein is crucial for the
development of substantiated, efficient and effective decisions aimed at the
improvement of the public procurement process.
The auditor’s final conclusions can be both general and local. General
conclusions are aimed at the improvement of the overall mechanism of public
procurement organization and may include assessments and indicators beyond the
framework of the audited entity. In particular, a conclusion on the need to improve
legislation in the entire area of public procurement might be made on the basis of
results of an audit carried out in the frame of a particular institution.
Local conclusions are aimed at the assessment of a specific public
procurement system functioning in the frame of the audited entity. Conclusions on
correctness and legality of ongoing operations, validity and rationality of
procurement, accuracy of their documentation, efficiency of the use of accepted
goods, works or services, etc. are formed on the basis of audit activities carried out
by the auditor. This approach is fully implemented in pilot projects represented in
Annex 5 to these Guide.
The conclusions reached should meet the following main requirements: they
should be justified, comply with requirements of the audit engagement and rely on
the audit assessment criteria and indicators of outcomes established at the stage of
planning. Besides, when preparing conclusions, the audit team members should
fully rely on the critical approach, which is largely based on the principle of
professional skepticism, but does not fully coincide with the latter.

118
 Understanding of the fact that arguments used in conclusions do not need to
be direct and unambiguous underlies the critical approach. The preparation and
justification of conclusions is a complicated and not necessarily unequivocal
process with its certain internal logic and content. Formation of assessments,
interpretation of audit evidence collected and analysis of results obtained is largely
subjective. Therefore, all arguments suggested during the audit and all analyzed
data should be considered from the positions of possible counterarguments or
additional information. Besides, although the auditor’s conclusions should rely on
objective, rational and specific for each particular audit engagement assessment
criteria, different, sometimes opposing viewpoints should be taken into
consideration.
From a practical point of view, all conclusions reached should be based on the
entire array of audit evidence collected and all information available for the audit
team members. To justify the conclusions, the audit report should contain direct
references to audit procedures performed, applicable regulatory framework, as
well, if necessary, to relevant confirming documents or analytical estimates.
Conclusions prepared on the basis of the audit findings may contain any
information related to the public procurement process, for example:
 description of the public procurement system, characteristics of national
programs or types and conditions of activities, in the frame of which procurements
are implemented;
 outcomes of procurement, both planned and actually achieved;
 statistical information used to estimate the probability of a positive or
negative impact of the results of national programs, in the frame of which public
procurements are implemented;
 the results of evaluation of whether financial, environmental, and social
requirements were complied with during the public procurement process, as well
as whether this process entailed unforeseen additional budgetary expenses, either
direct or indirect, etc.

119
 In order to prepare conclusions in all three areas, it is advisable to carry out a
quantitative and qualitative analysis of the information collected. Prior to the final
wording of conclusions on the basis of the findings, it is possible to carry out an
internal (in the frame of the audit team only) or extended (with the participation of
experts or representatives of stakeholders) discussion. Such a discussion allows
identifying and specifying the most relevant problem zones and adjusting the set of
estimation methods and assessment indicators used. During the discussion,
additional arguments might be elaborated in favor of adopting or refining of this or
that conclusion. A variety of factors may be taken into account: social,
environmental, economic interests, reasonable expectations of stakeholders,
additional assessment criteria, which have not been taken into consideration
before, etc.
Such discussions provide additional information to form final conclusions,
but also create the necessary basis for the preparation of recommendations based
on the audit findings. Requirements to their preparation are generally similar to the
requirements to making conclusions: recommendations should be asserted in a
logical, justified and rational way.
However, unlike conclusions, which are descriptive and retrospective by
nature, recommendations are regulatory by nature and future-oriented. They are a
tool of active influence on the activities of the audited entity and aimed at
addressing various problems that occur during the public procurement process.

Requirements to the preparation of recommendations based on the findings of

Public Procurement Audit – the experience of the Russian Federation10
Development of proposals (recommendations) is the final procedure of the process of Public
Procurement Audit results generation. If any deviations, violations and deficiencies have
been identified during the audit, and conclusions are indicative of the possibility to
significantly improve the quality and performance of the audited entities in the area

10
p. 4.3.1 of the Methodological Recommendations on Public Procurement Audit, approved by the Collegium of
the Accounts Chamber of the Russian Federation (minutes No. 15K (961) dated March 21, 2014)

120
procurement, it is necessary to prepare relevant proposals (recommendations) aimed at the
elimination thereof and at the improvement of public procurement activities of the audited
entity, which are to be included into the report on the findings of the Public Procurement
Audit, as well as to be sent as a submission or directive to the audited entity.
At this stage, it is required to:
- substantiate the need for a set of measures for systemic elimination of deviations,
violations and shortcomings, which will allow increasing the efficiency of the audited
entity’s activity in the field of procurement;
- develop proposals (recommendations) based on the results of the control activity, the
contents of which should correspond to the objectives set for the Public Procurement
Audit and be built on the findings and conclusions based on its results.
Proposals (recommendations) should be formulated so that they are:
- focused at elimination of the deviations, violations and shortcomings detected, as well as
the causes of their occurrence;
- be addressed to the audited entities;
- intended to urge the audited entities to take certain measures to eliminate the deviations,
violations and shortcomings detected;
- aimed at reaching results following implementation thereof, which can be assessed or
measured;
- sufficient and have a simple form.

121
ANNEXES

Annex 1. Main Types and Methods of Public Procurement

Annex 2. Examples of Issues to Be Addressed in the Frame of Public

Procurement Audit

Annex 3. Analysis of Principal Violations Faced in the Practice of

Public Procurement Audit

Annex 4. Identification of the Facts of Fraud, Abuse and Corruption in

Public Procurement Audit

Annex 5. Pilot Projects and Practical Examples

Annex 6. The summary of the EU CC Public Procurement Audit Guide

Annex 7. Glossary of Public Procurement Audit

122