Evaluation

UNIT 3 EVALUATION

Structure
3.1 Introduction
3.2 Evaluation: Meaning and Features
3.3 Types of Evaluation
3.4 Evaluation Design (How to do Evaluation?)
3.5 Various Aspects of Evaluation
3.6 Methods and Approaches of Evaluation
3.7 Let Us Sum Up
3.8 References and Selected Readings
3.9 Check Your Progress – Possible Answers

3.1 INTRODUCTION
Literally, ‘evaluation’ means ‘assessing the value of’. Evaluations are undertaken
in all spheres of life, in informal or formal ways, whenever, one wishes to know
and understand the consequences of some action or event. The acquired
knowledge and understanding are commonly used by the evaluator to perform
some activity in a better manner in the future; that is, one tries to learn from
one’s experiences in order to improve one’s performance.

Evaluation of development work may be undertaken during the implementation

of the development programme or project or after it has been completed. In
evaluation of development work, people are in focus. The overall purpose of
such evaluation is usually to document the results and the utility of the work for
individuals or groups of people in the respective society or societies as well as to
explore how the results have been created. Evaluating planned development work
means that features and processes will have to be described and explained in
relation to the plan or plans for that work. A good plan will also guide the evaluation
and facilitate collection and analysis of the information that is needed for it.

Evaluation is an assessment of end results or impact of a project with reference

to the objectives set in the project. Knowing why a programme succeeds or fails
is more important than knowing what it has done. Evaluation provides answers
to this crucial question and helps us to identify the strengths and weaknesses,
merits and demerits. A good and comprehensive evaluation should bring out
what is good/ bad about the project, what are the direct and indirect benefits and
what are the intended and un-intended outcomes. The evaluation should also
establish cause-effect relationship of project outcomes. By virtue of this,
evaluation is, generally, undertaken after the completion of a project. Evaluation
helps to refine our planning tools and future programmes.
After going through this unit, you will be able to:
• explain meaning and features of evaluation
• discuss various types of evaluation
• describe methods and approaches of evaluation
29
Monitoring and Evaluation
3.2 EVALUATION: MEANING AND FEATURES
3.2.1 Meaning of Evaluation
Evaluation is a process for determining systematically and objectively the
relevance, efficiency, effectiveness and impact of project activities in the light of
their stated objectives. Since objectives and unbiased assessment of project
outcomes are the essential ingredients of an evaluation, generally, an independent
agency is assigned the job of evaluation. It is also an organizational process for
improving activities still in progress and for aiding management in future
planning/programming and decision making. Evaluation provides information
about past or ongoing activities as a basis for modifying or redesigning future
strategies.

According to Shapiro “Evaluation is the comparison of actual project impacts

against the agreed strategic plans. It looks at what you set out to do, what you
have accomplished and how you accomplished it.” According to PSO “Evaluation
is assessing as systematically and objectively as possible an ongoing or completed
project, programme or policy. The objective is to be able to make statements
about their relevance, effectiveness, efficiency, impact and sustainability”.

3.2.2 Features of Evaluation

Some of the basic features of evaluation are as follows:
• It is always with reference to stated criteria
• It is always with reference to a point of time
• It starts where progress reporting/monitoring/estimation surveys end
• It establishes relationship between policies/methods and results
• It investigates and find out factors for success/failure and suggest remedies
• More qualitative in approach and emphasis is on variability than
standardization
• More purposive and less aggregative
• Its focus is - policy issues, problem formulation, organizational forms,
administrative practices, and extension of technical contents of the programmes,
people cooperation, attitude and impact

3.2.3 Difference between Appraisal, Monitoring and Evaluation

Before going to do evaluation, it is important to know the differences between
these three concepts ‘appraisal’, ‘monitoring’ and ‘evaluation’. Although they
are interchangeable yet distinction can be established among these three concepts.
In development terminology, appraisal is usually taken to mean a critical
examination of a proposal of a programme or project, normally before the latter
is approved for implementation and funding. Sometimes, evaluation and appraisal
are used interchanging as if they are synonymous. Appraisal deals with economic
viability, technical feasibility and social desirability of a project that is being
contemplated. It is generally undertaken for large but concentrated investment
project. Thus, appraisal is basically a planning and project formulation activity.
Appraisal may be guided by evaluation findings of similar projects / components,
which have already been implemented.
30
As far as distinction between monitoring and evaluation is concerned, both Evaluation
monitoring and evaluations are undertaken to find out how a programme or project
performs or has performed. The basic difference between monitoring and
evaluation is given in Table-3.1.

Table 3. 1: Basic Differences between Monitoring and Evaluation

Monitoring Evaluation
Continuous: Starts and ends with a One shot operation: At a point of
programme. time (usually at completion or mid
way of programme)
Required for immediate use and Used for future planning/ replication/
mid-course correction expansion
Done by implementing personnel Usually by outside agency
Quick but covers all units In-depth; covers a sample
Correcting / managing Learning process
Symptomatic, early warning system Diagnostic

The concept of evaluation can broadly be defined as a more thorough examination,

at specified points in time, of programmes or projects or parts of them, usually
with emphasis on impacts and additionally commonly on efficiency, effectiveness,
relevance, reliability and sustainability. Generally appraisal is undertaken before
the initiation of the project, monitoring during the implementation of the project
and evaluation at the end of the project. However, there are no water-tight
differences among these three concepts.

3.3 TYPES OF EVALUATION

Customarily, the evaluation can be categorized as: (i) evaluation by focus; (ii)
evaluation by agency; (iii) evaluation by stage; (iv) longitudinal evaluation; and
(v) ongoing evaluation.

i) Evaluation by Focus: It is of two types viz., formative and summative

evaluations.

Formative Evaluation: Done during the programme development stage

(Process evaluation: Mid term appraisals)

Summative Evaluation: Taken up once the programme achieves a stable state

of operation (outcome evaluation, ongoing or ex-post evaluation). Usually
conducted at the end of the project.

ii) Evaluation by Agency: It is done by internal agency as well as independent

external agency appointed by donors.

iii) Evaluation by Stage

Ongoing : During the project
Terminal: Immediately after completion of project
Ex-Post: After sometime from completion of project
31
Monitoring and Evaluation iv) Longitudinal Evaluation: It is a repeat evaluation to study the sustainability
of results/outcomes.

v) Ongoing Evaluation (Concurrent Evaluation): When the monitored

information is further analyzed and examined by the project management
(with the help of some ad-hoc or special studies) on a continuing basis, with
a view to determine the casual relationship between project input or activities
and outputs and the influence of external constraints on project performance,
it is classified as ongoing evaluation. More aptly, this is the effect and impact
monitoring exercise in programmes, which are long term in nature. The idea
here is to adjust or redefine policies, objectives, institutional arrangements
and resources affecting the project during implementation. Since the main
purpose here is to do mid- course correction, it can be taken by the project
staff also and therefore it can also be called as internal evaluation. Such
evaluation is also termed as concurrent or mid-term evaluation. Generally,
this type of evaluation is undertaken in long-term projects.

3.4 EVALUATION DESIGN (HOW TO DO

EVALUATION?)
Since the primary focus of evaluation is assessment of results with reference to
objectives i.e., end results or impact, evaluation would necessarily mean
comparison of current situation after the project intervention with the situation
that existed prior to project. The evaluation designs are of two types:

3.4.1 “Pre - Project” Versus “Post-Project” Evaluation

For obtaining pre and post-project situation, two methods could be adopted:
i) Carryout a sample survey at the end of the project and enquire from the
project beneficiaries simultaneously their present situation and the conditions
that existed prior to the project. This method, however, suffers from ‘memory
or recall bias’ of the respondents.

ii) To overcome the memory or recall bias, it is always preferable and necessary
that a ‘baseline’ or ‘bench mark’ study is carried out at the beginning of the
project on selected parameters of likely impact. A similar baseline at mid
term and at the close of project will provide us the changes brought about by
the project overtime. Unfortunately in most projects baseline studies are not
undertaken or when available it is inadequate. Hence the pre-post (before-
after design) may not give us reliable information.

3.4.2 With Project and Without Project Evaluation

Under the circumstances of ambiguity in pre-project and post-project evaluation
an alternative evaluation design needs to be adopted. This method is called “with
and without”, also called as “beneficiary - non beneficiary (Control Group)”
design. For this purpose, a comparable typical sample of beneficiaries and non-
beneficiaries can be selected and studied within the project area. Alternately, a
comparable typical area adjoining the project area where project has not been
implemented can also be studied along with project area. The key to both the
approaches is selection of comparable / typical sample.

32
In order to obtain the best results in evaluation, a combination of both the designs Evaluation
will be the most appropriate method.
Till now you have read about the meaning and features of evaluation, types of
evaluation and evaluation design. Now you try and answer the following questions
in Check Your Progress-1.

Check Your Progress 1

Note: a) Write your answer in about 50 words.
b) Check your answer with possible answers given at the end of the unit.
1) What do you mean by Evaluation?
.......................................................................................................................
.......................................................................................................................
.......................................................................................................................
.......................................................................................................................
.......................................................................................................................
.......................................................................................................................
2) Name different types of evaluations.
.......................................................................................................................
.......................................................................................................................
.......................................................................................................................
.......................................................................................................................
.......................................................................................................................
.......................................................................................................................
3) What is evaluation design?
.......................................................................................................................
.......................................................................................................................
.......................................................................................................................
.......................................................................................................................
.......................................................................................................................
.......................................................................................................................

3.5 VARIOUS ASPECTS OF EVALUATION

It is pertinent to know some of the important aspects which are evaluated when
an evaluation is carried out by any individual or an organization. Some of the
aspects of evaluation are discussed below:

33
Monitoring and Evaluation i) Efficiency Evaluation
This may be defined as the amount of outputs created and their quality in
relation to the resources (capital and personnel) invested. It is, then, a measure
of how productively the resources (as converted into inputs) have been used.
The main problem in evaluating efficiency is deciding on the amount of
various inputs which the evaluator considers to be reasonable for producing
the outputs which may have been documented in the project proposal. Specific
criteria for making such judgments are rarely given in the plan documents.
The evaluator may therefore have to make his or her own judgment about
relations between the magnitude of inputs of different kinds and the amount
and quality of outputs. Measuring the efficiency is one of the intricate tasks
before the project manager, which sometimes may convince or may not
convince the donor agencies of the project.

ii) Effectiveness Evaluation

This expresses to what extent the planned outputs; expected effects
(immediate objectives) and intended impacts (development objectives) are
being or have been produced or achieved. In practice, in effectiveness analysis,
it may be appropriate to focus mainly on the effects on the outcome side, for
two related reasons:
• the effect level is the first level at which benefits for the intended
beneficiaries are expressed, making effects a much more significant
measure of achievements than outputs; and
• being more directly derived from the inputs and activities of the respective
developmental scheme than are impacts, the effects will normally be
less influenced by intervening external factors and can, therefore, usually
be assessed quickly and more reliably.
If the immediate objectives (intended effects) have not been precisely stated
in the programme or project document, the evaluator may have to make
such further specifications by himself or herself which he or she considers
to be necessary for an effectiveness analysis, according to the evaluator’s
own best judgment.

iii) Impact Evaluation

Impact means long-term effect-positive and negative, intended and
unintended. These are the longer-term, largely indirect consequences of the
programme or project for the intended beneficiaries and any other people.
The main impacts are, of course, expected to be positive. However, there
may be negative impacts as well, on beneficiary groups or others. These
should also be analysed. Sometimes, negative impacts on some people may
have been suspected or even expected at the planning stage. These might
then have been mentioned and even specified in the plan documents, with or
without counteracting measures. In other cases, they may have been entirely
unforeseen. They may then be more difficult to trace for evaluators. Impacts
are normally generated through complex relations and processes and they,
therefore, need to be analysed through broad-focusing investigations. Impact
evaluation can use quantitative or qualitative methods or both.

34
iv) Evaluation of Sustainability Aspect Evaluation

This means the maintenance or augmentation of positive changes induced

by the programme or project after it is terminated. Sustainability may relate
to all the levels of the project intervention framework. In case of action
research project the sustainability aims at continuation of interventional
activities at all levels. In more specific terms, sustainability may mean:
• maintenance of physical facilities produced (for instance, a road);
• continued use of physical facilities (for instance, a road) or intangible
qualities (for instance, knowledge);
• continued ability to plan and manage similar development work (by
organisations which have been in charge of the programme or project or
any other organisations);
• continued production of the kinds of outputs created (for instance,
teachers from a teachers’ training college);
• maintenance of impacts created (for instance, continued improved health
due to new sanitation practices or continued ability to compete more
effectively in labors markets due to training given); and
• multiplication of effects and impacts (of the same kinds or of other kinds,
through inducements from facilities or qualities created by the programme
or project).

v) Evaluation of Quantitative and Qualitative Aspect

In the social sciences, it has become common to distinguish broadly between
two major approaches: quantitative and qualitative. The most obvious
distinction between the two is that quantitative methods produce numerical
data and qualitative methods produce information which is expressed in
words. In simple terms, the quantitative evaluation are made by using
scientific tools and measurements. The result can be measured and counted.
Compared to this, qualitative evaluations are more subjective in nature and
become harder to reproduce with accuracy.

vi) Evaluation of Result Oriented Aspects

The five main aspects in project operation and evaluation are input - activities
- output - effect (outcomes) -impact sequence and their need for monitoring
at different stages of the projects’ progress. There is change in approach, in
recent times; new approach is called Result Oriented/ Result based evaluation,
an improvement over the activities to outputs in impact approach is used.
The focus here is to ensure that resources ‘invested’ in any development
activity generate their intended results objectively as possible. Projects are
now only deemed successful, if the measured results achieve the projected
quantified indicators. This approach lays emphasis on ‘results’ and benefits
at every stage. The concept is to link development resources to the objective
of societal benefits, along with the means of achieving them, impact measures
and the risks involved in the process. This involves an integrated methodology
package to strengthen all components of the project cycle. The important
tools used are Logical Framework, problem analysis; GTZ objective oriented
analytical framework, etc.
35
Monitoring and Evaluation
3.6 METHODS AND APPROACHES OF
EVALUATION
The monitoring and evaluation is one of the important components of any project
and programme. The funding agencies always want that the money which they
sanction to the project and the purpose for which it is spend are utilized for the
purpose for which it is intended. Some of the methods and approaches used by
the World Bank (2004) for evaluation of projects are:
i) Logical Framework Approach
ii) Formal Survey Method
iii) Rapid Appraisal Method
iv) Public Expenditure Tracking Survey Method
v) Cost-Benefit and Cost-Effectiveness Analysis
vi) Participatory Method

3.6.1 Logical Framework Approach

The Logical Framework is simply a tool, which provides a structure for specifying
the components of any activity, and the logical linkages between a set of means
and a set of ends. It places the project in its larger framework of objectives within
the programme. It serves as a useful tool for defining inputs, time-tables,
assumptions for success, outputs and indicators for monitoring and evaluation
performance. The structure of the Log Frame is deceptively simple. It consists of
a 4 × 4 matrix in which the rows represent the levels of project objectives,
including the means required to achieve them (the vertical logic), while the
columns indicate how the achievement of these objectives can be verified (the
horizontal logic). The full matrix is shown in Table 3.2.

Table 3. 2: A Logical Framework Matrix

Narrative Objectively Verifiable Means of Important Assumptions

Summary Indicators (OVI) Verification (MOV)

GOAL Measures of Goal Sources of Assumptions Affecting

Achievement Information Methods Purpose-goal Linkage
Used

PURPOSE End Project Status Sources of Assumptions’ Affecting

Information Methods Output-Purpose Linkage
Used

OUTPUTS Magnitudes of Outputs Sources of Assumptions’ Affecting

Planned, Completion Information Methods Inputs-Output Purpose
Data Used Linkage

INPUTS Nature and Level of Sources of Initial Assumptions

Resources, Necessary Information about the Project
Cost, Planned Starting
Data

36
The two important component of logical frame work technique are (a) the vertical Evaluation
logic and (b) the hierarchy of project objectives.

a) The Vertical Logic: The vertical logic identifies what the project intends to
do, clarifies the relationship between project means and ends, and specifies
the uncertainties concerning both the project itself and the social/physical/
political environment within which the project is located.

b) The Hierarchy of Project Objectives: Four levels are specified in the Log
Frame:
Goal: The reason for undertaking the project: the ultimate objective of the
programme to which the specific project will contribute.
Purpose: What the project is expected to achieve in development terms once it
is completed within the allocated time. The purpose is the motivation behind the
production of the outputs.
Outputs: The specific results to be produced by the management of inputs.
Inputs: The activities to be undertaken and the resources available to produce
the outputs.
Some of the advantages of logical framework are:
a) It forces the project analyst to trace out the inter-locking components of a
project in a Logical manner right from the start. Shifts the emphasis from
inputs and outputs to objectives of the project.
b) It forces the project analyst to ask himself, from the very outset, the question
“How can I check whether or not progress is being made towards achievement
of objectives?”
c) It forces the project analyst to record systematically the risks and assumptions
inherent in the whole enterprise. Helps to minimise risk, keep a check if
assumptions are coming true.
d) Handing over to others is made easy.
e) Logical Framework is the intellectual baggage of Monitor and Evaluator.

3.6.2 Formal Survey Methods

Formal Surveys can be conducted from time to time during the project or
programme implementation period. The survey must try to collect information
with the help of a standardized tool from a carefully selected sample of people or
households. Surveys sometimes used to often collect comparable information
for a relatively large group of people in particular target groups in a project area.
The survey is intended to serve following purpose:
a) Providing baseline data against which the performance of the strategy,
programme or project can be compared.
b) Comprising different groups at a given point of time.
c) Comparing changes over time in the same group.
d) Comparing actual conditions with the changed condition of the target
established in a programme or project design.
37
Monitoring and Evaluation e) Describe prevailing conditions in a particular community or group.

Some of the advantages of this method are: (a) Its findings can be applied to
longer group (b) quantitative establishments can be made for the distribution of
impacts.

3.6.3 Rapid Appraisal Methods

Rapid Appraisal Methods are quick and low-cost means to gather information
from the beneficiaries or other stakeholders of the project area for providing
information to decision-makers. The rapid appraisal method falls between very
informal methods such as casual conversations or short-site visits and highly
formal methods such as Census, Sample Surveys or experiments.
Some of the popular methods of rapid appraisal are:
a) Key information interview
b) Focus group discussion
c) Community group interview
d) Direct observation
e) Mini-survey

The advantages of rapid appraisal method are they are low cost, can be conducted
quickly and provide flexibility to explore new ideas. As rapid appraisal methods
are short-term, they neither provide sound survey data nor do they provide in-
depth understanding of the survey.

3.6.4 Public Expenditure Tracking Survey (PETS) Method

Public Expenditure Tracking Surveys tracks the flow of public funds and
determine the extent to which resources actually reach the target groups for which
it is intended. The surveys examine the process, quantity, and timing of the release
of resources to different levels of government for example central, state, district
and local self-government institutions, particularly to the units responsible for
delivery of services. PETS is implemented as part of longer service delivery and
facility surveys which focus on the quality of service, characteristics of the
facilities, their management, incentive structure, etc.

PETS also intend to provide local communities with information about the level
of resources allocated to particular services in their area for example to the local
school or health clinic, etc and their pattern of utilization. PETS are widely used
by the International donor agencies in evaluating their social development projects
in African Countries.
The two important advantages of this method are:
a) Supports the pursuit of accountability
b) Improves management by pinpointing bureaucratic bottlenecks

3.6.5 Cost-Benefit and Cost-Effectiveness Analysis Method

The cost benefit and cost effectiveness analysis are tools used for assessing
whether or not the costs of an activity can be justified by the outcomes and
outputs. Cost benefit analysis measures both inputs and outputs in monitary terms.
38
On the other hand, cost effectiveness analysis estimates inputs in monetary terms Evaluation
and outcomes in non-monitary quantitative terms. The unique feature of cost
benefit analysis is that it can indicate explicitly whether benefits outweigh costs.
While the cost-effectiveness compare programme with similar outcomes. The
calculation of cost effectiveness is done with the help of cost-effectiveness ratio,
which is as follows:
Total Cost
Cost Effectiveness Ratio = —————————
Unit of Effectiveness

The benefit-cost ratio (BCR) represents the ratio of total benefits over total costs,
both discounted as appropriate. The formula for calculating BCR is:

PV benefits
BCR= —————
PV cost
Where:
PV benefits – Present Value of benefits
PV cost – Present value of cost

The steps followed in the cost benefit and cost effectiveness analysis are follow:
1) set the framework for the analysis
2) decide whose cost and benefit to be recorded
3) identify and categorize costs and benefits
4) calculate project costs and benefits over the life of the programme
5) monetize
6) quantify benefits in terms of units of effectiveness
7) discount costs and benefits to obtain present values
8) compute a effectiveness ratio
9) compute cost benefit ratio
One of the important advantages of cost effective method is that it is useful for
convincing policy-makers and funders that the benefits justify the activity. While
the benefit cost ratio is most suitable for the evaluation of single project.

3.6.6 Participatory Methods

Participatory Monitoring and Evaluation (PM&E) has emerged because of
recognition of the limitations of the conventional approach to M&E. It is attracting
interest from many quarters since it offers new ways of assessing and learning
from change that are more inclusive, and more in tune with the views and
aspirations of those most directly affected by the project interventions. PM&E
provides an opportunity for development organizations to focus better on their
ultimate goal of improving poor people’s lives. By broadening involvement in
identifying and analyzing change, a clearer picture can be gained of what is really
happening on the ground. It allows people to celebrate successes, and learn from
failures. For those involved, it can also be a very empowering process; since it puts
them in-charge, helps develop skills, and shows that their views counts.

39
Monitoring and Evaluation a) What is PM&E?
PM&E is not just a matter of using participatory techniques within a
conventional monitoring and evaluation setting. It is about radically rethinking
who initiates and undertakes the process, and who learns or benefits from
the findings. There are many different forms depending on who is
participating, at what stages they are involved and the precise objectives.
Community-based versions, where local people are the primary focus, sit
alongside other forms geared to engaging lower level staff in assessing how
it can be improved. PM&E has created new ways of measuring change,
while helping build the monitoring and evaluation capacity of the people
involved. Distinction between the conventional M&E approaches and
participatory M&E approached is given in Table-3.3 below.
Table 3.3: Conventional M & E approaches Vs. Participatory M&E Approaches

Parameter Conventional M&E Participatory M&E

Who plans and Senior managers, or Local people, project staff,
manages the outside experts. managers and other stakeholders,
process often helped by a facilitator.
Role of ‘primary Provide information only. Design and adapt the methodology,
stakeholders’ (the Externally-defined, collect and analyses data, share
i n t e n d e d mainly quantitative findings and link them to action.
beneficiaries) indicators. Internally-defined indicators,
How success is including more qualitative
measured judgments.
Approach Predetermined. Adaptive.

b) The Principles of PM&E

PM&E provides the opportunity to analyse both the qualitative and
quantitative information, thus providing more complete information on which
to base decisions. Often, an information system focuses on only (quantitative)
information, but numbers alone produce an incomplete picture of which is
really taking place in the community. If the “story behind the numbers” is
available to people, a different evaluation may be possible. PM&E has been
built on a number of overall principles which will help understand the PM&E
approach (Table 3.4).
Table 3.4: Participatory Monitoring and Evaluation Approach
Tools chosen to fit community: Information that is useful rather than perfect:
PM&E seeks to find the appropriate It is often only necessary to have information
tool for each unique community. that shows trends rather than statistically
exact information. Information showing
trends and delivered on time is often more
useful to decision-making than precise
information delivered late.
Outsiders help community to focus Supports existing community skills:
on a specific issue: Outsiders PM&E builds on existing community
facilitate but do not direct. skills.
Attitude of partnership: Insiders and
outsiders can both contribute to
community development.
40
 Evaluation
Essential information: The community is the final evaluator:
PM&E seeks to identify information Since it is the community alone who decides
that is “necessary to know” rather whether or not to continue the project, they
than “nice to know”. This ensures are the final and most important evaluators.
that the information system is not
overloaded with unnecessary
information.

Two way communication and clear People who make decisions need timely,
messages: reliable and useful information:
PM&E is built on ideas, methods and The right information at the right time will
tools that support equal and clear facilitate better decisions.
communication between insiders and
outsiders.

c) What are the Steps or Stages of PM&E?

There are four major steps or stages of applying PM&E in practice:
• Planning or establishing the framework for a PM&E process, including
identification of objectives and indicators
• Gathering data
• Data analysis
• Documentation, reporting and sharing of information
d) PRA and PRA - related techniques
Participatory assessment also known as ‘participatory rural appraisal’ (PRA)
and ‘participatory rapid assessment’ (PRA), comprises a range of
visualization, interviewing and group work methods. These techniques have
proven valuable in enabling people to express their views and share
information, in uncovering their realities and priorities, and in stimulating
discussion and analysis. Following methods are used:

i) Methods used in participatory rural appraisal (PRA) Visualized Analysis:

Participatory mapping and modeling Time lines and chronologies
Aerial photograph analyses Matrix scoring and preference ranking
Transect and group walks Venn and network diagramming
Seasonal calendars Flow diagrams on systems and impact
Daily and activity profiles Pie diagrams
Historical profiles and trend analyses
ii) Interviewing and sampling methods:
Semi-structured interviewing Ethno-histories
Direct observation
Focus group Well-being and wealth ranking
Key informants Social maps

41
Monitoring and Evaluation iii) Group and team dynamics method:

Team contracts
Team review sessions
Interview guides
Rapid report writing
Work sharing (taking part in local activities)
Villager and shared presentations
Process notes arid diaries
Some of the advantages of participatory evaluation method are:
a) Examines relevant issues by involving key players in the design process
b) Establishes partnership and local ownership of the project
c) Enhances local learning, management capacity and skills.
In this section you have read about the various aspects of evaluation and techniques
of evaluation. Now try and answer the questions in Check Your Progress-2.

Check Your Progress 2

Note: a) Write your answer in about 50 words.
b) Check your answer with possible answers given at the end of the unit.
1) What is participatory evaluation?
.......................................................................................................................
.......................................................................................................................
.......................................................................................................................
.......................................................................................................................
.......................................................................................................................
.......................................................................................................................
.......................................................................................................................
2) Name different aspects of evaluation?
.......................................................................................................................
.......................................................................................................................
.......................................................................................................................
.......................................................................................................................
.......................................................................................................................
.......................................................................................................................
.......................................................................................................................
.......................................................................................................................
42
 Evaluation
3.7 LET US SUM UP
The basic meaning, principles, design and use of evaluation have been briefed
and discussed in this unit. Evaluation has been neglected and mistakenly seen as
a ‘fault finding’ mechanism. It is to be understood evaluation as a ‘learning
process’ and our past experiences guide us to a better future. Unless we know
our strengths and weaknesses of actions and outcomes, which an evaluation helps
to bring out, we cannot build a better tomorrow. Over the years, evaluation has
evolved into a community based participatory exercise than being an externally
guided activity. Several illustrations have been given in this unit to facilitate
easy understanding of the concept.

3.8 REFERENCES AND SELECTED READINGS

ADB (2002), Project Performance Management System Operations Manual,
Section 22.
Anees, Arul (2000), Process Evaluation of Rural Development Programmes,
Authors Press, Delhi
D.J. Casley and D A Lury (1982), A Hand Book on Monitoring and Evaluation
of Agriculture and Rural Development Project, The Word Bank
Davis-Case, D. and P. Grove. (1990), The Community’s Tool Box: The Idea,
Methods, and Tools for Participatory Assessment, Monitoring and Evaluation
in Community Forestry. Rome: FAO
Davis-Case, D’Arcy (1990), Community Forestry. Participatory Assessment,
Monitoring and Evaluation.
Feuerstein, Marie-Therese. (1986), Partners in Evaluation. Evaluating
Development and Community Programmes with Participants, Macmillan
Education Ltd, Hong Kong.
Gosling, Louisa and Mike Edwards (1995), Toolkits. A Practical Guide to
Assessment, Monitoring, Review and Evaluati, Save the Children, London.
Guijt, I. (1998), Participatory Monitoring and Impact Assessment of Sustainable
Agriculture Initiatives: An Introduction to the Key Elements, Discussion Paper
No.1, Sustainable Agriculture and Rural Livelihoods Programmes, IIED, UK: IIED
http://Inweb18.worldbank.org/oed/oeddoclib.nsf/24cc3bb1f94ae11c
85256808006a0046/ a5efbb5d7
http://www.adb./org/Documents/Manuals/Operations/om22.asp.
Mukherjee, Amitava, (2004), Participatory learning and action: monitoring and
evaluation and participatory monitoring and evaluation: essays in Honour of
Robert Chambers, Concept Publishing Company, New Delhi.
Oakley Peter (1989), Community Involvement in Health Development – An
Examination of the Critical Issues, World Health Organisation, Geneva.
Participatory Monitoring and Evaluation: A Handbook for Training Field
Workers. 1998 (1990), Bangkok: Regional Office for Asia and the Pacific (RAPA),
FAO.
43
Monitoring and Evaluation Reider Dale, (2004), Evaluation Development Programmes and Projects, Sage
Publication, Delhi.
Somesh Kumar, (2002), Methods for Community Participation. A complete Guide
for Practitioners, Vistaar Publilcations, New Delhi.
World Bank (2004), Monitoring & Evaluation: Some Tools, Methods and
Approaches, The World Bank, Washington, DC, USA.

3.9 CHECK YOUR PROGRESS – POSSIBLE

ANSWERS
Check Your Progress 1
1) Evaluation is a process for determining systematically and objectively the
relevance, efficiency, effectiveness and impact of project activities in the
light of their stated objective.
2) Customarily the evaluation can be categorized as (i) evaluation by focus – it
is 2 types’ formative and summative evaluation (ii) evaluation by agency – it
is 2 types like: internal and independent evaluation (iii) evaluation by stage
– ongoing, terminal and ex-post evaluation; (iv) longitudinal evaluation;
and (v) ongoing evaluation.
3) The primary focus of evaluation design is assessment of results with reference
to objectives i.e., end results or impact; evaluation would necessarily mean
comparison with situation that existed prior to project. The evaluation designs
are of two types: “Pre - Project” Versus “Post-Project” Evaluation and With-
Without Project design. In other words, the evaluation design aims at
measuring what evaluation intends to measure.
Check Your Progress 2
1) Participatory evaluation is a means of achieving the wider objectives of
community participation projects. Community involvement will enable
evaluation to be used as a learning device – a form of fundamental education
which can contribute to empowerment and partnership. Evaluation needs to
be seen in a changing light: from being commonly a tool for judging project
managers, it can be transformed into a shared process of community self –
assessment and project improvement.
2) It is pertinent to know some of the aspects which are evaluated when an
evaluation is carried out by any individual or an organization. The various
aspects of evaluation are: efficiency evaluation, effectiveness evaluation,
impact evaluation, evaluation of sustainability aspect, evaluation of
qualitative and quantitative aspect, and last one is evaluation of result oriented
aspect.

44