M&E Training for AREU and the

GoA in Afghanistan
Module 9:

Basic Data Analysis for Project

Evaluations
 Module Topics
• Data Analysis Objectives
• Typical Calculations Of Indicator Values
From Collected Data
– Averages
– Percentages
– Percentage change in Indicator values (over
time or space)
• Estimating Population Results from
Survey Sample Results
• Z-Scores
• Introduction to SPSS (Optional)
 Data Analysis Objectives
• Determine Change in Indicator Values Over
Time
– Achievement of Goals and Objectives
• Understand and Interpret Reasons for the
Changes in Indicator Values
– Support Results Based Management (RBM)
– Develop “Best Practices” and “Lessons Learned”
• Document Project Outcomes and Impacts on
Beneficiaries
– Organization contribution toward meeting
Millennium Development Goals (MDGs)
• Measurements Used Are Often “Percentage
Change Since Baseline”
 Millennium Development Goals
1. Eradicate Extreme Poverty
2. Achieve Universal Primary Education
3. Promote Gender Equality And Empower
Women
4. Reduce Child Mortality
5. Improve Maternal Health
6. Combat HIV/AIDS, Malaria And Other
Diseases
7. Ensure Environmental Sustainability
8. Build A Global Partnership For Development
 Basic Analysis of Questionnaire Data
• Calculation of most indicators based on simple formulas
– Average over all HHs, e.g., average income, average production
– Totals, summed over all cases, e.g., total kg of rice produced by
participating HHs)
– Simple algebraic formulas of data elements, e.g., yield = kg/ha
• Some indicators may require complex rules
– “Adequate” access to markets
– “Adopting technology”
– HHs that practice at least “X” of “Y” promoted methods
• Disaggregation of data may be necessary to allow
consideration of subgroups of HH separately
– Gender
– Geography
– HH that meet specific conditions
 Calculating Averages
• Method
1. Add up all of the values to be averaged
2. Divide this total by the number of values
that were averaged
• Example: Find the average of :
25, 15, 33, 72, 125, 63

Answer: (25+15+33+72+125+63)/6 = 55.5

 Calculating Percentages
1. Determine what Percentage A is of B
• Formula: Percentage = (A / B) x 100
• Example: Find what percentage of 200 is 150:
Answer: Percentage = (150/200) x 100 = 75%
2. Determine what Percentage a Subset is of a
Total Set
• Formula: Percentage = (Number of items in
Subset) / (Number of items in Total Set)
• Example: What percentage of households is
headed by females if there are 25 female headed
households in a sample of 300 households.
Answer: Female headed households = (25/300) x 100 = 8.33%
of all households in the sample
 Calculating Percentage Change

• “Percentage Change” Refers to the

change from one value to another
– Value at an earlier time compared to present
– Between two groups
• Formula:
((New Value – Old Value) / (Old Value))x100
• Note That If The Old Value Is Larger Than
The New Value, The Percentage Change Will
Be Negative
 Calculating Percentage Change:
Example 1
• Baseline Average Income = USD
1000
• Midterm Average Income = USD
1250
• Percentage Change in Average
Income from Baseline to Midterm:

= ((1250 – 1000)/1000) x 100 = 25%

 Calculating Percentage Change:
Example 2
• 1995 Childhood Malnutrition Rate = 40%
• 2007 Childhood Malnutrition Rate = 25%
• The Percentage Change in Childhood
Malnutrition From 1995 to 2007:

Percentage Change = ((25-40)/40) x 100 = - 37.5%

Note that a negative (less than 0.0) change

means a decrease in value – which is
sometimes what is desired
 Calculating Percentage Change:
Example 3
• Group 1 Has 150 Members
• Group 2 Has 250 Members
• How Do the Sizes of Group 2 and
Group 1 Compare in Percentage
Terms?
Group 2 Compared to Group 1: ((250-150) /150) x 100 = 67%
Therefore, Group 2 is 67% bigger than Group1

Group 1 Compared to Group 2: ((150-250)/250) x 100= - 40%

Therefore, Group 1 is 40% Smaller than Group 2
 Indicators, Data Elements, and
Formulas: Example 1
1. Indicator: Average Annual Income of all Target
Households
• Data Element: The Income of Each (surveyed)
Household
• Formula:

(Total Annual Incomes of ALL surveyed Households)

(Total Number of surveyed Households)
 Indicators, Data Elements, and
Formulas: Example 2
2. Indicator: Average Annual Income of all
Target Households Headed by Women
• Data Elements
• The Income of Each (surveyed) Household
• The Gender of the Head of the Household
• Formula

(Total Annual Incomes of ALL Surveyed Households Headed By Women)

(Total Number of Sampled Households)
 Indicators, Data Elements, and
Formulas: Example 2- Example
• 1500 HHs surveyed
• 345 HHs headed by women

Average income of women-headed HHs

= ( Σ incomes of all surveyed women)/ 345

 Indicators, Data Elements, and
Formulas: Example 3
3. Indicator: Percentage of all Children Under 2
Years of Age in Target Households that Have
Received a Critical Inoculations
• Data Elements
• How many children under 2 years are in each household?
• How many of the children under 2 years in the household
have received the inoculation?
• Formula
(Total Number of ALL Children Under 2 Years that Were Inoculated) X 100
(Total Number of All Children Under 2 Years)
 Indicators, Data Elements, and
Formulas: Example 4
4. Indicator: Average Amount of time For Target Households to
Go from Home to Local Government Offices, Get Services,
and Return Home
• Data Elements
• Amount of time it takes each sampled household to travel from their
home to the nearest local government offices
• Amount of time it takes each sampled household to get services
• Amount of time it takes each sampled household to return home

• Formula: For Each Surveyed Household, Calculate

Total for ALL Households of (Time it usually takes to go to local

government offices + Time it usually takes to get services +
Time it usually takes to go home)

(Number of Surveyed Households)

 Indicators, Data Elements, and
Formulas: Example 5
5. Indicator: Average yield (kg/ha) of wheat for those farmers
growing wheat
– Note that average will only be calculated for wheat-growing
farmers
• Data Elements (for each surveyed farmer)
– Does the farmer grow wheat (Yes/No)?
– How many Kg of wheat did the farmer produce?
– Now many hectares (ha) of land did the farmer use to grow
wheat?
• Formula (calculate only for each farmer (i) that grows wheat)

(1) Yi = (total kgi of wheat) / (total hai used to grow wheat)

(2) Average Y = (Sum of all Yi) / (Total number of wheat farmers)

 Example 5: Calculation Details
Farmer Ha Land
Grow Kg Wheat Yield
Number Used for
Wheat? Produced (kg/ha)
(i) Wheat

1 Y 3000 2 1500
2 N --- --- ---
3 Y 7000 7 1000
4 Y 5500 5 1100
5 N --- --- ---
(1500+1000+1100)/3
Average Yield = 3600/3 = 1200
 Small Group Exercise-1
• An NGO Project Wants To Promote Goat-raising
Among Poor Households (HHs)
• Indicator: Percentage of HHs that Raise Goats
• At Project Baseline:
– 300 (HHs) were surveyed
– 75 HHs raised goats
• At Project End:
– 500 HHs were surveyed
– 250 HH raised goats
– Question: What is the percentage change that occurred
over the life of the project in the percentage of HHs that
raise goats?
 Small Group Exercise 1: Answer

• At Baseline, 75/300 = 25% of HHs

Raise Goats
• At Project End 250/500 = 50% of HHs
Raise Goats
• Percentage Increase from 25% to
50%:
((50-25)/25) x 100 = 100% Increase
 Small Group Exercise - 2
• Suppose That There Are Only 5 Farmers
That Raised Goats at Baseline AND at the
End of the Project
• The Numbers of Goats that They Had at
Baseline and the End of the Project Are
Shown on the Next Slide
• Question: What is the Percentage Change
in the Average Number of Goats Owned by
Goat-Raising Farmers Over the Life of the
Project?
 Small Group Exercise - 2:
Data Table
Number of Number of
Farmer Number
*
Goats at Goats at Project
(i)
Baseline End
1 2 5
2 3 2
3 5 8
4 1 3
5 6 5
* This calculation only makes sense for farmers that raised
goats at Baseline AND at the end of the project
 Small Group Exercise - 2:
Answer
• At Baseline the Average Number of Goats
Owned by the 5 Farmers was =
(2+3+5+1+6)/5 = 17/5 = 3.4
• At End of Project the Average Number of
Goats Owned by the 5 Farmers was
= (5+2+8+3+5)/5 = 23/5 = 4.6
• The Percentage Change from 3.4 to 4.6
= ((4.6 - 3.4)/3.4) x 100 = (1.2/3.4) x 100
= 35.3%
 Using Sample Statistics to
Estimate Population Statistics
• A Sample Should Be Chosen to Be
Representative of the Population
– Statistics calculated from a sample are
approximately the same for the population
• Percentage of female-headed households
• Average income
• Average household size
• Therefore, Statistics from a Survey
Sample Can Be Applied to the Total
Population
– Sample Statistics are Exact
– Population Statistics Derived from Sample Are
An Estimate
 Using Sample Statistics to Estimate
Population Statistics: Example
• The Project Target Population Has About 25,000
HHs
• The Sample of HHs that Was Surveyed = 500 HHs
• 100 Of The Surveyed HHs Grow Wheat
• Approximately How Many Target HHs in the Total
Population Grow Wheat?

Answer: 100/500 = 20% of Sample HHs Grow Wheat

Therefore, We Estimate That Approximately 20% of Total

Population HHs Grow Wheat = 20% x 25,000 = 5000 HHs
 Small Group Exercise
• A Project Is Trying To Increase Income For
Female-headed HHs By Training Them To Raise
and Sell Goats
• Target Population = 50,000 HHs
• Sample HHs Surveyed = 750 HHs
– 150 Female-headed HHs
– 50 Female-headed HHs raise and sell goats
– Average Female-headed HH income from goats =
USD 100
• Estimate the Number of Female-headed HHs in
the Target Population that Raise and Sell Goats
and their Average Income from Goats
Small Group Exercise: Answer
• Analysis of Sample HHs shows that:
– 150/750 = 20% of HHs are female-headed
– 50/150 = 33% of Female-headed households raise
and sell goats
– Average income = USD 100
• Therefore, in the Total Population
– Estimated 20% x 50,000 = 10,000 HHs are female-
headed
– Estimated 33% x 10,000 = 3,333 female-headed
HHs raise and sell goats
– Estimated average income = USD 100
Z-Scores
 Z-Scores: Objectives
• Convert a set of raw scores into a set
of normalized scores
• Provide a sense of relative magnitude
• Permit comparison of values from
different data sets via standardized
values
• Ensure that the ranges of all data set
values have the same magnitude
 Z-Score:
Definition
• A set of Z-Scores are created from a set of numbers by the
equation:

Zi = (Xi – μ) / σ
where μ is the average of the Xi

and

where σ is the standard deviation of the set of Xi

 Z-Scores: Example

Xi Zi Observations
9 -0.61 – Z-Scores represent the
12 0.61 “number of standard
8 -1.02 deviations from the mean
10 -0.20 of the original values”
7 -1.43
13 1.02 – Positive and negative Zi
values
11 0.20
• Xi <  implies Zi < 0.0
14 1.43
• Xi >  implies Zi > 0.0
 = 10.5  = 0.0
 = 2.45  = 1.0 – Typically, -5 < Zi < 5
 Z-Scores: Applications

• Comparison of dataset to standard values

– Malnutrition scores, standards

• Comparison of a set of geographically

based values from a single time
– Indicator values from different areas

• Comparison of a set of temporal values

from the same area
– Change in an indicator value for one area over
time
 Z-Scores: Applications to
Index Construction
• Indices typically combine variables with
different magnitude scales and different
levels of importance
– Example: Quality of Life Index evaluated for
each region of the country
• Income in USD (1,000s, 10,000s)
• Farm size in hectares (1s, 10s)
• Literacy rate for HH (0.0 - 1.0)
– Each indicator evaluated for each region
 Z-Scores: Applications to
Index Construction (continued)
• Index weights must accommodate these scales to
avoid being dominated by a few indicator variables
– Example: Quality of Life Index, QL

QL = WII + WFF + WLL

QL evaluated for each region

• By converting each set of indicator values to Z-

Scores, all indicator datasets will have same scale
of magnitudes
 Z-Scores: Applications to
Index Construction (continued)
• Index weights can reflect relative
importance of indicators without scale
issues
• These index values reflect relative
cumulative scores of different regions
– Index will represent an abstract quantity
 Application to Logframes
• The analyst must define and use equations to
link the different levels of Logframe variables
• Different types of equations may be required
in different situations
• Indicator “Weights” may need to be defined
• An abstract Index may be needed, especially
if unlike quantities are being combined or
simultaneously considered
 Application to Logframes-2

• Indicators may be calculated directly from data

elements and/or other indicators
– “Real world” equations may be possible
– Weights may be required
• To assess abstract Outcomes/Impacts in terms of
selected indicators with different levels of
importance the analyst may wish to define an index
– Weighted linear sum is most common form
• Z-Scores can be used to provide a relative index
without the complication of different variable
magnitudes
 Software for Data Analysis
• Several “common” software systems can provide
automated data analysis support
– SPSS
– Excel
– Access
• Specialized software systems can also be used
– Harmoni
– MER
• In general, all of the above software systems can
provide all of the needed analysis capabilities
– Different software systems have relative +, -
• Question and answer formatting and data entry
procedures must be keyed to the software to be used
– Common spreadsheet format (SPSS, Excel,)
– Customized or external data entry (Access)
 SPSS

• Very Powerful Commercially Available

Statistical Software
• Can Perform all Calculations Typically
Needed for Development Data
Analysis
• Training Required for its Use
 Summary
• Determine Change in Indicator Values Over Time
– Support Results Based Management (RBM)
– Develop “Best Practices” and “Lessons Learned”
• Understand and Interpret Reasons for the Change
• Most Calculations Needed for Data Analysis Are Not
Too Complex
• Practice and Experience Are Needed
• Analysis Using Z-Scores Offers Some Advantages
Over Using “Original” Data Values
• Different Software Packages Can Support Data
Analysis