The Limits of Nonprofit

Impact:
A Contingency Framework for
Measuring Social Performance
Alnoor Ebrahim
V. Kasturi Rangan

Working Paper
10-099

Copyright 2010 by Alnoor Ebrahim and V. Kasturi Rangan

Working papers are in draft form. This working paper is distributed for purposes of comment and
discussion only. It may not be reproduced without permission of the copyright holder. Copies of working
papers are available from the author.

Ebrahim and Rangan, May 2010

THE LIMITS OF NONPROFIT IMPACT:

A Contingency Framework For Measuring Social Performance

Alnoor Ebrahim and V. Kasturi Rangan

Social Enterprise Initiative, Harvard Business School

ABSTRACT
Leaders of organizations in the social sector are under growing pressure to demonstrate
their impacts on pressing societal problems such as global poverty. We review the debates
around performance and impact, drawing on three literatures: strategic philanthropy, nonprofit
management, and international development. We then develop a contingency framework for
measuring results, suggesting that some organizations should measure long-term impacts, while
others should focus on shorter-term outputs and outcomes. In closing, we discuss the
implications of our analysis for future research on performance management.

Key words: performance measurement, impact, nonprofit management, social enterprise,

philanthropy, accountability, management control systems.

Ebrahim and Rangan, May 2010

INTRODUCTION
The world of nonprofit organizations, philanthropy, and social enterprise has been
preoccupied with two powerful mantras in recent years. Since the early 1990s, the refrain of
accountability has been ascendant, with demands from funders, taxpayers, and concerned
citizens and clients for nonprofits to be more transparent about their fundraising and spending,
how they are governed, and what they have achieved with the resources entrusted to them
(Ebrahim & Weisband, 2007; Gibelman & Gelman, 2001; 2008; Kearns, 1996; Panel on the
Nonprofit Sector, 2005, 2007; Young, Bania, & Bailey, 1996). A more recent manifestation of
this discourse has centered on the mantra of impact, or demonstrating results in addressing
complex social problems such as poverty and inequality (Brest & Harvey, 2008; Crutchfield &
Grant, 2008; Monitor Institute, 2009; Paton, 2003). This attention to impact, following on the
heels of accountability, is driven both by funders who want to know whether their funds are
making a difference or might be better spent elsewhere, as well as by committed nonprofit
leaders and social entrepreneurs looking for solutions to pressing societal problems. It is also
driven by an increasing professionalization of the sector, which has led to the emergence of
common administrative norms including the use of credentialed experts such as auditors and
evaluators (Hwang & Powell, 2009; Power, 1999).
In this paper we offer a critical look at the debates around impact and their implications
for performance management in nonprofit organizations. What kinds of performance metrics do
nonprofit managers need in order to assess impact? What are some examples of good practice
and innovation in the field? What are the implications for new research and management theory
on the social sector? Although the term impact has become part of the everyday lexicon on
nonprofits and philanthropy in recent years, with frequent references to high-impact nonprofits

Ebrahim and Rangan, May 2010

or impact philanthropy, it has not been consistently defined. Borrowing from an established
literature in international development and evaluation, we use the term to refer to significant or
lasting changes in peoples lives, brought about by a given action or series of actions (Roche,
1999: 21). Many definitions of impact refer to a logic chain of results in which organizational
inputs and activities lead to a series of outputs, outcomes, and ultimately to a set of societal
impacts. More recently, the term has also come to be associated with results that target the root
causes of a social problem (Crutchfield & Grant, 2008: 24). We lay out our understanding of
these distinctions in a basic logic model in Table 1.
-----------------------------Insert Table 1 about here
-----------------------------The social sector is in the midst of a search for metrics of impact. Our contribution in this
paper is to provide a synthesis of the current debates and, in so doing, to offer a typology and
contingency framework for measuring social performance. Our contingency approach suggests
that given the varied work, aims, and capacities of social sector organizations some
organizations should be measuring long-term impacts, while others should stick to measuring
shorter-term results. We offer a logic for determining which kinds of measures are appropriate,
as driven by the mission and goals of the organization.
The implications for theory building are significant. This work aims to contribute to two
bodies of knowledge in the social sector: performance measurement and management control
systems. Most models of performance in the for-profit sector are driven by measures of profit or
shareholder wealth, typically captured in profitability ratios such as profit margin, return on
assets, and return on equity, as well as other ratios for publicly traded companies such as price

Ebrahim and Rangan, May 2010

to-earnings or earnings per share. These measures may be supported by non-financial measures
such as key performance indicators (KPIs) or balanced scorecards that include metrics on
customers, internal processes, and learning and growth (Kaplan, 2001; Kaplan & Norton, 1996).
In the social sector, financial ratios are no doubt important (e.g., the profit margin becomes a
measure of surplus that can be used to support future activities rather than dividends to
shareholders), but they vary widely depending on industry (Anthony & Young, 2004) and are
increasingly being questioned in terms of their helpfulness, even by nonprofit ratings agencies
(Philanthropy Action, 2009). And they fall far short of meeting the needs of mission-based
performance, given that financial performance is often a means rather than an end to social sector
activity.
Our normative argument is that it is not feasible, or even desirable, for all organizations
to develop metrics at all levels on the logic chain. The more important challenge is one of
alignment: building systems and structures for measurement that support the achievement of
organizational mission, especially the goals that an organization can reasonably control or
influence. We contend that organizational efforts extending beyond this scope are a
misallocation of scarce resources.
Like the literature on performance measurement, research on management control
systems in the social sector is at an early stage of development. In the for-profit sector, there is
an established body of knowledge that links key elements of management control to business
strategy, most notably on financial management (Anthony & Young, 2002), levers of control
available to managers (Simons, 1995), balanced scorecards and strategy maps (Kaplan & Norton,
1996, 2004), performance rewards (Epstein & Manzoni, 2008), aspiration levels (Greve, 2003),
as well as on incentives including personnel and cultural controls (Bruns Jr., 1992; Merchant &

Ebrahim and Rangan, May 2010

Van der Stede, 2007; Meyer, 2002). This paper begins to address this gap in knowledge in the
social sector, and suggests an agenda for research that will be essential to pursue if management
theory is to be relevant to the measurement and control challenges facing social sector
organizations.
The paper is divided into three main sections. First, we take a look at the debates around
performance and impact in nonprofits, drawing on a range of literature. These debates are
centered largely in practitioner communities and have received little scholarly attention. Second,
we articulate a contingency framework for measuring social performance. We hope this
framework will be helpful to professionals in the social sector in critically differentiating among
the types of results they can and should measure. We discuss the implications of our analysis for
nonprofit leaders and their funders. And finally, we explore the implications of this work for
further research and theory development on performance measurement and management control
systems.
THREE LITERATURES AND DEBATES
ON PERFORMANCE MEASUREMENT AND IMPACT
Debates on measuring outcomes and impacts are playing out in three arenas that we
discuss below: in private foundations aiming to be more strategic about their philanthropy; in
U.S. nonprofit organizations in response to pressures from foundations and government; and,
among international development organizations such as bilateral government agencies and
nongovernmental organizations (NGOs) seeking to improve development effectiveness. The
pressures to demonstrate impact are likely to increase across all of these players in times of

Ebrahim and Rangan, May 2010

economic crisis, as public and private resources diminish and as competition for existing
resources heightens.
There are a couple of caveats to our review of the debates. First, much of the literature on
this topic in the social sector is under-theorized and in need of conceptual framing. As a result,
the bulk of the material we cite is not from academic journals; instead, it is taken directly from
practice from influential think tanks, government agencies, and funding organizations that
are shaping the terms of the debate on social performance. The academic literature in nonprofit
studies, philanthropy, and management lags behind in providing theoretical and analytical
insights to this burgeoning field. Second, because of the vastness and diversity of the social
sector, we focus our discussion on organizations working broadly in the field of poverty
alleviation. This field is wide but well-established, with an increasingly coherent set of debates
on performance measurement. And third, while there are many conceptual overlaps across the
three literatures discussed below, we introduce them separately in order to capture them as
distinct communities of practice that are only now beginning to interact. We bring together key
cross-cutting themes about measurement in the subsequent discussion.
Private Foundations: Strategic Philanthropy
Impact measurement debates have taken center-stage in the world of U.S. private
philanthropy, as foundations place renewed emphasis on measuring the impacts of their
grantmaking and thinking more strategically about their giving (Frumkin, 2006; Porter &
Kramer, 1999). A number of relatively young entrants to the funding field The Bill and
Melinda Gates Foundation, Robin Hood Foundation, and Acumen Fund, to name just a few of
the more visible players have advocated an explicit emphasis on measurement (Alliance,

Ebrahim and Rangan, May 2010

2009; Trelstad, 2008; Weinstein, 2009). Leadership changes at more established institutions,
such as the Ford Foundation, have also been accompanied by new attention to metrics. The basic
issues and motivation are captured by Paul Brest and Hal Harvey, the presidents, respectively, of
The William and Flora Hewlett Foundation and the ClimateWorks Foundation:
[A]ccomplishing philanthropic goals requires having great clarity about what those goals are and
specifying indicators of success before beginning a philanthropic project. It requires designing
and then implementing a plan commensurate with the resources committed to it. This, in turn,
requires an empirical, evidence-based understanding of the external world in which the plan will
operate. And it requires attending carefully to milestones to determine whether you are on the
path to success with a keen eye for signals that call for midcourse corrections. These factors are
the necessary parts of what we regard as the essential core of strategic philanthropythe concern
with impact. (Brest & Harvey, 2008: 7, emphasis in the original)

This core concern with impact has fueled efforts to quantify it in order to allocate funding
among competing possibilities what Brest and Harvey have called impact on steroids. But
according to some observers, foundations have a mixed record in using measurement and
evaluations. In an analysis of evaluation methods used by philanthropic foundations from the
early 1970s onwards, nonprofit historian Peter Dobkin Hall has argued that such evaluations lack
rigor, and that key decision makers are often indifferent to the findings (Hall, 2005: 33). Others
have suggested that philanthropic giving is often motivated by expressive interests of donors and
not necessarily by evidence of what works and what doesnt (Frumkin, 2006).
It is an empirical question whether the new push towards measurement of impact will be
any different. What is apparent is the emergence, over the past decade, of a range of tools and
approaches to measuring social value creation, such as the pioneering work on social return on

Ebrahim and Rangan, May 2010

investment carried out by REDF (formerly The Roberts Enterprise Development Fund), Acumen
Funds best available charitable option (BACO) methodology, Robin Hood Foundations
benefit-cost ratios, and several other related approaches (Kramer, 2005; Tuan, 2008). Table 2
provides a listing of these methodologies, along with many others discussed below. These
methodologies are influenced by concepts from the business world, particularly notions of
internal rate of return or expected return. They rely on numerous assumptions: what to measure,
how to quantify the benefits and costs of an intervention, and how much of any impact can be
attributed to the investment made by the funder. A key challenge in such quantification and
attribution lies in addressing the thorny issue of causality: impacts are likely to be affected by
multiple factors and actors, and that attribution may be possible in some interventions such as in
the provision of shelter, food, and job training, but much less likely in complex programs such as
those targeting civil and human rights.
------------------------------Insert Table 2 about here
------------------------------Proponents of such quantification readily acknowledge these problems, arguing instead
that it is the discipline of assessment that matters rather than the actual numbers, pressing one to
clarify goals, to be explicit about assumptions, and to consider risks and limitations. For
example, in an effort to invite dialogue and critique from their peers, Acumen Fund and the
Robin Hood Foundation have explicitly documented their metrics methodologies. They argue
that all funders necessarily rank order potential grants or investments anyways, but that
quantitative methodologies make the decision process more transparent and accountable, without
which the basis for grant allocation remains opaque and, therefore, largely beyond effective

Ebrahim and Rangan, May 2010

challenge (Weinstein, 2009: 23). They also worry about the misuse of metrics, warning that a
culture of measurement matters more than systems, that information should not be confused with
judgment, and that it is sometimes more feasible to focus on metrics of outputs and immediate
outcomes rather than longer term outcomes or impacts (Trelstad, 2008: 112-114). This last point,
on whether to focus on outputs or outcomes, is highly contested, with very few actors pushing
back on the tide to measure outcomes and impacts.
Many foundations continue to struggle with how to integrate a range of measurement
approaches into their decision making. A 2007 survey of emerging approaches to evaluation in
philanthropy found that there has been a pronounced shift from the use of evaluation to measure
the impact of past grants and toward a more timely and pragmatic process of gathering forwardlooking information that will enable both grantors and grantees to make ongoing improvements
in their work (Kramer, Graves, Hirschhorn, & Fiske, 2007: 5). The report finds this tension to
be fairly common in foundations, motivated on one hand by a need to identify the long-term
outcomes and impacts of past projects, while also seeking more flexible and timely sources of
data to help with present performance.
At the same time, two broader dilemmas remain. First, the growing emphasis on
quantifiable metrics raises an old concern that has long plagued the social sector could this
emphasis bias foundation grant-making in favor of work that can readily be measured, such as
delivery of services in education or healthcare, at the expense of work where the outcomes are
harder to measure, such as in policy advocacy, democratization, or civil rights? And second, the
new measurement approaches are oriented primarily towards assessing the performance of
grantees and selecting among them to get more bang for the buck. They do little to assess the

Ebrahim and Rangan, May 2010

performance of foundations themselves, particularly when having an impact might require

strategic coordination among grantees and other foundations, rather than atomized investments.
The challenges for management theory are closely tied to these practical dilemmas. First,
there is a need to better conceptualize various types of performance and what kinds of metrics
might be useful to managers not only for assessing what they have achieved, but also for
motivating continuous improvement. Second, management theory can lend insight to the
problem of results aggregation and benchmarking. Because every nonprofit approaches a social
problem in its own way, how are institutional funders and individual donors to select among
nonprofit organizations to support, or to assess the cumulative impacts of their work? This
challenge is not simply about measurement but is also about strategy.
Nonprofit Organizations in the United States: Outcome Measurement
Within the U.S. nonprofit sector, there has been increasing attention to questions of
impact, partly in response to the basic challenge from donors, trustees, regulators and taxpayers:
Can you prove that you are making a difference? There has been a steady shift over the past
two decades among nonprofits to try to measure performance at all levels on the logic chain:
from inputs, activities and outputs that are under their direct control, to measuring the broader
outcomes and impacts of their work.
A key proponent of outcome measurement in the United States, since the early 1990s, has
been the United Way of America. It was one of the first national agencies to ask members of its
network to distinguish between outputs and outcomes, supported by a series of resources
designed to assist agencies in outcome measurement. Based on a survey it conducted of 391
agencies in 2000, the United Way noted that an overwhelming proportion of its partner

10

Ebrahim and Rangan, May 2010

organizations found outcome measurement useful for communicating results and identifying
effective practices (84-88%), as well as for helping to improve service delivery of programs
(76%). But the results were not all positive: a significant number also reported that
implementing outcome measurement has led to a focus on measurable outcomes at the expense
of other important results (46%), has overloaded the organizations record-keeping capacity
(55%), and that there remains uncertainty about how to make program changes based on
identified strengths and weaknesses (42%) (United Way of America, 2000).
At the same time, nonprofits are under increasing governmental scrutiny to demonstrate
impact and improve accountability to the public. The Government Performance and Results Act
of 1993 led many state agencies to develop milestones and adopt performance-based contracts
with service providers, including nonprofit agencies, as part of broader efforts to reinvent
government (Behn, 2001; Poister, 2003: xv). In a report to the U.S. Congress, and motivated by
concerns raised by the U.S. Senate Finance Committee in the wake of Sarbanes Oxley
legislation, the independent Panel on the Nonprofit Sector recommended that every charitable
organization should provide more detailed information about its operations, including methods
it uses to evaluate the outcomes of its programs, to the public through its annual report, website,
and other means (Panel on the Nonprofit Sector, 2005: 5, 37). At about the same time, the
Urban Institute and the Center for What Works developed a common outcome framework and a
series of indicators for 14 categories of nonprofits, as well as a set of tools for assisting
nonprofits with outcome management (Urban Institute & Center for What Works, 2006). A
broader set of performance management tools has also gained traction over the past decade, most
notably the balanced scorecard, which aims to help managers better monitor performance against
strategic goals by integrating a range of financial and non-financial measures (Kaplan, 2001;

11

Ebrahim and Rangan, May 2010

Kaplan & Norton, 1996). Some nonprofit funders, such as New Profit Inc., require their portfolio
organizations to use this tool as a way of thinking more systematically about performance.
The growth in attention to outcomes has also given rise to a chorus of skeptical voices,
particularly from practitioners. Some have suggested that while outcome measurement appears
to be a good tool to help funders see what bang theyre getting for their buck it runs the risk of
being counterproductive in the long run, both by drawing precious resources away from services
and by putting too much emphasis on outcomes for which the causal links are unclear (Glasrud,
2001: 35), thus reflecting more of an obsession with upward accountability to funders than an
interest in actually finding ways of improving services and results (Torjman, 1999: 5). Case
studies of 36 nonprofits conducted by the Independent Sector and the Urban Institute noted that
only about half of these organizations undertook some form of data analysis to help improve
programs (Morley, Vinson, & Hatry, 2001). Similarly, The James Irvine Foundations efforts to
assist nonprofit agencies in California to improve systems for gathering and assessing data on
performance outcomes concluded that establishing these systems alone was not good enough.
In the end, the projects success had less to do with whether measurement systems were
developed and more to do with whether the organizations were able to create a culture that
valued the process of self-evaluation (Hernndez & Visher, 2001: 2).
To be fair, there is some evidence outcome measurement can provide significant benefits,
both in terms of helping nonprofit staff to think in terms of broader impacts rather than simply in
terms of outputs, and also with respect to catalyzing staff energy towards important goals
(Torjman, 1999; United Way of America, 1996b). A study of thirty leading U.S. nonprofits
found that measurement was most useful when nonprofits set measurable goals linked to mission
(rather than trying to measure mission directly), kept measures simple and easy to communicate,

12

Ebrahim and Rangan, May 2010

and selected measures that created a culture of accountability and common purpose in the
organization, thus helping to align the work of disparate units and chapters (Sawhill &
Williamson, 2001).
All of this literature points to two basic tensions confronting nonprofit managers. First,
nonprofits that focus on measuring results at the front end of the logic chain (inputs and outputs)
risk being seen as failing to be accountable, failing to convince funders and citizens that they are
making a difference. Those that do try to demonstrate broader societal outcomes and impacts risk
overreaching by taking credit for social changes beyond their actual control (Campbell, 2002). A
key implication for management theory is that it will be necessary to get a better handle on this
problem of control where the long-term goals that managers have for their organizations (e.g.,
poverty alleviation) might be well beyond the boundaries of their organizations.
Second, for measurement to influence actual performance, it likely requires explicit staff
skills and organizational capacities, ranging from analytical and research skills, to processes for
quality management, benchmarking, and adaptive learning (Blumenthal, 2003; Epstein &
Buhovac, 2009; Letts, Ryan, & Grossman, 1999; Light, 2004; Saul, 2004). Yet, it is these
capacities that tend to be underfunded. For management scholars, these tensions point to a need
for research on how to design performance measures and systems when the problem of causality
arises i.e., when the ultimate social goals of the organization (e.g., eliminating poverty in a
region) is affected by multiple factors that cannot be easily disentangled. We return to these
challenges in developing a contingency framework for measuring results.

13

Ebrahim and Rangan, May 2010

International Development: Impact Evaluation and Results Aggregation

A third and more established literature on performance measurement exists in the field of
international development, particularly on impact evaluation (IE), and monitoring and evaluation
(M&E). The former, as the nomenclature suggests, typically refers to the assessment of end
results or impacts, while the latter is oriented towards informing mid-course correction by
tracking the progress of a project or program. Since the 1950s, the international development
field has seen various approaches to IE and M&E come in and out of fashion, with logic models
and frameworks having become ubiquitous in the past 40 years (Roche, 1999: 18-20),
accompanied by the growth of a global industry of international development professionals,
particularly consultants and evaluators.
The debates on measurement in international development are driven in significant part
by the needs of national governments and their funders. These actors include bilateral aid
agencies such as the United States Agency for International Development (USAID) or the United
Kingdoms Department for International Development (DfID) and their counterparts elsewhere,
as well as by multilateral institutions such as the World Bank and various agencies of United
Nations. An extensive review of the evaluation literature in development agencies found that
many agencies use the term IE to refer to evaluations involving a counterfactual what would
have happened in the absence of an intervention and consider experimental designs using
randomized control trials (RCTs) to be the gold standard for assessing impact (Asian
Development Bank, 2004; European Evaluation Society, 2007; as cited in Jones, 2009).
The RCT methodology is best known for its use in clinical field trials of new
medications, where it is fairly straightforward to separate those receiving treatment from those

14

Ebrahim and Rangan, May 2010

that do not, and where the interventions are discrete and homogenous (White, 2009). Where
random selection of program beneficiaries is not possible, quasi-experimental methods are
preferred, for example, by collecting data on a group that is demographically similar but did not
receive the intervention. The proponents of experimental designs have argued that multilateral
development banks and UN agencies already spend considerable sums on operational
assessments of projects (i.e., M&E) but not nearly enough on impact evaluations, and that there
is much greater need for methodological rigor in order to better understand what works and what
doesnt in development (Banerjee, 2007; Center for Global Development, 2006).
On the other hand, the rise of experimental approaches to IE has been challenged by a
number of development practitioners and scholars who urge greater methodological pluralism.
They contend that rigor in impact evaluation requires multiple types of comparisons and
triangulation, supported by explicit linkages to theories of change and action (Chambers, Karlan,
Ravallion, & Rogers, 2009; Khagram, Thomas, Lucero, & Mathes, 2009). Randomized control
trials may be well suited to activities such as the provision of vaccines, conditional cash transfers
to the poor, or the distribution of new seed varieties to farmers, but less appropriate for activities
where a comparison group is hard to isolate such as in policy advocacy, macroeconomic policy
changes, reforms in labor markets, or even some investments in infrastructure (Jones, Jones,
Steer, & Datta, 2009; Prowse, 2007). Nor are they suited to examining complicated development
pathways which involve multiple, interacting, and non-linear causal factors (Rogers, 2009;
2007).
The emerging literature in this arena offers new approaches to evaluation, seeking to
marry methodological rigor with techniques that lend themselves to a variety of complex and
non-linear contexts, and which serve both IE and M&E functions such as case and

15

Ebrahim and Rangan, May 2010

comparative case study designs involving the use of process tracing and structured-focused
comparison (Brady & Collier, 2004; George & Bennett, 2005; Gerring, 2007), and various
participatory and adaptive learning and planning approaches such as outcome mapping,
constituency feedback, and most significant changes techniques, as well as complexity science
and systems thinking (ActionAid International, 2006; Bonbright, Campbell, & Nguyen, 2009;
Davies & Dart, 2005; Earl, Carden, & Smutylo, 2001; Guijt, 2008; Ramalingam, Jones, Reba, &
Young, 2008). Some of these methods draw on recent advances in comparative case study
designs that enable cautious causal inferences to be drawn while also elaborating interactive
causal mechanisms and processes (Khagram et al., 2009). They are also a partial response to the
Paris Declaration on Aid Effectiveness, an international agreement signed in 2005, and the Accra
Agenda for Action in 2008, both of which emphasized not only the need to measure results, but
to do so in a way that reflects national priorities and helps build country systems for managing
development.1 A primary contention of this new suite of evaluation approaches is that isolating
causal factors and measuring impact is not enough, even in the limited circumstances when it is
feasible; rather, policymakers and development managers need an integrated set of assessment
methods that can help them build, refine, and adapt their interventions in real time.
There is also an emergent measurement discussion on the aggregation of development
results. Bilateral agencies that support hundreds, if not thousands, of projects through
governments and civil society organizations around the world are looking for ways to assess the
sum total of results that they have achieved. Their interests in doing so are twofold: to respond to
accountability demands from their own finance ministries and taxpayers who want to know the
impacts of their aid, and to increase development effectiveness and progress towards global
targets such as the Millennium Development Goals. The civil society units within donor agencies

16

Ebrahim and Rangan, May 2010

began to explore this challenge in 2009, launching a roundtable organized by the Canadian
International Development Agency with its counterparts from Denmark, the United Kingdom,
Sweden, the Netherlands, and Norway (Canadian International Development Agency, 2009a, b).
This discussion on results aggregation is very similar to that in the field of strategic philanthropy,
where foundations and social investors are seeking measures of social performance that can be
benchmarked and added up.
In short, the challenges of measurement facing international development actors are
similar to those confronting the U.S. social sector, although they are perhaps further along in
methodological sophistication and plurality of techniques. They all revolve around basic
concerns of how to assess impacts with methodological rigor, how to be accountable to various
constituents, and how to enhance the adaptive capacities of organizations in order to act more
effectively.
WHAT TO MEASURE? PROPOSITIONS ABOUT PERFORMANCE
The growing emphasis on impact and accountability, both in the United States and
globally, has increased the pressure on social sector organizations to pay attention to
performance measurement. The key challenges can be summed up by the following three
questions that many nonprofit managers frequently ask:

What can my organization reasonably measure on the logic chain inputs, activities,
outputs, outcomes, or impacts?

What does my organization need to measure for accountability to funders?

How can my organization use measurement to help it better achieve its mission?

17

Ebrahim and Rangan, May 2010

Underlying these pragmatic questions are two deeper analytical problems endemic to the
social sector: (a) The problem of causality. How well understood is the relationship between
cause and effect of an intervention? Following convention in the social sector, we refer to an
organizations causal logic as its theory of change. (b) The problem of control over results. How
do managers exercise control over their interventions and results? In particular, management
control is reflected in an organizations boundaries and its operational strategy.
For the purposes of this paper, we focus our discussion primarily on the first question
above on measurement along the logic chain but with an interest in improving both
accountability and mission achievement.
Positioning Measurement on the Logic Chain
Conventional wisdom in the social sector suggests that one should measure results as far
down the logic chain as possible: outcomes and impacts. This expectation is based on a
normative view that organizations working on social problems, especially if they seek public
support, should be able to demonstrate impact in solving societal problems. In the international
development world, the use of such logical framework analysis (LFA) grew in popularity
among foundations and bilateral government agencies through the 1980s and 1990s, having been
originally developed for the U.S. Agency for International Development in 1969 (e.g., AusAID,
2005; Commission of the European Communities, 1993). The United Way of America developed
its manual for outcome measurement in 1996 (United Way of America, 1996a). Both types of
manuals emphasize the importance of establishing measurable causal relationships between a
projects objectives, outputs, and outcomes, and identifying appropriate indicators for measuring
results.

18

Ebrahim and Rangan, May 2010

Yet it is worth considering whether, and to what degree, such measurement makes sense
for all social sector organizations. A crude typology of the social sector in the field of poverty
alleviation distinguishes among at least three broad types of activities below (for more detailed
typologies of the sector, see Najam (1996b) and Vakil (1997)):

Emergency relief activities that address urgent survival needs, such as food and
temporary shelter, as well as disaster, crisis and conflict mitigation.

Service delivery activities that address basic needs, such as education, healthcare,
longer-term shelter, community development, employment and income generation.

Policy and rights advocacy activities that address structural issues related to rights,
public policy and regulation, and societal norms.
Some examples may be helpful for illustrative purposes. For instance, many international

NGOs such as CARE USA, Oxfam America, Doctors Without Borders, and the Red Cross
are engaged in emergency relief work. Measuring the emergency relief work of such
organizations is conceptually fairly straightforward: count the delivery of emergency supplies
such as tents, food, water, and medical clinics, as well as the numbers of people assisted.
Emergency relief is thus typically measured in terms of inputs, activities, and outputs. While it is
a complex activity, requiring highly sophisticated coordination and supply chain management
capabilities, it is initially focused on meeting immediate survival needs rather than long-term
development outcomes or impacts. At the same time, many of these same organizations are
engaged in post-emergency community development and service delivery that is also frequently
measured in terms of outputs the number of children and their families receiving health care,
education, drinking water, and so on. This information is readily communicated to donors who
want to know how their money has been spent.

19

Ebrahim and Rangan, May 2010

Outcome measurement, on the other hand, requires answers to a more complex causal
question: Are the activities and outputs leading to longer-term benefits? One would expect
outcome measurement to be less common and more difficult to do, given that organizations have
the most control over their immediate activities and outputs, whereas outcomes extend beyond
their organizational boundaries. Without a causal analysis that links various factors that can
affect outcomes, it is hard to imagine how outcomes might be measured or even anticipated.
A widely used example of linking outputs to outcomes concerns immunization. A key
output measure of an immunization campaign is the number of people vaccinated, against polio
for instance. Although each person immunized is important, the most significant impacts are
only achieved once a certain threshold of immunization is reached, say 80% of a population, as
this makes it possible for the disease to be eradicated. Such eradication is not only an outcome,
but an impact. The logistics of such a feat are highly complicated, but conceptually the basic
links between outputs and outcomes in such an intervention are premised on one key factor:
scale.
A more complex example is the work of one the worlds largest non-governmental
organizations, BRAC (formerly the Bangladesh Rural Advancement Committee). With over
100,000 staff that claim to reach over 110 million people in parts of Asia and Africa, BRAC
offers a comprehensive set of services in economic and social development, education, health,
human rights, and legal services. BRACs development model relies on two factors for its
impact: scale and scope. In terms of scale, the organization reaches most, if not all, villages in
the regions where it works, effectively becoming a parastatal organization. In terms of scope, it
provides a comprehensive and integrated set of activities related to poverty alleviation. Within
the United States, a more modest example of scale and scope might be the Harlem Childrens

20

Ebrahim and Rangan, May 2010

Zone, which seeks to improve the life chances of urban youth by offering comprehensive
services to young people (scope) covering over 100 city blocks in Harlem (scale). In the public
sector, bilateral agencies such as the United States Millennium Challenge Corporation (MCC)
also attempt to provide both scale and scope through their nation-wide economic development
interventions.
The aggregation of outputs to outcomes and eventually impact, however, breaks down in
cases of policy advocacy and rights-based work. Well-known organizations like CARE, Oxfam,
Amnesty International, Human Rights Watch, ActionAid International, and others, all aim to
address the root causes of problems of poverty and injustice through their policy-based work.
But their abilities to measure long-term results, and to attribute those results to their
interventions, are severely limited. Their advocacy campaigns may achieve many successes
for example, in getting the attention of public officials to the rights of children, women, political
prisoners, and others; in forming national-level constituencies and coalitions; and in improving
public awareness about marginalized groups. However, these measures are largely about
activities, processes, and outputs, rather than about ultimate impacts on rights. This focus on
activities and process is due, at least in part, to the complex nature of causality when it comes to
shaping public policy and social norms. It is also about the difficulty in attributing any one
organizational intervention to ultimate impacts. The problem is, that while a nonprofit
organization may influence or contribute to a policy or attitudinal change, it is more difficult to
attribute that change solely to its interventions.
These preliminary observations lead to several propositions that relate performance
measurement to the type of social sector activity:

21

Ebrahim and Rangan, May 2010

1. Performance in emergency and relief work can be measured in terms of inputs, activities,
and outputs.
2. Performance in service delivery work can be measured in terms of activities and outputs.
3. Performance in service delivery work, when of large scale and scope, can be measured in
terms of outcomes and sometimes impacts.
4. Performance in advocacy and rights-based work can be measured in terms of outputs and
influence, an intermediary outcome.
Perhaps what is most surprising about these preliminary observations is that only one
category of intervention can readily be measured at the impact end of the logic chain service
delivery activities that achieve a threshold of scale and scope. Moreover, rights-based and
advocacy interventions, despite aiming to address root causes of social problems, cannot easily
demonstrate impact and are thus left with measuring outputs and intermediary forms of
outcomes. These constraints on measuring performance also present a dilemma for
accountability: many funders increasingly want to see measures of impact, whereas it may not be
feasible for many nonprofits to measure performance beyond outputs and outcomes.
A CONTINGENCY FRAMEWORK FOR MEASURING RESULTS
We offer a contingency framework for assessing what types of results to measure in
Figure 1. The vertical axis of the matrix theory of change refers to the causal logic
underlying an intervention. A theory of change articulates the causal logic or pathway through
which a set of interventions is expected to lead to a long-term goal (ActKnowledge, 2010;
Keystone, 2008). How complex is the relationship between cause and effect, and how well is it
understood? For a focused theory of change, the relationship between cause and effect is linear

22

Ebrahim and Rangan, May 2010

and clearly understood. A complex theory of change refers to cause-effect relationships that are
only weakly understood, and where there may be multiple causal factors at play.
Many emergency and basic services operate on a linear, or focused, theory of change: get
shelter, food and water to people facing a crisis in order to avert further disaster. Or provide
access to clean water and sanitation to improve human health. The logistics and process of doing
so can be highly complicated, but the basic intervention logic is fairly straightforward. Efforts to
influence public policy, however, are typically more complex, and they are shaped by many
factors which may be poorly understood and non-linear. In such contexts, it is hard to know how
much of a difference a particular intervention makes. They may also involve constellations of
many actors, making it difficult to attribute a policy change to the actions of any single
organization.
-------------------------------Insert Figure 1 about here
-------------------------------The horizontal axis of the matrix operational strategy refers to what an organization
actually does in implementing its mission. Under a focused operational strategy, the nonprofit
concentrates on a highly specific task or intervention. For example, an ambulance service
transports an individual to a hospital, the hospital provides treatment, and a family practice gives
follow up care. Each organizational actor fills a specific and critical niche. Under a more
complex operational strategy, an organization expands its boundaries to absorb other key
functions or niches that it deems important to achieving its mission. For example, a hospital
might acquire an ambulance service, a homeless shelter might provide in-house counseling and

23

Ebrahim and Rangan, May 2010

health care to its clients, or a job training program might create its own job placement pipeline
with employers. In none of these examples is there a change in the organizations theory of
change; it is simply expanding its operational strategy to exercise more control over that causal
logic chain. An analogous process in the private sector is vertical integration in a supply chain.
An organization expands its boundaries to absorb upstream or downstream functions or tasks,
thus reducing uncertainties or risks associated with adjacent actors, or to take advantage of
transaction cost efficiencies.
Taking into account these contingencies i.e., the complexity of an organizations
theory of change, and the complexity of its operational strategy we distinguish among four
broad types of results: niche, integrated, institutional, and ecosystem (see Figure 1). These four
types should not be read as a normative continuum, but instead as a reflection of the real
circumstances faced by social sector actors. It is possible to be high-performing and to operate at
scale in each of the four quadrants.
Organizations in the niche quadrant are driven by a linear or well-understood causal logic
(a focused theory of change) that they implement through very specific or well-defined
interventions (a focused operational strategy). Many basic and emergency service providers such
as homeless shelters and soup kitchens, for example, fulfill a critical function that is part of a
longer chain of interventions for assisting homeless individuals. Their emergency work is
focused on meeting immediate and highly tangible needs, and it is reasonable for them to
measure their activities (the services they provide) and sometimes their outputs (immediate
effects on clients). Any longer-term outcomes or impacts (getting people out of homelessness),
however, are typically beyond the control and roles of such organizations. Similarly, an
ambulance service might measure its response time and the quality of its paramedic services

24

Ebrahim and Rangan, May 2010

(activities and outputs) but not the health of the patient after treatment at a hospital (outcomes).
Crisis hotlines such as suicide or poison control call centers can offer support or information to
callers on the phone, but they have little control over how, or even if, that information is used.
For such nonprofits, it is feasible to stick with simple measures of the organizations
inputs, activities, and outputs. It would not make much sense to try to measure long-term
impacts, since that is not what niche efforts can reasonably control, nor necessarily what the
organization aims to achieve (Jacobs, 2008). Yet they offer crucial services, attract considerable
public donations, and may provide a basis for longer-term interventions.
The second quadrant refers to circumstances where it is possible to achieve integrated
results. Here, too, the theory of change is focused and typically includes organizations delivering
basic services such as in health, sanitation, job training, and so on. But this work is different
from niche contexts in that the organization occupies several niches in the causal chain, and thus
has more control not only over outputs but also over outcomes. The Aravind Eye Hospital in
India, for example, provides cataract eye surgeries. Its control over outcomes (restored vision) is
made possible by the integration of many activities, ranging from the technology for cataract
surgery and qualified medical staff, to extensive community outreach, setting up eye camps for
the poor, transporting people, and providing follow up support. Aravinds theory of change is
linear and straightforward, but its operational strategy is broad or complex. Similarly, many
nonprofits that provide job training programs offer not only training, but also placements with
employers, plus follow-up support to clients.
These organizations are able to measure their outputs (numbers of patients seen and eye
surgeries completed, or people trained and placed in jobs) and also outcomes (number of people

25

Ebrahim and Rangan, May 2010

with vision restored, or jobs retained for more than a year and increased incomes). Sometimes,
the results can literally be added up to arrive at an aggregate measure of outputs and outcomes.
Where organizations can isolate their interventions, they may be able to use quasi-experimental
methods such as randomized control trials (RCTs) to test for effectiveness. The use of
experimental methods is less feasible in contexts where it is hard to isolate an intervention.
A more complex example of an integrated result is an immunization campaign, where the
metrics are typically expressed as outputs (number or percentage of people vaccinated) and
outcomes (declines in illness) in order to get at impacts (prevention, containment or eradication
of a disease). While the causal logic in preventing or eradicating a disease may be relatively well
established, the operational strategy is extremely complex it requires not only having a proven
vaccine and the technology for distribution, but also strategies for community organizing, for
addressing cultural norms and fears that may limit immunization uptake, and for coordinating
public health workers. Only one disease, smallpox, has ever been completely eradicated through
an immunization campaign.2
Similarly, NGOs engaged in humanitarian and disaster relief must be able to handle
highly complex logistics in getting relief supplies to a region hit by an earthquake, flood, or
conflict. In order to deliver aid quickly, they must acquire and stock critical supplies such as
blankets, tents, food and water, while also transporting, delivering, and distributing those
supplies, and ensuring that they are delivered equitably and safely on the ground. Their
challenges require a complex operational strategy as well as extensive coordination with other
actors such as governments, militaries, U.N. agencies, private sector suppliers, and other
international and local NGOs. Their theories of change are focused, while their operational
strategies are complex. They can measure aggregate outputs (such as number of people receiving

26

Ebrahim and Rangan, May 2010

shelter, food, and clean water), and may be able to measure outcomes over a longer term (such as
number of people with improved living conditions and health).
Organizations in the institutional quadrant have a focused operational strategy, like those
in the niche quadrant, but their theories of change are more complex. Advocacy organizations,
such as Amnesty International or Greenpeace, fall into this category. They benefit from having
highly focused strategies in advocacy rather than in service delivery. However, their efforts to
shift fundamental societal institutions and norms such as those concerning human rights,
democratic freedoms, and environmental governance do not follow a linear logic. A nonprofit
organization working on the rights of children, women, or workers in the informal economy may
lobby legislators for years without seeing a change in public policy or regulation, but does that
mean it has not influenced the policy discourse? Or if there is a change in public attitudes about
civil and human rights, how much of that change was a direct result of the efforts of that
organization, as compared to media coverage, the predispositions of legislators, or other
unanticipated factors?
For actors in this quadrant, it may be feasible to measure their outputs and their
influence in shaping key policies, rights, and freedoms (i.e., intermediate outcomes), but not
necessarily their impacts. Arguably, impacts are more likely to be achieved by networks or
coalitions of actors working in concert, than by single organizations acting alone. There has been
a flurry of activity in recent years towards developing suites of tools to help organizations
identify interactive pathways and their effects in such complex and nonlinear contexts for
example, outcome mapping, most significant changes methodologies, and systems thinking
(noted above). While these methods have received less attention than quasi-experimental

27

Ebrahim and Rangan, May 2010

methods, they show considerable promise for helping managers deal with the difficulties of
measuring results in complex and messy environments.
Finally, organizations in the ecosystem quadrant focus on problems that are not well
understood (complex theory of change) while also engaging in multiple interventions and roles
(complex operational strategy). They aim for synergistic results that affect entire systems. For
example, many international NGOs, such as Oxfam, Save the Children, CARE, and ActionAid,
originally provided basic services to poor communities in health, sanitation, education, and so
on. Over the past two decades, however, they have expanded their strategies to include rightsbased policy advocacy at national and global levels. They continue much of their service
delivery work, but see it as part of a more complex set of interventions that includes building the
rights of marginalized communities to make demands of their own governments. There are many
other examples already noted. The Harlem Childrens Zone offers a conveyor belt of
educational services for children and youth while also engaging a range of actors such as city
government, parents, and other nonprofits to create a caring environment for those youth. In
Bangladesh, BRAC not only offers a wide range of services to thousands of villages, but is also a
key national policy player. And the Millennium Challenge Corporation seeks to improve the
incomes of farmers, in countries such as Ghana, through a combination of interventions in
infrastructure and agriculture, coupled with policy interventions intended to create efficient and
transparent markets for goods and services.
Each of these organizations aims for systemic changes that address the root causes of
poverty (impacts). Measuring these impacts, however, is likely to require long time frames: the
MCC anticipates increases in farmer incomes over a 20-year time horizon, while the Harlem
Childrens Zone expects to see impacts on its youth over the course of a generation. Attributing

28

Ebrahim and Rangan, May 2010

those results to the work of any single organization is only possible where the organization has a
near-monopoly (through scale and scope) on the interventions in that location: BRAC has
supplanted government services in many regions of Bangladesh, the MCC implements is
programs through national governments, and the Harlem Childrens Zone attempts to provide
comprehensive services in a 100-block area of Harlem. For other organizations such as CARE,
Oxfam, Save the Children, and ActionAid, impacts are more likely to be achieved through
partnerships and collaborations, although it may not be possible to attribute impacts solely to any
one actor.
In sum, our contingency framework offers some general cautions about performance
measurement. First, it suggests that measuring impacts makes sense under a severely limited set
of circumstances when an organization operates at an ecosystem level, and yet can exercise
sufficient control over results to attribute impacts to its work. With few exceptions, most
organizations will not be able to exert such control (despite their explicit focus on impacting root
causes of poverty) when their work requires complex theories of change. Similarly, nonprofits
trying to influence the institutional and policy environment, such as advocacy organizations, are
more likely to be able to measure intermediate outcomes or influence on policy decisions,
rather than impacts. Organizations operating with a focused theory of change are more likely to
be able to demonstrate a causal link between their work and results, but these results are limited
to activities and outputs in the case of focused operational strategies, and outcomes where there
is an integrated or complex strategy.
Second, we expect that many organizations will not fit neatly into the quadrants of our
matrix, with some traversing many of them. Oxfam, CARE, and ActionAid for instance, are
engaged in a portfolio of activities ranging from emergency relief to service delivery to rights

29

Ebrahim and Rangan, May 2010

based work. They face the double-challenge of measuring performance in each of these areas
separately, while also integrating across them in order to gauge their possible synergistic effects
at the ecosystem level.
Third, funders such as foundations, governmental departments, and international aid
agencies, are far better positioned than most nonprofits to measure impacts. A foundation that
supports efforts in health care, for example, is uniquely situated to see how the work of all its
grantees might link together to connect the dots among a series of outputs and outcomes, to
analyze how they lead to impacts. A central strategic challenge for funders is to articulate a logic
for achieving impacts (a theory of change), and then to put together a portfolio of nonprofits to
achieve those impacts (an operational strategy).
And finally, given the diversity of actors engaged in social change, the four broad types
of results in the framework should be taken as suggestive rather than as silver bullets. The very
basis of the framework contingency suggests that there are no panaceas to results
measurement in complex social contexts.
IMPLICATIONS FOR MANAGEMENT PRACTICE AND FUTURE RESEARCH
Our contingency framework aims to address the debate around impact by focusing on
what results to measure on the logic chain. The discussions around impact measurement, both
within the U.S. and internationally have been somewhat polarized between those who demand
clear and quantifiable measures of impact and those who contend that social sector work is so
unique and context-specific that it cannot be readily measured. Our contingency framework
attempts to move beyond such reductionism by differentiating among types of results.

30

Ebrahim and Rangan, May 2010

This work also has implications for accountability in social sector organizations. The
contingency framework offers a way for organizational leaders and managers to clarify what
types of results they seek to achieve, and thus for what they should be held to account. We
recognize, however, that nonprofit leaders often face competing demands from different funders,
beneficiaries, regulators and other stakeholders. We hope our framework offers a diagnostic tool
for engaging nonprofits and their funders in a conversation around results. There is a need for
additional research to examine how nonprofit leaders negotiate the competing demands of
different stakeholders and funders, and the role of measurement in such negotiation.
Moreover, how measurement and analysis occur can be as important as what gets
measured. Nonprofit organizations rely on various processes to build credibility and legitimacy
with the communities they serve, ranging from consultation and participatory decision making in
the setting of goals and metrics, to participatory monitoring and third party evaluations. Our
contingency framework does not address this dimension of measurement; it would be useful to
understand what constitutes a performance goal that has legitimacy among key stakeholders, and
how nonprofit leaders can build such legitimacy.
Beyond these immediate issues about accountability and legitimacy, there are many
avenues for further inquiry that we believe will be crucial to the development of the field of
performance measurement in the social sector. We lay out three promising directions here: scale
and scope, methods for assessing performance, and alignment of measurement with strategy.
Scale and Scope
Our contingency model suggests that social sector organizations can increase their
control over long-term impacts in at least two ways: by expanding their operations in order to

31

Ebrahim and Rangan, May 2010

reach a threshold population or critical mass (scale), and by offering more comprehensive
services or partnering with others in order to tackle a problem (scope). For nonprofit leaders,
doing so would require systematic analysis of what they control and what they dont and how
they might reasonably expand their control. Under conditions of complexity, we expect that it
will be especially difficult for organizations to achieve impacts single-handedly.
There is an urgent need for better knowledge on the challenges of scale and scope in the
social sector. There may be lessons to be learned from the corporate sector, where scale and
scope have been more thoroughly and systematically studied (for example, Chandler (1990)).
Scale in the social sector can be achieved not only through organizational growth but also via a
myriad of other means, particularly through influencing public policy, building collaborative
networks, or even through the creation of new industries. For instance, nonprofits served as the
initial pioneers in the microfinance industry, but the explosion of the industry was fueled by the
creation of a supportive regulatory environment and the entry of opportunistic private sector
players (World Microfinance Forum Geneva, 2008). Much remains poorly understood about
scale: What are the various avenues to scale in the social sector? What are the main deficiencies
in capital markets and regulation that are preventing scale, and how can they be addressed?
The complex nature of poverty also suggests a need for solutions of broad scope. Many
nonprofits, especially in international development, offer multiple kinds of services to their
clients. But as in the for-profit world, organizations that offer a diverse portfolio of products and
services run the risk of losing focus, stretching themselves too thinly, and failing to build
efficiencies and synergies across product lines or services. Further research on organizations that
operate at scale and scope can help uncover both the challenges and opportunities facing social
sector managers. While there has been significant work on collaboration and networks in the

32

Ebrahim and Rangan, May 2010

social sector (e.g.,Austin, 2000; Brown, 2007; Weber, 2003; Wei-Skillern & Marciano, 2008) the
challenges of scope and scale remain understudied, as do the opportunities for interconnections
beyond organizational and sectoral boundaries (Kanter, 2010).
Methods for Assessing Social Performance
Over the past twenty years, there has been an explosion in methodologies and tools for
assessing social performance and impact, but with little systematic analysis and comparison
across these approaches (see Table 2). The various approaches to measuring social return on
investment and internal rates of return are most popular among the impact investment
community. But there is an even longer history in international development with the use of logic
models, variations of which have included logical framework analysis, outcome measurement,
and impact evaluation, and which have been supplemented with experimental and quasiexperimental approaches to evaluation such as randomized control trials. These approaches,
which tend to rely on linear causal models, have increasingly been challenged by new
approaches involving outcome mapping, constituency feedback, participatory monitoring and
evaluation, impact planning and learning systems, and complexity thinking and systems analysis.
A comparison of these approaches could provide a useful perspective for nonprofit managers,
evaluators, and funders, while also providing a sound basis for deciding which to employ and
under what circumstances.
There is also a dearth of analysis on industry-level measures of social performance. For
example, in the world of social investment and venture philanthropy, there is a general concern
that asset owners and institutional investors such as pension funds, foundations, and banks, are
hesitant to put their money into social investments because of the lack of benchmarking or

33

Ebrahim and Rangan, May 2010

simple rating systems. This problem may be seen as an industry-level accountability gap
characterized by a lack of consistently reportable information. In response, some key players in
the U.S. social investing field teamed up in early 2009 to develop a common framework for
defining, tracking and reporting the performance of impact capital.3 A report by Monitor
Institute estimates that investing in social and environmental impact has the potential to grow
from about $50 billion in 2008 to $500 billion in 10 years (Monitor Institute, 2009). The
microfinance industry may serve as a model, where the emergence of rating agencies and
standards has helped create an asset class.
At the same time, established nonprofit ratings agencies such as Charity Navigator and
the Better Business Bureaus Wise Giving Alliance have begun to revamp their rating metrics.
Their original indicators, which relied heavily on financial and overhead cost ratios, are widely
seen as being problematic and even misleading for the social sector. In late 2009, they launched
an effort to develop a package of more comprehensive measures that are not just financial but
also social (Charity Navigator et al., 2009). Such industry-wide efforts remain poorly
understood: What constitutes a meaningful and reliable comparative metric? What incentives and
disincentives exist for organizations to participate in such benchmarking efforts? What is
appropriate to measure for rating and benchmarking purposes activities, outputs, outcomes, or
impacts?
Aligning Measurement Systems with Strategy
Finally, as noted in our introduction, there is a marked gap in knowledge on management
control systems in the social sector, particularly on the means of aligning performance systems
with strategy and mission. To date, the preponderance of literature related to performance has

34

Ebrahim and Rangan, May 2010

focused on accountability challenges, drawing primarily from the disciplines of sociology,

political science, political economy, public administration, law, and anthropology.
(e.g.,Benjamin, 2008; Ebrahim, 2005; Edwards & Hulme, 1996; Koppell, 2005). There has been
little work from the perspective of building applied knowledge for management purposes, with
some notable exceptions (e.g., Epstein & Buhovac, 2009; Kaplan & Norton, 1996; Letts et al.,
1999; Oster, 1995; Paton, 2003).
At face value, the control problems facing managers appear similar in for-profit and
nonprofit organizations lack of direction, lack of motivation, and lack of ability but the
solutions must necessarily differ because, in the social sector, the goals and results tend to be
more ambiguous, and managers find it harder to motivate behavior through financial incentives
(Day, 2004; Merchant & Van der Stede, 2007: 789). Simons (1995) work, for example, opens
up numerous questions relevant to the social sector: How can managers effectively use core
values as a control lever? How can managers communicate, incentivize, and monitor their staff
on critical performance outcomes? And what kinds of interactive systems and approaches to
organizational learning lend themselves to improving social performance? Current innovations in
the social sector offer a rich landscape for study, particularly on adaptive learning and
participatory governance among large international NGOs and social movements.
In sum, the basic challenge is twofold: understanding what an alignment of mission and
strategy might look like in the social sector, and how to design performance and control systems
to enable and support this alignment. The problems associated with causal logics and strategy
highlighted in our contingency model suggest that this task is very different in socially-driven
organizations than in profit-driven ones. Our paper is a step towards visualizing what such an
alignment might look like in practice, and the contributions that scholarly analysis might make.

35

Ebrahim and Rangan, May 2010

NOTES
1. The Paris Declaration on Aid Effectiveness aimed to improve the quality of aid and its
impact on international development. First, it emphasizes country ownership as fundamental to
improving the impact of aid on the basis that aid is more effective when partner countries take
leadership over their development policies and strategies. Second, the Declaration focuses on
results by identifying 12 indicators of aid effectiveness. These indicators measure the developing
countrys ability to use aid (e.g., clear strategic priorities for poverty reduction, reliable systems
for procurement and public financial management) as well as the efforts by donor agencies to
help build country capacity, for example, by using the partner countrys procurement and
financial management systems rather than the donors own systems, by disbursing aid on a more
predictable schedule, and by reducing requirements that aid be tied to the purchases of goods
and services from the donor country (OECD2008).
2. There is some evidence that success in immunization efforts may not follow as linear a
logic as anticipated by many funders and global health organizations. Efforts to eradicate polio,
for example, through a vertical strategy of immunization campaigns have been set back by the
re-emergence of polio cases in countries thought to have eliminated it. Global actors, such as the
Bill and Melinda Gates Foundation, are now considering combining immunization campaigns
with horizontal strategies that include strengthening basic health systems and services in poor
countries. The causal logic of such a combined vertical-horizontal strategy is less well
understood, but it is driven by a concern that without a broader system of functioning health,
hygiene, and sanitation services for the poor, many diseases will return (Guth, 2010).

36

Ebrahim and Rangan, May 2010

3. Acumen Fund, The Rockefeller Foundation, and B-Lab organizations piloted a

framework, called the Impact Reporting and Investment Standards (IRIS), built in collaboration
with PricewaterhouseCoopers and Deloitte. The use of open-source Extensible Business
Reporting Language (XBRL) aimed to allow investment data from different databases to be
housed together, aggregated, and compared at the industry, fund, or sector level (see http://irisstandards.org/). There is a similar effort underway in South Africa called the Nexus for Impact
Investing (see http://www.nexii.com/).

37

Ebrahim and Rangan, May 2010

REFERENCES
ActionAid International. 2006. ALPS: Accountability, Learning and Planning System.
Johannesburg: ActionAid International.
http://www.actionaid.org/assets/pdf/ALPSENGLISH2006FINAL_14FEB06.pdf
ActKnowledge. 2010. Website: www.theoryofchange.org. www.theoryofchange.org
Alliance. 2009. Interview Paul Brest, Jed Emerson, Katherina Rosqueta, Brian Trelstad and
Michael Weinstein, Alliance Magazine. http://www.alliancemagazine.org/node/2128
Anthony, R. N., & Young, D. W. 2002. Management Control in Nonprofit Organizations
(Seventh Edition). Burr Ridge, IL: McGraw Hill.
Anthony, R. N., & Young, D. W. 2004. Financial Accounting and Financial Management. In R.
D. Herman, & Associates (Eds.), The Jossey-Bass Handbook of Nonprofit Leadership
and Management, 2nd Edition. San Francisco: Jossey-Bass/Wiley.
Asian Development Bank. 2004. The Public Communications Policy of the Asian Development
Bank: Disclosure and Exchange of Information Manila: Asian Development Bank
AusAID. 2005. The Logical Framework Approach: Australian Agency for International
Development (AusAID), Commonwealth of Australia
Austin, J. E. 2000. The Collaboration Challenge: How Nonprofits and Business Succeed
through Strategic Alliances. San Francisco: Jossey-Bass.
Banerjee, A. V. 2007. Making Aid Work. Cambridge, MA: MIT Press.
Banerjee, A. V., & Duflo, E. 2009. The Experimental Approach to Development Economics.
Annual Review of Economics, 1: 151-178.
Behn, R. D. 2001. Rethinking Democratic Accountability. Washington, D.C.: Brookings
Institution Press.

38

Ebrahim and Rangan, May 2010

Benjamin, L. M. 2008. Account Space: How Accountability Requirements Shape Nonprofit

Practice. Nonprofit and Voluntary Sector Quarterly, 37(2): 201-223.
Blumenthal, B. 2003. Investing in Capacity Building: A Guide to High-Impact Approaches.
New York: The Foundation Center.
Bonbright, D., Campbell, D., & Nguyen, L. 2009. The 21st Century Potential of Constituency
Voice: Opportunities for Reform in the United States Human Services Sector: Alliance
for Children & Families, United Neighborhood Centers of America, and Keystone
Accountability
Brady, H. E., & Collier, D. (Eds.). 2004. Rethinking Social Inquiry: Diverse Tools, Shared
Standards. Lanham, MD: Rowman and Littlefield.
Brest, P., & Harvey, H. 2008. Money Well Spent: A Strategic Plan for Smart Philanthropy.
New York: Bloomberg Press.
Brown, L. D. 2007. Multiparty Social Action and Mutual Accountability. In A. Ebrahim, & E.
Weisband (Eds.), Global Accountabilities: Participation, Pluralism, and Public Ethics.
Cambridge, UK: Cambridge University Press.
Bruns Jr., W. J. 1992. Performance Measurement, Evaluation, and Incentives. Boston, MA:
Harvard Business School Press.
Bryson, J. M. 2004. Strategic Planning for Public and Nonprofit Organizations: Third Edition.
San Francisco: Jossey-Bass.
Butler, L. M. 2007. The Nonprofit Dashboard: A Tool for Tracking Progress. Washington,
D.C.: BoardSource.
Campbell, D. 2002. Outcomes Assessment and the Paradox of Nonprofit Accountability.
Nonprofit Management and Leadership, 12(3): 243-259.

39

Ebrahim and Rangan, May 2010

Canadian International Development Agency. 2009a. Donor Roundtable on Aggregation of

Donor-Funded Civil Society Organizations' (CSO) Development Results. Hull: Canadian
Partnerships Branch, CIDA
Canadian International Development Agency. 2009b. Results Management Methods of Six
Bilateral Donors (Draft). Hull: Canadian Partnerships Branch, CIDA
Center for Global Development. 2006. When Will We Ever Learn? Improving Lives through
Impact Evaluation. Washington, D.C.: Evaluation Gap Working Group, Center for
Global Development
Chambers, R. 1994. Participatory Rural Appraisal (PRA): Analysis of Experience. World
Development, 22(10): 1253-1268.
Chambers, R., Karlan, D., Ravallion, M., & Rogers, P. 2009. Designing Impact Evaluations:
Different Perspectives: International Initiative for Impact Evaluation
Chandler, A. D. 1990. Scale and Scope: The Dynamics of Industrial Capitalism. Cambridge,
MA: The Belknap Press of the Harvard University Press.
Charity Navigator, GiveWell, Great Nonprofits, Guidestar, Philanthropedia, & Philanthropy
Action. 2009. The Worst (and Best) Way to Pick a Charity This Year, Vol. 2010: Press
Release: Authors.
http://www.philanthropyaction.com/documents/Worst_Way_to_Pick_A_Charity_Dec_1_
2009.pdf
Coleman, G. 1987. Logical Framework Approach to the Monitoring and Evaluation of
Agricultural and Rural Development Projects. Project Appraisal, 2(4): 251-259.
Commission of the European Communities. 1993. Project Cycle Management: Integrated
Approach and Logical Framework. Brussels: Commission of the European Communities

40

Ebrahim and Rangan, May 2010

Crutchfield, L. R., & Grant, H. M. 2008. Forces for Good: The Six Practices of High-Impact
Nonprofits. San Francisco: Jossey-Bass.
Dart, J., & Davies, R. 2003. A Dialogical, Story-Based Evaluation Tool: The Most Significant
Change Technique. American Journal of Evaluation, 24(2): 137-155.
David, R., & Mancini, A. 2004. Going against the flow: making organisational systems part of
the solution rather than part of the problem. Lessons for Change in Policy &
Organisations(No. 8).
Davies, R., & Dart, J. 2005. The 'Most Significant Change' (MSC) Technique: A Guide to Its
Use
Day, N. E. 2004. Total Rewards Programs in Nonprofit Organizations. In R. D. Herman, &
Associates (Eds.), The Jossey-Bass Handbook of Nonprofit Leadership and
Management, 2nd Edition. San Francisco: Jossey-Bass/Wiley.
Earl, S., Carden, F., & Smutylo, T. 2001. Outcome Mapping: Building Learning and Reflection
into Development. Ottawa: International Development Research Centre.
Ebrahim, A. 2005. Accountability Myopia: Losing Sight of Organizational Learning. Nonprofit
and Voluntary Sector Quarterly, 34(1): 56-87.
Ebrahim, A., & Weisband, E. (Eds.). 2007. Global Accountabilities: Participation, Pluralism,
and Public Ethics. Cambridge: Cambridge University Press.
Edwards, M., & Hulme, D. (Eds.). 1996. Beyond the Magic Bullet: NGO Performance and
Accountability in the Post-Cold War World. West Hartford, Connecticut: Kumarian
Press.
Epstein, M. J., & Buhovac, A. R. 2009. Performance Measurement of Not-For-Profit
Organizations. Mississauga and New York: The Society of Management Accountants of

41

Ebrahim and Rangan, May 2010

Canada (CMA Canada) and the American Institute of Certified Public Accountants, Inc.
(AICPA)
Epstein, M. J., & Manzoni, J.-F. (Eds.). 2008. Performance Measurement and Management
Control: Measuring and Rewarding Performance. Bingley, U.K.: Emerald JAI.
European Evaluation Society. 2007. EES Statement: The importance of a methodologically
diverse approach to impact evaluationspecifically with respect to development aid and
development interventions. Nijnerk: EES Secretariat
Frumkin, P. 2006. Strategic Giving: The Art and Science of Philanthropy. Chicago: University
of Chicago.
Gasper, D. 1997. Logical Frameworks: A Critical Assessment: Managerial Theory, Pluralistic
Practice, Vol. Working Paper Series No. 264. The Hague: Institute of Social Studies.
http://biblio.iss.nl/opac/uploads/wp/wp264.pdf
George, A., & Bennett, A. 2005. Case Studies and Theory Development in the Social Sciences.
Cambridge, MA: MIT Press.
Gerring, J. 2007. Case Study Research: Principles and Practices. Cambridge: Cambridge
University Press.
Gibelman, M., & Gelman, S. R. 2001. Very Public Scandals: Nongovernmental Organizations in
Trouble. Nonprofit and Voluntary Sector Quarterly, 12(1): 49-66.
Glasrud, B. 2001. The Muddle of Outcome Measurement: Be Careful How You Measure
Programs. Nonprofit World, 19(6): 35-37.
Goldstein, J. A., Hazy, J. K., & Silberstang, J. (Eds.). 2009. Complexity Science and Social
Entrepreneurship: Adding Social Value through Systems Thinking Litchfield Park,
AZ: ISCE Publishing.

42

Ebrahim and Rangan, May 2010

Greve, H. R. 2003. Organizational Learning and Performance Feedback: A Behavioral

Perspective on Innovation and Change. Cambridge: Cambridge University Press.
Guijt, I. 2008. Critical Readings on Assessing and Learning for Social Change: A Review, Vol.
IDS Development Bibliography 21. Brighton, U.K.: Institute of Development Studies
Guth, R. A. 2010. Gates Rethinks His War on Polio, The Wall Street Journal. New York
Hall, P. D. 2005. A Solution is a Product in Search of a Problem: A History of Foundations and
Evaluation Research
Hernndez, G., & Visher, M. G. 2001. Creating a Culture of Inquiry: Changing methods and
minds on the use of evaluation in nonprofit organizations. San Francisco: The James
Irvine Foundation
Hwang, H., & Powell, W. W. 2009. The Rationalization of Charity: The Influences of
Professionalism in the Nonprofit Sector. Administrative Science Quarterly, 54: 268-298.
Jacobs, A. 2008. New Management Agenda (unpublished manuscript). London: Keystone
Accountability
Jones, H. 2009. The "gold standard" is not a silver bullet for evaluation. London: Overseas
Development Institute
Jones, N., Jones, H., Steer, L., & Datta, A. 2009. Improving Impact Evaluation Production and
Use. London: Overseas Development Institute
Kanter, R. M. 2010. Think Outside the Building Harvard Business Review, 88(3).
Kaplan, R. S. 2001. Strategic Performance Measurement and Management in Nonprofit
Organizations. Nonprofit Management and Leadership, 11(3): 353-370.
Kaplan, R. S., & Norton, D. P. 1996. The Balanced Scorecard: Measures that Drive
Performance. Boston: Harvard Business School Press.

43

Ebrahim and Rangan, May 2010

Kaplan, R. S., & Norton, D. P. 2004. Strategy Maps: Converting Intangible Assets into
Tangible Outcomes. Boston: Harvard Business School Press.
Kearns, K. P. 1996. Managing for Accountability: Preserving the Public Trust in Nonprofit
Organizations. San Francisco: Jossey-Bass.
Keystone. 2008. Developing a Theory of Change. London: Keystone: Accountability for Social
Change. www.keystoneaccountability.org/node/115
Khagram, S., Thomas, C., Lucero, C., & Mathes, S. 2009. Evidence for Development
Effectiveness. Journal of Development Effectiveness, 1(3): 247-270.
Koppell, J. G. S. 2005. Pathologies of Accountability: ICANN and the Challenge of "Multiple
Accountabilities Disorder". Public Administration Review, 65(1): 94-108.
Kramer, M., Graves, R., Hirschhorn, J., & Fiske, L. 2007. From Insight to Action: New
Directions in Foundation Evaluation: FSG Social Impact Advisors
Kramer, M. R. 2005. Measuring Innovation: Evaluation in the Field of Social Entrepreneurship:
Skoll Foundation and FSG Social Impact Advisors
Leeuw, F., & Vaessen, J. 2009. Impact Evaluations and Development. Washington, D.C.:
Network of Networks on Impact Evaluation (NONIE)
Letts, C. W., Ryan, W. P., & Grossman, A. 1999. High Performance Nonprofit Organizations:
Managing Upstream for Greater Impact. New York: John Wiley & Sons.
Light, P. C. 2004. Sustaining Nonprofit Performance: The Case for Capacity Building and the
Evidence to Support It. Washington, D.C.: The Brookings Institution.
Merchant, K., & Van der Stede, W. A. 2007. Management Control Systems: Performance
Measurement, Evaluation and Incentives. Upper Saddle River, NJ: Prentice Hall.

44

Ebrahim and Rangan, May 2010

Meyer, M. W. 2002. Rethinking Performance Measurement: Beyond the Balanced Scorecard.

Cambridge: Cambridge University Press.
Monitor Institute. 2009. Investing for Social and Environmental Impact: A Design for Catalyzing
an Emerging Industry: Monitor Institute
Morley, E., Vinson, E., & Hatry, H. P. 2001. Outcome Measurement in Nonprofit Organizations:
Current Practices and Recommendations. Washington, D.C.: Independent Sector
Organisation for Economic Co-operation and Development (OECD). 2008. Paris Declaration on
Aid Effectiveness and the Accra Agenda for Action. Paris: Organisation for Economic
Co-operation and Development. http://www.oecd.org/dataoecd/30/63/43911948.pdf
Oster, S. M. 1995. Strategic Management of Nonprofits. Oxford: Oxford University Press.
Panel on the Nonprofit Sector. 2005. Strengthening Transparency, Governance, Accountability
of Charitable Organizations: A Final Report to Congress and the Nonprofit Sector.
Washington, DC: Independent Sector.
www.nonprofitpanel.org/Report/final/Panel_Final_Report.pdf
Panel on the Nonprofit Sector. 2007. Principles for Good Governance and Ethical Practice: A
Guide for Charities and Foundations. Washington, DC: Independent Sector.
http://www.nonprofitpanel.org/final/
Paton, R. 2003. Managing and Measuring Social Enterprises. London: Sage Publications.
Philanthropy Action. 2009. The Worst (and Best) Way to Pick A Charity This Year: Press
Release by Great Nonprofits, GiveWell, Philanthropedia, Chartity Navigator, Guidestar,
Hewlett Foundation, and Philanthropy Action.
http://www.philanthropyaction.com/documents/Worst_Way_to_Pick_A_Charity_Dec_1_
2009.pdf

45

Ebrahim and Rangan, May 2010

Poister, T. H. 2003. Measuring Performance in Public and Nonprofit Organizations. San

Francisco: Jossey-Bass.
Porter, M. E., & Kramer, M. R. 1999. Philanthropy's New Agenda: Creating Value. Harvard
Business Review, 1999(November-December): 121-130.
Power, M. 1999. The Audit Society: Rituals of Verification. Oxford, UK: Oxford University
Press.
Prowse, M. 2007. Aid Effectiveness: The Role of Qualitative Research in Impact Evaluation.
London: Overseas Development Institute
Ramalingam, B., Jones, H., Reba, T., & Young, J. 2008. Exploring the Science of Complexity:
Ideas and Implications for Development and Humanitarian Efforts. London: Overseas
Development Institute. http://www.odi.org.uk/resources/download/583.pdf
Robb, C. M. 2002. Can the Poor Influence Policy?: Participatory Poverty Assessments in the
Developing World. Washington, D.C.: International Monetary Fund, World Bank.
Roche, C. 1999. Impact Assessment for Development Agencies: Learning to Value Change.
Oxford: Oxfam GB.
Rogers, P. 2009. Matching Impact Evaluation Design to the Nature of the Intervention and the
Purpose of the Evaluation. In R. Chambers, D. Karlan, M. Ravallion, & P. Rogers (Eds.),
Designing Impact Evaluations: Different Perspectives, Working Paper 4: International
Initiative for Impact Evaluation.
Rogers, P. J. 2007. Theory-Based Evaluation: Reflections Ten Years On. New Directions for
Evaluation, 2007(114): 63-67.
Saul, J. 2004. Benchmarking for Nonprofits: How to Measure, Manage, and Improve
Performance. St. Paul, Minnesota: Fieldstone Alliance.

46

Ebrahim and Rangan, May 2010

Sawhill, J. C., & Williamson, D. 2001. Mission Impossible? Measuring Success in Nonprofit
Organizations. Nonprofit Management and Leadership, 11(3): 371-386.
Simons, R. 1995. Levers of Control: How Managers Use Innovative Control Systems to Drive
Strategic Renewal. Boston, MA: Harvard Business School Press.
Torjman, S. 1999. Are Outcomes the Best Outcome? Ottawa: Caledon Institute of Social Policy
Trelstad, B. 2008. Simple Measures for Social Enterprise. Innovations, 3(3): 105-118.
Tuan, M. T. 2008. Measuring and/or Estimating Social Value Creation: Insights Into Eight
Integrated Cost Approaches: Impact Planning and Improvement, Bill & Melinda Gates
Foundation. http://www.gatesfoundation.org/learning/Documents/WWL-reportmeasuring-estimating-social-value-creation.pdf
United Way of America. 1996a. Measuring Program Outcomes: A Practical Approach.
Alexandria, VA: United Way of America.
http://www.liveunited.org/Outcomes/Resources/MPO/excerpts.cfm
United Way of America. 1996b. Outcome Measurement Resource Network.
http://national.unitedway.org/outcomes/resources/mpo/why.cfm
United Way of America. 2000. Agency Experiences with Outcome Measurement: Survey
Findings (Report 0196). Alexandria, VA: United Way of America
Urban Institute, & Center for What Works. 2006. Building a Common Outcome Framework to
Measure Nonprofit Performance. Washington, DC and Chicago, Illinois: Authors
Weber, E. P. 2003. Bringing Society Back In: Grassroots Ecosystem Management,
Accountability, and Sustainable Communities. Cambridge, MA: MIT Press.
Wei-Skillern, J., & Marciano, S. 2008. The Networked Nonprofit. Stanford Social Innovation
Review(Spring): 38-43.

47

Ebrahim and Rangan, May 2010

Weinstein, M. M. with C.E. Lamy. 2009. Measuring Success: How Robin Hood Estimates the
Impact of Grants. New York, NY: Robin Hood Foundation
White, H. 2009. We all agree we need better evidence. But what is it and will it be used? In M.
W. Lipsey, & E. Noonan (Eds.), Better Evidence for a Better World: International
Initiative for Impact Evaluation (3ie), Working Paper 2.
World Microfinance Forum Geneva. 2008. Is it fair to do business with the poor? Report on the
debate between Muhammad Yunus and Michael Chu. Geneva: Author
Young, D. R., Bania, N., & Bailey, D. 1996. Structure and Accountability: A Study of National
Nonprofit Associations. Nonprofit Management and Leadership, 6(4): 347-365.

48

Table 1: Logic Model

Inputs

Activities

Outputs

Outcomes

Impacts

What goes in

What happens

What results immediate

What results
medium- and longterm

What results effects on root

causes; sustained significant change

funds
equipment and supplies
knowledge and
technical expertise

basic needs delivered,

such as food and shelter
services delivered, such as
training programs
capacity building
construction of
infrastructure
policy dialogues and
workshops

people fed, treated or

housed
people trained or educated
roads built
goods transported to
market and sold
policy papers written
coalitions and networks
built

improved living
conditions, health,
etc.
increased incomes
enhanced political
voice

sustained drop in poverty

changes in public policy and
regulation
increased rights and political
power
fundamental changes in social
norms and attitudes (on rights,
freedoms, governance, and
markets)

49

Table 2: Methods for Assessing Social Performance

Method
Expected return

Innovators and Users

Acumen Fund best available
charitable option (BACO)
REDF social return on investment
(SROI)
Robin Hood Foundation benefit-cost
ratios (BCR)
Millennium Challenge Corporation
economic rate of return (ERR)

Resources
http://www.acumenfund.org/investments/investment-performance.html
http://www.redf.org/learn-from-redf/publications/119
http://www.robinhood.org/approach/get-results/metrics.aspx
http://www.mcc.gov/documents/factsheet-042808-err.pdf
See also Tuan (2008) and Trelstad (2008)

Experimental methods
Randomized control
trials

Abdul Latif Jameel Poverty Action

Lab, MIT
Innovations for Poverty Action

http://www.povertyactionlab.com/research/rand.php
http://poverty-action.org
See also Banerjee (2007), Banerjee and Duflo (2009), Jones(2009), Prowse (2007)

Logic models
Logic models or
frameworks, Results
Based Management

Outcome
management

United States (USAID)

Australia (AusAID)
Sweden (SIDA)

http://pdf.usaid.gov/pdf_docs/PNABI452.pdf
http://www.ausaid.gov.au/ausguide/pdf/ausguideline3.3.pdf
http://www.sida.se/shared/jsp/download.jsp?f=SIDA1489en_web.pdf&a=2379

W.K. Kellogg Foundation

https://sites.aces.edu/group/commhort/vegetable/Vegetable/logic_model_kellogg.pdf

Innovation Network

http://www.innonet.org/client_docs/File/logic_model_workbook.pdf

Urban Institute
United Way of America

http://www.urban.org/publications/310776.html
http://www.liveunited.org/Outcomes/Library/pgmomres.cfm
See also Coleman (1987), Gasper(1997), Leeuw and Vaessen (2009) Roche (1999)

Strategy approaches
Balanced scorecards
and strategy maps,
dashboards, and
related tools

NewProfit Inc.
Blue Avocado

http://newprofitinc.net/learn_meas.asp
http://www.blueavocado.org/sites/default/files/Nonprofit%20Dashboard%20article%
20from%20Blue%20Avocado.pdf
See also Kaplan (2001) and Kaplan and Norton (1996, 2004); Butler (2007), Bryson
(2004) and Paton (2003)

50

Participatory and
relationship-based
methods
Outcome mapping

International Development Research

Centre, Canada
Center for Effective Philanthropy
Keystone Accountability
Youth Truth

http://www.idrc.ca/en/ev-26586-201-1-DO_TOPIC.html

Most significant
changes; story-based
evaluation

M&E News (Rick Davies)

Clear Horizon (Jess Dart)

http://www.mande.co.uk/docs/MSCGuide.pdf
www.clearhorizon.com.au/category/publication/msc-publications/

Participatory rural
appraisal (PRA) and
variants

Institute of Development Studies

(Robert Chambers)
International Institute for Environment
and Development
U.N. Food and Agriculture
Organization (FAO)

http://www.eldis.org/go/topics/resource-guides/manuals-andtoolkits&id=11513&type=Document
http://www.planotes.org/

http://www.fao.org/docrep/W3241E/w3241e09.htm

World Bank

http://go.worldbank.org/QAASG4TK80 ;

Constituency feedback
and perception reports

Participatory poverty
assessment

http://www.effectivephilanthropy.org/index.php?page=assessment-tools
http://www.keystoneaccountability.org/about/publicreport/what
http://www.youthtruthsurvey.org/students

See also Chambers (1994), Dart (2003), Earl et al. (2001), Robb (2002)
Integrative approaches
Organizational systems
for impact planning,
assessment,
accountability and
learning

ActionAid International
Keystone Accountability
iScale
Grassroots Business Fund

Complexity science
and systems thinking
approaches

http://www.actionaid.org/assets/pdf/ALPSENGLISH2006FINAL_14FEB06.pdf
http://www.keystoneaccountability.org/sites/default/files/1%20IPAL%20overview%
20and%20service%20offering_0.pdf
http://www.scalingimpact.net/innovations/impact-planning-assessment-andlearning-systems
http://www.gbfund.org/sites/default/files/iPAL%20Framework%20Factsheet.pdf
See also Chambers et al. (2009), David and Mancini(2004), Ebrahim & Weisband
(2007), Guijt (2008), Khagram et al. (2009)

See Ramalingam et al. (2008), Goldstein et al. (2009)

http://aidontheedge.info/

51

Figure 1: A Contingency Framework for Measuring Results

Institutional Results

Complex

policies (on rights and freedoms,

good governance, efficient markets)

Measure outputs and influence

(intermediate outcomes)

Economic development,
comprehensive rural development
and natural resource management,
collaborative development
Measure outcomes and impacts

Integrated Results

Niche Results
Focused

Theory of Change

Change in societal norms and

Ecosystem Results

Basic and emergency services, soup

kitchens, crisis drop-in centers and
hotlines

Service delivery (in health,

education, employment),
immunization campaigns, complex
emergency services

Measure inputs, activities, outputs

Measure aggregate outputs,

outcomes, and sometimes impacts

Focused

Complex

Operational Strategy

52