Evaluating Your Project

You may have a pilot project coming to an end and you want further funding for it. Or perhaps you
already have funding but you need to know if it is working? Or maybe you simply want to know
how you can do it better.

To do this, you want to get findings that are ‘demonstrably true’ i.e. when a ‘solid mix of evidence
supports a conclusion at or above the level of certainty required in that decision-making context.’
(Evaluation Methodology Basics: E. Jane Davidson)

So how can we monitor and evaluate projects, programmes and whole organisations in the
voluntary and community sector?

This briefing takes you through a few key pointers; it is a good idea to find further resources to
support you as you undertake the evaluation.

What are monitoring and evaluation?

Monitoring and evaluation are linked to each other, but they are different
processes.

Monitoring means regularly collecting information on your project that will

help you to answer basic questions about your work. This data collection,
however, must be well planned and organised.

To do this, you can look at quantitative information, i.e. hard facts such as how many people
attended a training session, how much money and time you've spent on the project and how many
people you've reached.

You can also consider qualitative information. This is often harder to measure but can provide
more detailed insights about how the project is going through satisfaction levels, the change in
how people feel about an issue before and after your intervention etc.

Monitoring information can be used to report on your project. A monitoring report commonly
presents the information you gathered with limited analysis and interpretation. This information can
be used to help you evaluate your work as well. However, only through an effective evaluation of
your project you will be able to demonstrate the value of your programme and share good
practices, or to learn from it and improve what you do.

Monitoring is the routine and systematic collection of information against a plan. The
information might be about activities, products or services, users, or about outside factors
affecting the organisation or project. You can use this information to report on your project to
help you evaluate. (Charities Evaluation Services (CES))

1 Updated: October 2015

Registered Charity No. 1105835. Company limited by guarantee in England and Wales No. 5090324
Evaluation is a when you use the information you collected to make judgements about your
project, assessing the changes it created. An evaluation report contains enough evidence to
demonstrate the difference you make and answer important evaluation questions. Evaluation is
usually undertaken for one or two main reasons:
 1.To find areas for improvement
and/or
 2. To look at its overall quality or value.

Evaluation is about using monitoring and other information you collect to make judgements
about the value of any component part of an organisation or its projects, products, services or
benefits, or about the organisation as a whole. It is also about using the information to make
changes and improvements. (Charities Evaluation Services (CES))

You can use the information you've collected during monitoring, but in addition you will need to
collect information to evidence change more effectively. Doing an evaluation will help you see how
and where your project has been successful and how it could be improved. It will let you see how
well funds have been used, and what the benefits of your project have been to those involved. A
good evaluation should help you identify whether you met your outcomes (the change you wanted
to support) and if there were any unexpected outcomes, both good and bad. You can use this
information to improve your future projects.

Why are monitoring and evaluation important?

There are several reasons why monitoring and evaluating your project is particularly important for
voluntary and community based projects:

Accountability
When you win funding from a grant giving
organisation, they will usually make it compulsory for
you to report back on what you're doing and what
you're spending their money on. You might have
agreed targets with them, such as the number of
people your project will reach within a given time. As
well as the outcomes your project aimed to meet. You
will need to monitor your project regularly to collect
this information.

To secure future funding

Funding bodies will want to see strong evidence of the effect your group has (i.e. the outcomes) to
help them decide whether to give you money. Ongoing monitoring and evaluating of your projects
will provide this. It can provide evidence of what a project has achieved and what might be
achieved as this project continues in the future.

To check the project's progress against your original plans

There are two reasons to do this:
1. sometimes needs change over time and what you started with is not working as well as hoped;
2. it can be easy to lose sight of the original aims you had for your project.

Monitoring it on a monthly, quarterly or yearly basis and checking whether you're still on course to
achieve your goals will prevent this. It will also help you keep an eye on whether you're keeping to
your timescales and budgets.
2 Updated: October 2015

Registered Charity No. 1105835. Company limited by guarantee in England and Wales No. 5090324
To learn from your experience and innovate
Proper monitoring and evaluation of your project means you can see what has worked well and
what hasn't. You can then use this information to improve future projects or funding applications.

To motivate staff and volunteers

Staff and volunteers are likely to be involved in the monitoring and evaluation of a project – this
can help give real ownership over a project. Showing your volunteers and staff real, concrete
evidence of the impact their hard work has had will make them feel great. It can also encourage
them to continue working with your group, or to take on new projects.

How can you monitor and evaluate your project?

Before a project even begins you should be thinking about how you will monitor and evaluate it.

Exactly what do you want your project to achieve? It is important to know exactly where you
are going before starting planning your evaluation, or even planning your activities. You will only
be able to assess whether your project is being successful in having the desired impact on the
beneficiaries if you are able to articulate what this impact means. For that, you will need to be able
to describe your project in detail.

It can be helpful to develop a theory of change. The theory of change is about mapping all the
small changes your project brings about in a way that it is possible to measure. This can also be
called your programme theory; results based management or impact map.

This process will link your services to your impact in a logic model, taking into consideration most
of the possible effects your work has. The most common terms used to evaluate VCS’s project are
‘input’, ‘outputs’, ‘outcomes’ and ‘impact’.

Input: the resources used for delivering your services

Outputs: the direct products and services you deliver as part of your work

Outcomes: the changes (effects) that occur as a result of your work – as a direct result of
project outputs. They can be wanted and unwanted, expected or unexpected.

Impact: the effect of a project at a higher or broader level, in the longer term, after shorter
terms outcomes have been achieved. Impact often describes change in a wider user group
than the original target, and many organisations may play a part in achieving impact.

The relationship between these terms can be demonstrated in your theory of change. The picture
below describes how these terms are linked.

3 Updated: October 2015

Registered Charity No. 1105835. Company limited by guarantee in England and Wales No. 5090324
The input refers to resources that will enable you to deliver your activities and its outputs.
Therefore, outputs are the direct results of your activities. Outputs are important sources of
monitoring data, which will help you to describe how your resources were used and how the
project was delivered. These bring about outcomes, which may eventually lead to the impact.
Therefore, outcomes are the change, benefit, learning or other effects that actually do occur as a
result of your activities. In order to evaluate your project you will need to measure outcomes, as
these can explain why your project is needed and why it benefits the recipients.

See the example below describing how the input may lead to impact

From inputs to impact: The Women’s Project

The Women’s Project works with female sex workers and with local healthcare
services.
Inputs
Staff salary costs Staff travel costs Office costs Outing expenses
 
Outputs for healthcare providers Outputs for women
Training One-to-one support sessions
Group work
Outings
 
Outcomes for healthcare providers Outcomes for women
Improved service delivery Increased confidence
Increased awareness of alternative lifestyle
options
Improved access to education and training
opportunities
Increased ability to keep themselves safe
4 Updated: October 2015

Registered Charity No. 1105835. Company limited by guarantee in England and Wales No. 5090324
 and healthy
 
Impacts
Some women build new lives away from sex work
Improved local health care services for local sex workers
Source: CES Monitoring and Evaluation on a shoestring p. 11 http://www.ces-vol.org.uk/shoestring.html

If you are interested in developing the theory of change for your project, you will find useful
information at the Centre for Theory of Change website http://www.theoryofchange.org/ . There
you will also be able to use an online learning tool that will help you to create and implement your
theory of change.

Creating your outcomes framework

Once you know all the details of the project and have mapped how the changes come about, you
can look at how you're going to monitor and evaluate each of the different areas of your project.
Doing this thinking upfront should mean that you won't miss opportunities to gather information
along the way.

As a voluntary and community organisation you are likely to need to collect information on the
following four key areas:

 the services you offer (your outputs)

 the people you work with (diversity)
 what stakeholders, particularly users, think of your work (user satisfaction)
 the changes you bring about (your outcomes).

Organising your data in a framework will help you to visualise the type of information you will need
to collect, so you can plan how, when and who will be in charge of the process.

In order to complete your framework, you will need to define indicators and data collection
methods.

Outcome indicators are signs or signals that can be used to determine if a given outcome has
occurred or been achieved. These indicators can be qualitative or quantitative.

These are some suggestions of headline to use in your framework:

For measuring activities and outputs:

Activity – Outputs Output How will When will Who will How this
the area indicators we collect it be collect it? information
of work (what will it? collected? will be
we plan to we used?
deliver measure)

For measuring outcomes

5 Updated: October 2015

Registered Charity No. 1105835. Company limited by guarantee in England and Wales No. 5090324
Aims – Outcomes Outcomes How will When will Who will How this
the area indicators we collect it be collect it? information
of change (what will it? collected? will be
we want we used?
to create measure)

Charities Evaluation Services (CES) developed an easy tool to help you to create your outcomes
framework (define outputs and outcomes for your project and demonstrate the difference you
make) - ‘The Planning Triangle’. You will find detailed information about this tool on CES’s website
http://www.ces-vol.org.uk/tools-and-resources/planning-for-monitoring-evaluation . There you will
also find examples of different triangles and frameworks.

How can you collect the appropriate data of your project?

It is a common mistake when setting up a monitoring system to start with methods of data
collection. This misses out a vital step of deciding what information you need first. Only when
you have your outcomes and outputs defined you can choose appropriate methods. If you
think through your information needs systematically first, you are also more likely to limit your
monitoring to the collection of useful data.
(Charities Evaluation Services (CES))

In order to be able to monitor and evaluate your project, you are likely to gather a lot of
quantitative information: how much does it cost, how many people attended, over what time
period, what percentage of people said your intervention was helpful etc.

You will also need to collect qualitative information to find out more in-depth information about the
change in the people who used your services and the impact you have.

There are a number of ways to collect data, these are some methods available:

Questionnaire or survey
These are simple to put together and distribute. You can reach a lot of people quickly and easily,
although you will need to send out to a high number of people as not everyone will reply.

You can have questionnaires with multiple choice boxes people can tick, and you can ask them to
give you more detailed replies. The questionnaire can be anonymous if you think you'll get better
responses. However, you need to make the questions as clear as possible so that you get exactly
the information you need, but be sure not to make the questions leading. You'll need to analyse
the information once you've collected all the questionnaires to get an overall picture of what people
think. Questionnaires can be especially useful for monitoring attitudes or behaviour change.

You can try https://www.surveymonkey.com/ which has free options as well as payment plans.

Feedback forms
When holding events and training, it is a good idea to ask attendees to fill in a feedback form. If
you're holding an event, why not ask people to fill in a feedback form at the end? If you are running
training, you may want to gather attendees’ thoughts and skills prior to the event and after to
measure any change through the event.

6 Updated: October 2015

Registered Charity No. 1105835. Company limited by guarantee in England and Wales No. 5090324
The event will be fresh in their mind, so you should get accurate responses. However, some
people might rush the form if they're in a hurry to leave, so do offer them the option of emailing it
back to you. You can use their feedback to improve your future events.

Interviews
Although they can be time consuming, interviews carried out face-to-face or on the phone are a
great way to get detailed information on what people think of your project.

Interviews can be formal with a set list of questions that don’t deviate too much or semi-formal
where you have a few key questions but let the conversation lead and can ask other questions. 1-
2-1 interviews can be really useful for gaining a full idea of an individual’s experiences before and
after using your services.

Focus groups
Getting a group of people together to discuss your project is an
easy way to gather information. Bringing a group together, rather
than just interviewing individuals, means people can spark new
ideas in each other, and you often get better responses. However,
you need to think carefully about the information you want to
collect from your focus group, and continually steer the
conversation in that direction. It's easy to let the discussion drift off
track.

Planning for evaluation

Identifying your expected outputs and outcomes is part of good project planning. When you are
doing this, it may be helpful to look at the existing aims and objectives of your project to help you
define them. It is also helpful to involve different people from your organisation when planning your
evaluation.

Asking yourself the following questions before beginning your evaluation will help you do it well:

1. What is the purpose of your evaluation? What do you want your evaluation to demonstrate?

2. Who are you evaluating for? Is it for your funders, the community, or your organisation?

3. What are you going to evaluate? Which elements of the project do you intend to look at?

4. How are you going to evaluate? What methods are you going to use to carry out your
evaluation? You should use a range, and collect both quantitative and qualitative data.

5. How will you collect the data you need? Is the data available? If you need data from across the
project, have you set up monitoring to capture it?

6. What will you do with the data you collect? Data by itself is just data; you'll need to analyse it to
come to a conclusion about what it shows. You'll also need to consider how you're going to
present the analysis and your conclusions: different people will want to know different things in
ways appropriate to them i.e. funders may have a template or want a full report, possible service
users may want a video on the website etc.

7 Updated: October 2015

Registered Charity No. 1105835. Company limited by guarantee in England and Wales No. 5090324
7. How will you act on your analysis? All too often the findings of monitoring and evaluation end up
gathering dust on a shelf. Make sure your findings see the light of day. Use them to plan for the
future.

And finally…
Be realistic about how long it's going to take to see
effects from your project. Don't expect instant results.

You can measure instant outputs but outcomes will take

longer and impact will take longer still.

Also, remember that you may not be the only

organisation working with the individuals so be careful in
taking full responsibility for change, unless they say
otherwise it is more likely that your organisation/project
contributed to the change.

Monitoring and evaluation should be seen as something positive by everyone involved in the
project, not something to worry about. It is a chance to make your project or organisation even
better, to save money and resources if things are not working well and to really involve staff,
volunteers and service users in the future of your organisation.

Resources
 Charities Evaluation Services
 ‘Evaluation Methodology Basics: The Nuts and Bolts of sound Evaluation’: E. Jane
Davidson: (2005): Sage Publishing
 Inspiring Impact
 United Nations Development Programme: Handbook on Planning, Monitoring and
Evaluating for Development Results

Support
 If you would like any support with evaluation and impact please contact Mariana Meirelles
at [email protected] or for any other issues facing your organisation,
please contact the Development Team at Community Southwark:
[email protected] or 020 7358 7020.

8 Updated: October 2015

Registered Charity No. 1105835. Company limited by guarantee in England and Wales No. 5090324