Annex E - ACIAR Capacity Building Monitoring and Evaluation Framework PDF
Annex E - ACIAR Capacity Building Monitoring and Evaluation Framework PDF
Table of Contents
1 Introduction and Overview .................................................................................................. 1
2 MEL Framework Overview ................................................................................................... 1
3 Principles .............................................................................................................................. 2
4 Theory of Change ................................................................................................................. 3
5 Utilisation ............................................................................................................................. 8
6 Tools and Methods .............................................................................................................. 9
7 Information Management ................................................................................................. 14
8 Resources and Responsibilities .......................................................................................... 14
9 Updating the MEL Framework ........................................................................................... 15
10 Matrix of Evidence ............................................................................................................. 16
ACIAR Capacity Building MEL Framework
Updated March 2020
1
ACIAR Capacity Building MEL Framework
Updated March 2020
3 PRINCIPLES
This MELF and its implementation will be guided by a set of important principles:
The MEL system will meet the needs of program management, planning and
Useful
communication, with relevant and timely information.
The MEL system and its methods are proportional to the scale, value and
Proportional
strategic priority of each the Program and its sub-programs.
All MEL activities will reflect high standards of ethics in the collection,
Ethical
management, analysis and use of data3.
Gender, disability and other social exclusion factors will be addressed in MEL, to
Inclusive
monitor the extent to which activities are inclusive (or not).
1
Some programs and activities also funded through the Capacity Building Program are initially out of scope.
While most are small and ad hoc, the Capacity Building Unit determined that the funding provided to The
Crawford Fund ($1 million per year, and which includes support for the RAID Network), would not be included.
An annual Memorandum of Understanding is negotiated with the Crawford Fund, and while it is understood
that this does not provide any MEL or reporting obligations, or articulate specific intended outcomes, it does
present an opportunity to introduce MEL obligations at some point in the future.
2
Estimated for 2019/20, and totalling $8.4 million in that financial year.
3
For example, the Australian Evaluation Society:
https://www.aes.asn.au/images/stories/files/membership/AES_Guidelines_web_v2.pdf
2
ACIAR Capacity Building MEL Framework
Updated March 2020
4 THEORY OF CHANGE
A 2017 Review of Capacity Building4 observed that: “Within ACIAR there is a consistent
recognition that capacity building is an integral part of the business model…However [there]
is a range of views about what are the objectives” (p12) and that there is: “…a consistently
narrow view about what are capacity building activities”, focusing on formal education and
training, not ‘informal’ learning-by-doing. Analysis that informs this MELF corroborates this:
there is commitment to capacity building, it is prominent in ACIAR’s organisational identity
and mandate, but it is not systematically approached, managed or assessed.
ACIAR’s most formal, and most recent, articulation of intent with regards to capacity building:
the Capacity Building Policy5, responds to this view by establishing an articulation of what
capacity building is, within ACIAR. According to the policy, capacity building includes “on-the-
job training, leadership, mentoring, two-way-transfers of ideas and technologies, and
empowerment to undertake research” and recognises that capacity building is much more
than just building skills and knowledge through training, and that it occurs at individual,
organisational and institutional levels6. But the Policy omits any reference to the expected
outcomes – what will be changed – which result from investments in capacity building.
Therefore, to strengthen program planning and management, the Capacity Building Program
developed a theory of change for its work. This theory articulates the outcomes ACIAR seeks
to achieve through its formal capacity building investments, and is the basis for the
monitoring, evaluation and learning framework documented herein. The theory of change
was developed within the Capacity Building Program, and then ground-truthed and refined
with input from a wider group of ACIAR internal stakeholders, including the Chief Executive
Officer, Chief Scientist, and Research Program Managers. It is designed to align with and
contribute to the overall Results and Performance Framework which are in development for
ACIAR as a whole, and can be adjusted as necessary as that Framework is finalised.
For a program to have robust monitoring and evaluation, and to enable sound program
planning, management and implementation, it is essential that there is shared clarity of
intent. What this means is that the key stakeholders in a program, including staff and senior
management, should have a common understanding of what the program is aiming to
achieve and the broad strategy for how it will get there. The Capacity Building Program
theory of change is designed to provide this clarity of intent. It illustrates – in diagrammatic
form– the changes sought (the intended outcomes) and summarises the pathways towards
those changes. A theory of change is not an implementation plan, nor a blueprint for
delivery, but a conceptual model based on logic which can provide the underlying scaffolding
for a program and its management, and communication of program intent.
The theory of change is conceptualised as a sequence whereby inputs and activities generate
capabilities and outputs. These in turn deliver direct outcomes which – through utilisation
and influence – should contribute to significant indirect outcomes. Within the theory there
are interconnections, reinforcing influences, and feedback loops which are important in its
success. Ultimately the outcomes of the CBP are designed to contribute to ACIAR’s six overall
4
Coffey, S. 2017.High-Level Review of ACIAR’s Capacity Building Program and Related Activities
5
ACIAR. March 2016. Capacity Building Policy.
6
ibid.
3
ACIAR Capacity Building MEL Framework
Updated March 2020
objectives as set out in the ACIAR Ten Year Strategy 2018-20277 - the development impact.
To contribute to this impact, the CBP must work in tandem with the other things ACIAR does,
most especially its research projects around the world, and it must function effectively within
the context of each partner country and partner organisation.
At a summary level, the CBP theory of change is as follows:
Inputs and Impact – ACIAR
Capabilities Direct Outcomes Utilisation Indirect Outcomes
Activities Effect Objectives
and Outputs Results and Influence Results
Performance Results
Institutions undertake
more and better quality
Institutions have a more
agricultural research,
skilled, qualified and
including through
diverse workforce, and
collaborations with ACIAR
valuable links and
and other Australian Food Security
networks
institutions, and continue and Poverty
to grow and develop Reduction
Natural
ACIAR provides Resources and
Agriculture professionals
education, training, Climate Change
Agriculture professionals (all genders) advance in
work experience and
(all genders) have higher their careers and Human Health
mentoring support for
qualifications, skills, more undertake good quality and Nutrition
agriculture
experience, and expanded agricultural research
professionals in Indo- Gender Equity
links (including with ACIAR) and
Pacific and Women’s
policy work
Empowerment
ACIAR Research Projects contribute to, and benefit from, capacity building in the Indo-Pacific Inclusive Value
Chains
Capacity
Building
ACIAR has links with skilled
ACIAR projects are
agriculture professionals in
effective and ACIAR has
the Indo-Pacific, and
strong links, understanding
knowledge and
and reputation in Indo-
understanding about
Pacific
building capacity
While it is presented diagrammatically in linear form, the Capacity Building Program is deeply
conscious of the fact that capacity building is far from a linear process. In practice, achieving
the changes described by the outcomes will be much messier and less predictable than the
theory of change implies. But as a conceptual structure around which to build the
monitoring, evaluation and learning framework, as well as a means to communicate the
intentions of the program as a whole, the theory of change is useful.
The theory recognises that there are three inter-connected domains of change: changes for
individual agriculture professionals, changes for the institutions in which they work, and
changes for ACIAR itself. Change in one domain is often influenced by changes in other
domains. For example, many of the outcomes sought for individual agriculture professionals
cannot happen unless they have opportunities and support within their institutions.
Therefore, for the Capacity Building Program to be most effective, it must engage with those
institutions in ways which enhance opportunities and support for ACIAR alumni. Similarly,
ACIAR cannot continue to work with its alumni in research projects if they are not remaining
in their scientific fields and within their home countries, so the program must maintain
contact with them and find ways to continue support them. The theory of change is detailed
further in the attached diagram below. It provides more comprehensive articulation of the
intended outcomes of the Capacity Building Program.
7
https://www.aciar.gov.au/publication/Ten-Year-Strategy
4
ACIAR Capacity Building MEL Framework
Updated March 2020
Networking Agriculture professionals have an expanded world view Agriculture professionals share their expertise and knowledge and retain
opportunities and greater life experience links with ACIAR, other alumni, and their wider networks
Gender Equity
Extension Agriculture professionals increase their scientific output, Agriculture professionals demonstrate and are advocates for gender and Women’s
opportunities including publications equality, diversity, merit-based management Empowerment
(conferences etc)
Alumni activities Agriculture professionals have increased understanding of Agriculture professionals work in ACIAR projects and with Australian
gender equality, diversity etc partners Inclusive Value
Chains
ACIAR Research Projects
5
ACIAR Capacity Building MEL Framework
Updated March 2020
Progress towards these outcomes will depend on the performance of the sub-programs
which comprise the Capacity Building Program. Not all of these will contribute to all
outcomes. Some, for example, may be more narrowly focused on supporting individual
researchers, regardless of their institutional links. In this case the MEL for that sub-program
would focus on outcomes for those researchers. Other sub-programs, however, will have a
more explicit focus on supporting institutional as well as individual development, so their
MEL will be broader and encompass more of the outcomes.
It is important to note that the focus of the Capacity Building Program – and in this case, its
monitoring and evaluation – extends as far as the Indirect Outcomes. Impact (in this case
meaning the achievement of ACIAR objectives more broadly) is beyond the line of
accountability for the Capacity Building Program: because change at this level is the result of
countless enabling and constraining factors and it is impossible to meaningfully identify the
contribution of the capacity Building Program. As such, tools, methods or sources of evidence
are not included in this MELF to assess impact. However future impact studies undertaken by
the ACIAR Impact Assessment Research Program would include consideration of the
contribution of relevant Capacity Building Program activities.
The theory of change provides the structure for the collection, analysis and use of data
through the MELF. The matrix in section 10 below maps the source of evidence for each
element in the theory of change, and section 6 outlines each of the methods which will
provide that evidence.
Capacity Building Program and Poorly articulated sub-program Working with service providers
its service providers have clear designs, or those without in the design and management
and mutually agreed intentions sufficiently detailed program of all relevant sub-programs,
for each sub-program. designs, will lead to confusion or requiring stronger sub-program
uncertainty about sub-program designs and explicit attention to
aims, undermining effective monitoring, evaluation and
management and learning within them, and
implementation and causing ensuring that resources are
difficulties in the relationship allocated to support MEL within
between ACIAR and service sub-program budgets and
providers. workplans.
There is a culture of openness Service providers may feel they Taking a constructive approach
and trust within ACIAR and cannot raise challenges or to managing sub-contracts
between ACIAR and service instances of under-achievement which demonstrates openness
providers which enables robust because of the risk of negative and values transparency and
discussion and reflection about reactions from ACIAR, which will genuine learning, and which also
the successes, failures, reduce the extent to which they explores ways to provide explicit
challenges and opportunities can genuinely work with ACIAR incentives within contracts
6
ACIAR Capacity Building MEL Framework
Updated March 2020
All relevant sub-programs within Sub-programs do not have Ensuring that all sub-programs
the Capacity Building Program sound MEL arrangements which develop and implement
will contribute evidence towards align the overarching CBP MELF, appropriate MEL arrangements
the CBP MELF. or which are not resourced which align with and feed into
adequately, or not effectively the overall MELF
implemented.
Including contractual obligations
on service providers which
provide incentives for effective
MEL
ACIAR Research Programs If the CBP and the research Strategic engagement with
engage effectively with the programs are not well linked, Research Program Managers,
Capacity Building Program and with capacity building activities proactively sharing evidence of
its relevant sub-programs. purposefully integrated within Capacity Building Program
projects from their design outcomes, bringing learning to
onwards, there is a risk that bear on project design, review
positive outcomes will be lesser and approval processes, and
than they would otherwise be. providing practical and creative
suggestions about effective ways
to link research and capacity
building.
There is sufficient provision for Insufficient MEL resources will ACIAR has allocated internal
MEL activities within the mean that data is not collected, resources to support the
Capacity Building Program analysed, reported or used for implementation of the
budget, and within the budgets management or improvement. overarching MEL Framework,
of sub-programs. and is progressively introducing
MEL resources into each sub-
program, starting with those
currently being designed and
contracted.
8
These characteristics of a sound MEL system were utilised in the internal review and analysis of existing
monitoring, evaluation and learning arrangements supporting the Capacity Building Program, which was
completed in December 2018 and which led to this MELF.
7
ACIAR Capacity Building MEL Framework
Updated March 2020
new institution-focused John Dillon Fellowship (iJDF), the next phase of the USP-PSS program
and a new Meryl Williams Fellowship Program. This MEL Framework, including the theory of
change and the tools and methods for M&E and learning (see below), will guide the
finalisation of those program designs. This will enable the Capacity Building Program to
manage an increasingly effective and coherent program that delivers an expanding evidence
base as well as stronger outcomes.
5 UTILISATION
Everything within the MELF is designed with an eye to its use, and usefulness, in support of
program planning, management, improvement and learning. Utilisation of the evidence
created through MEL broadly falls into three categories aligned with the purposes of the
MELF: Management, Learning, and Improvement, accountability (reporting) and
Communications (outreach).
8
ACIAR Capacity Building MEL Framework
Updated March 2020
outcomes which can be used in outreach and communications products and activities, as a
result of the implementation of the MELF.
Disaggregation Variables
As the MEL system for the Capacity Building Program is consolidated it will progressively
systematise the collection of data, including the use of consistent definitions for key variables
which will enable aggregation and disaggregation of data across sub-programs, as well as
simple analysis such as cross tabulations.
All sub-program service providers will align their relevant M&E tools to ensure that data is
collected in alignment with the following key variables:
Variable Definition
Gender People will uniformly be asked to indicate their gender on all forms as follows:
• Male
• Female
• Prefer to specify another way
Age Information about individual participants will include details that enable analysis and
reporting of the ages within cohorts.
Highest level of Individuals will be consistently asked to indicate their previous highest qualification
previous as follows:
qualification
• Certificate or Diploma
• Bachelor degree/ undergraduate
• Postgraduate
• PhD
9
ACIAR Capacity Building MEL Framework
Updated March 2020
Variable Definition
Disability At least initially, the requirement to consistently report disability status for people
involved in the Capacity Building Program will rely on existing practices within service
providers. The key obligation however will be to indicate disability status – as a
simple yes/no – for all participants.
Collecting accurate disability identity data is notoriously challenging and international
best practice involves the use of the Washington Group Short Set of questions9.
Designed for surveys and censuses they have also proven effective in application and
registration forms, including in the Pacific region. Introducing this approach to
disability data collection does require investment in personnel skills and
understanding, so where this form of questioning is not already in use, the Capacity
Building Program will consider its introduction in future.
Institution type When a person provides the name of the institution at which they work, they must
also indicate what type of institution it is, from the following categories:
• University or tertiary institution (i.e. awards formal qualifications)
• Government ministry or department
• Government research institution (does not award formal qualifications)
• Private research institutions (does not award formal qualifications)
• NGO, development or other organisation not specified above
This categorisation will enable sampling and analysis to explore whether there are
differential experiences or outcomes for people working within different types of
institutions.
9
http://www.washingtongroup-disability.com/wp-content/uploads/2016/12/WG-Document-2-The-
Washington-Group-Short-Set-on-Functioning.pdf
10
ACIAR Capacity Building MEL Framework
Updated March 2020
Service Provider monitoring tools (e.g. participant evaluations, online modules and
surveys, participatory methods)
Service providers utilise a range of methods to collect feedback from participants in training,
workshops and other activities within capacity building programs, and to monitor their
progress. Each has their own format for these, with individually-developed questions,
categories, ratings and other components10.
To progressively bring this important data together in a more consistent way the Capacity
Building Program will collect and compare the various forms and other tools used across the
different service providers, to analyse the extent to which they are in alignment, and to
which they diverge. Based on this initial comparison of existing tools, specific areas for
refinement will be identified, and negotiated with service providers. As with the
Application/Registration Forms, the aim is not to mandate specific monitoring methods or
instruments. Instead the aim is to achieve consistency across tools and service providers in
common areas, such as key questions and definitions of variables. This will maximise
opportunities for data aggregation and disaggregation, while leaving the necessary scope for
sub-program-specific monitoring tools to continue to be used.
Tracer Studies
Because the individual people supported through the Capacity Building Program are central
to so many of the intended outcomes, data about their experiences and their outcomes are
essential in the monitoring, evaluation and management of the program. Thus a routine cycle
of tracer studies is an essential component of the MELF, examining the experiences and
career paths of graduates from long- and short-term fellowships. The Capacity Building
Program will design and plan a robust tracer study which will be implemented regularly. The
design will be detailed separately but will aim to collect evidence at one year, three years and
ten years after an individual completes an ACIAR fellowship.
Tracer studies are reliant on sound alumni data to enable contact with as many alumni as
possible. In the absence of an alumni database a tracing exercise may be needed as the initial
step in a tracer study, but the relatively small number of ACIAR alumni and the fact that they
10
For example, in managing the JAF Executive Leadership program, University of New England is monitoring
participant progress through verbal M&E feedback, interactive online modules and surveys, and a post-program
evaluation. These would continue to be used but with consistency in core areas with those in other sub-
programs, where it is meaningful and feasible.
11
ACIAR Capacity Building MEL Framework
Updated March 2020
are concentrated in the agricultural sphere should make this a more feasible undertaking
than it might otherwise be.
A formal tracer study report will be prepared each year, suitable for publication. It may
include individual and/or institutional case studies (see below) and will be a centrepiece of
the Capacity Building Program MEL system. The tracer study approach will be piloted in 2019-
20, and will also feed into the planned review of JDF and JAF programs, which will
subsequently inform a refreshed design for both of these flagship programs.
12
ACIAR Capacity Building MEL Framework
Updated March 2020
Careful selection of institutions for case studies will be important, and will be done in
collaboration with ACIAR Country/Regional Offices and the Country Program, as the nature of
institutional engagement varies significantly both in depth and duration.
13
ACIAR Capacity Building MEL Framework
Updated March 2020
The research study would require comprehensive research design and development. It may
be best approached as a multicase study11, examining capacity building through research
projects in a number of different fields and countries. The aim would be to embed the
research within a sample of current research projects. It would, over time, investigate the
ways that capacity is built, what enables and constrains this process and the eventual use of
new capacity, and identify ways that research projects can be supported so their capacity
building efforts can be most effective.
Other ACIAR research projects or impact studies, commissioned outside the Capacity Building
Program, may also serve the aims of such a longitudinal study to provide evidence on
effective approaches to capacity building. In that case the Capacity Building Program will
work closely with any such projects.
7 INFORMATION MANAGEMENT
At present there is no single information management system within ACIAR, nor within the
Capacity Building Program. Capacity Building Program data is held independently within
individual service providers, relating to the specific sub-programs they implement, and data
held within ACIAR is fragmented.
Consolidation of information management across the Capacity Building Program will be a
medium-term undertaking. The first step is the preparation of this MELF and the clearer
definition of the types, formats, and categories of information that is included within it.
Service providers will be required to collect, analyse and manage their data in alignment with
the MELF, and to communicate it to ACIAR through reports and other outputs in ways which
increasingly allow the Capacity Building Program to meaningfully aggregate data.
During 2020 the Capacity Building Program will develop and implement a new Alumni
Database which will substantially strengthen the management and use of alumni data, and
bring a consistent global approach to alumni data across ACIAR for the first time. This will
strengthen several aspects of this MELF, especially the implementation of Tracer and Case
Studies of alumni.
Over time, the Capacity Building Program will establish more robust information
management systems which support the consolidation of data across activities. This may
include initially considering how to establish and maintain an effective alumni database.
11
Stake, R.E. Multiple Case Study Analysis. The Guildford Press, NY. 2006.
14
ACIAR Capacity Building MEL Framework
Updated March 2020
15
ACIAR Capacity Building MEL Framework
Updated March 2020
10 MATRIX OF EVIDENCE
The follow table, aligned with the Capacity Building Program theory of change, documents the sources of evidence which will be routinely used
to monitor and assess progress towards the intended outputs and outcomes.
The component sub-programs will use this matrix as a menu of options to identify the relevant tools and methods for deployment within their
individual M&E systems, and will supplement these with additional tools and methods specific to each sub-program, as required.
1. Academic study (long- and Who is applying (scientific field/ gender/ Service provider data Service Provided to ACIAR in regular reports
short-term) location/ institution type/ disability etc)? collection through Providers (e.g. at end of each selection cycle or
2. Mentoring application/ annual program delivery)
What are the more and less popular
3. Skills training (technical, assessment/
programs (long-term/ short-term etc)? Used in discussion between CBP and
scientific) registration forms
SP to agree any adjustments in future
4. Skills training (leadership, Who is being selected?
CBP MELF provides selections to address any gaps or
management, Where are people studying or being trained? definitions of key inequities identified
communications, policy)
What sorts of organisations are hosting work disaggregation
5. Work experience Used by CBP to inform learning
variables which will
(placements and placements and internships? across sub-programs
ensure consistent
internships Who is dropping out and who is completing? Used by CBP to identify potential
data collection across
sub-programs issues requiring further investigation
or evaluation
6. Networking opportunities Who is participating in networking Alumni Annual Country Offices
7. Extension opportunities opportunities (both participants/fellows and Reports
Service
(conferences etc) other people)?
Providers
What sorts of extension opportunities are
most popular?
8. Alumni Activities Which alumni are more or less active in Alumni Annual Country Offices Provided to CBP at the end of each
alumni networks and activities? Reports alumni event
16
ACIAR Capacity Building MEL Framework
Updated March 2020
9. Institutions have a more Where are ACIAR alumni working? Tracer Study CBP CBP will aggregate and analysis,
skilled, qualified and provide report internally and through
Which institutions have multiple alumni?
diverse workforce internal learning processes
Issue: establishing a baseline against
which to measure ‘more’
10. Institutions have stronger In what ways are institutions which employ Institutional Case CBP As above
networks with Australian ACIAR alumni connecting with Australian Studies
research institutions and partners?
potential commercial
partners
11. Agriculture professionals How many individuals have been awarded a Service Provider Service CBP will monitor the completion
(all genders) have higher formal qualification as a result of CBP reports Providers (success) rates to identify any
qualifications support? patterns of under-performance by
service providers or particular groups
What sorts of individuals did not complete a
of researchers.
qualification, if any?
Aggregated data will be used in
(disaggregated by levels of qualification,
internal reporting and public
gender/age of researchers etc)
communications.
12. Agriculture professionals How relevant were the skills and knowledge Individual feedback at Service Monitoring the relevance of the new
(all genders) have increased gained through ACIAR support? end of program Providers skills and knowledge will inform CBP
skills and knowledge (Feedback Forms) and service provider decisions about
Are there certain skills which have been more CBP
(research, scientific, cross-
(or less) useful in practice?
17
ACIAR Capacity Building MEL Framework
Updated March 2020
13. Agriculture professionals How do agriculture professionals feel about Evaluation Forms Service Service providers and CBP will use
(all genders) are more their career plans and prospects providers this data to understand the elements
confident and ambitious (immediately on completion, and then again within capacity building activities
Tracer Study CBP
some time later)? which are most effective at building
non-technical outcomes which are
essential for career progression – and
to maximising these within programs.
14. Agriculture professionals To what extent are agriculture professionals Tracer Study CBP Service providers, Country Offices
(all genders) have useful (alumni) maintaining their links with the and CBP will use insights into if and
links with people and people and organisations they met during how links are being sustained to
organisations their ACIAR program? inform choices about what sorts of
networking and collaboration
(disaggregated by Australians/ other alumni/
activities are supported.
Australian orgs/ others)?
15. Agriculture professionals How have agriculture professionals changed Tracer Study CBP CBP will use insights as to these
(all genders) have an their general views about the world? broader personal changes in
expanded world view and communications and learning about
greater life experience the benefits of capacity building
programs.
18
ACIAR Capacity Building MEL Framework
Updated March 2020
16. Agriculture professionals How many publications are researchers Application/ Service This will guide CBP assessments of
(all genders) increase their producing? (during the program/ in the first Registration Form (for providers the extent to which service providers
scientific output, including year/ three years?) baseline) are effectively supporting
publications researchers as scientists, and the
Service provider
CBP extent to which links with ACIAR
monitoring tools
research projects are contributing to
Tracer Study scientific output.
17. Agriculture professionals In what ways have agriculture professionals Service provider Service CBP will use insights as to these
(all genders) have increased shifted their thinking about equality and monitoring tools providers broader attitudinal changes in
understanding of gender diversity? communications and learning about
Tracer Study CBP
equality, diversity etc the extent to which alumni are
becoming active advocates for
greater equality and inclusion, and
contributing to these changes within
their fields and institutions.
18. There is a diverse pool of Who are the ACIAR alumni in the region and Tracer Study CBP CBP will share this information with
agricultural professionals in where are they working? ACIAR colleagues to support ongoing
Alumni database
the Indo-Pacific with skills networking and recruitment of
and experience relevant to scientists for projects and
future projects who have collaborations, and will include it in
links to ACIAR reports and public communications.
19. ACIAR learns and building How is ACIAR making use of its capacity Internal Reflection CBP CBP will facilitate learning and
knowledge from experience building experience? and Learning internal communications which
supporting and shares the experience of effective
Publications
collaborating in CB capacity building within ACIAR, and
activities Longitudinal Research will also encourage participation in
Project MEL (including the longitudinal
research study), so ACIAR staff and
19
ACIAR Capacity Building MEL Framework
Updated March 2020
20. ACIAR has evidence, How are the communications team utilising ACIAR CBP Strategy and Capacity Building use
learning and evidence from M&E in the production of Communications evidence and stories from MEL in
communications content communication collateral? Team feedback public and strategic communications.
generated through CB
How well is the MELF supporting the Internal Reflection
activities
collection and provision of information which and Learning
is useful for communications and learning?
What is the CBP doing to support the influential use of evidence arising from the Internal Reflection CBP CBP will reflect on these questions in
CBP? and Learning its own planning and management to
aim for its MEL work to have
How effective is this proving to be? Internal Survey
maximum value, both internally and
How are research programs and their partners responding to the evidence externally.
arising out of CBP?
21. Institutions undertake good What sort of research are ACIAR alumni and Institutional Case CBP CBP will use this evidence in
quality agricultural research their institutions doing? Studies reporting (accountability),
for development communications, and to inform
How are alumni contributing to, or leading, Tracer Studies
decisions about ongoing and future
this research?
Researcher Case capacity building activities. This will
Studies include discussions with service
providers and Research Program
22. Institutions secure How are institutions expanding their research Institutional Case CBP Managers about how to design and
additional research and/or programs or organisational resources and to Studies deliver capacity building which makes
organisational funding what extent is their ACIAR support effective contributions to
Tracer Studies
contributing to this?
20
ACIAR Capacity Building MEL Framework
Updated March 2020
23. Institutions support alumni How are the institutions where alumni work Institutional Case CBP
to build their peers’ supporting them to share their new skills and Studies
capacity and share their knowledge?
Tracer Studies
knowledge
Researcher Case
Studies
24. Institutions support and For those institutions which have a policy Institutional Case CBP As above
advocate for agricultural role, how are they advocating for agricultural Studies
research research, and to what extent has ACIAR
Researcher Case
support contributed to this?
Studies
25. Institutions have a more In what way are institutions changing the mix Institutional Case CBP CBP will use this evidence to
gender diverse workforce, of people in their workforce, and how are Studies understand whether capacity
including at leadership and alumni contributing to that? building sub-programs provide
Tracer Studies
management levels enough additional support and
incentive for institutional behaviour
change relating to the management
of human resources, and thus to
make changes to the program if
needed.
26. Agriculture professionals How have alumni careers progressed over Tracer Studies CBP Understanding the enablers and
(all genders) are promoted, time? barriers to career progress for alumni
Researcher Case
secure new and better will help CBP and service providers
How do alumni view the contribution ACIAR Studies
employment, win other adjust capacity building sub-
support has made to their career pathways?
academic scholarships or programs, and/or identify necessary
opportunities other links and strategies, to
21
ACIAR Capacity Building MEL Framework
Updated March 2020
27. Agriculture professionals What research are alumni undertaking? Tracer Studies CBP Understanding the enablers and
(all genders) undertake barriers for alumni to continue doing
Who are they collaborating with?
good quality research, good quality research in their home
including collaborations To what extent is this research utilising the countries will help CBP, service
skills, knowledge and networks developed providers and ACIAR more generally
with ACIAR support? identify ways to support alumni after
their capacity building.
CBP will also share this evidence
within ACIAR and encourage research
projects to also support alumni to
continue undertaking research.
28. Agriculture professionals How are alumni sharing their expertise and Tracer Studies CBP This evidence will inform CBP sub-
(all genders) share their knowledge? programs,
Researcher Case
expertise and knowledge
To what extent are they maintaining the links Studies CBP will also share this evidence
and retain links with ACIAR,
they developed during their ACIAR program? within ACIAR and encourage research
other alumni, and their
projects to also support alumni to
wider networks
continue sharing their expertise.
29. Agriculture professionals For those alumni who are in management or Institutional Case CBP CBP will use insights as to these
(all genders) demonstrate, leadership positions, to what extent are they Studies broader attitudinal changes in
and are advocates for, advocating for and practicing inclusive communications and learning about
Tracer Studies
gender equality, diversity practices? the extent to which alumni are
becoming active advocates for
greater equality and inclusion, and
22
ACIAR Capacity Building MEL Framework
Updated March 2020
30. Agriculture professionals To what extent are alumni working with Tracer Studies CBP CBP will monitor this to identify ways
(all genders) work in ACIAR Australian partners? to increase these ongoing links and
projects and with Australian collaborations, if necessary.
How many alumni are working in ACIAR ACIAR Internal Survey
partners
projects are completion of their capacity
building program? (disaggregated)
How are ACIAR projects making use of ACIAR
alumni?
31. ACIAR projects are more How are ACIAR projects making use of ACIAR Internal Survey CBP CBP, through internal learning,
effective alumni to meet skills and expertise needs? reporting and communications, will
enable ACIAR staff to use these
To what extent are alumni playing key roles
relationships and this understanding
in project successes?
in the design and implementation of
its research projects and institutional
32. ACIAR has rich institutional How do ACIAR staff perceive the contribution Internal Survey CBP
relationships.
and scientific networks of CB activities to their professional and
organisational networks?
33. ACIAR has a greater How is involvement in the Capacity Building Internal Survey CBP
understanding of the Program (e.g. through hosting fellows within
Internal Reflection
challenges in agricultural research projects) contributing to broader
and Learning
research and partner ACIAR knowledge of challenges agricultural
institutions research?
34. ACIAR and Australia have a What examples indicate positive views of Institutional Case CBP Informed by this evidence CBP will
positive reputation in the ACIAR and Australia’s support within partner Studies work with Country Offices and DFAT
Country Offices
Indo-Pacific as an effective countries and institutions? to identify additional ways to
23
ACIAR Capacity Building MEL Framework
Updated March 2020
35. ACIAR continues to build a What examples do we see of ACIAR being Examples of positive CBP Informed by this evidence the
positive reputation within referred to in a positive light within Australia? media coverage, communications team in the Strategy
Australia policy and political and Capacity Building Program
references to ACIAR
24