Smits and Denis Implementation Science 2014, 9:28

http://www.implementationscience.com/content/9/1/28
Implementation
Science

RESEARCH Open Access

How research funding agencies support science

integration into policy and practice:
An international overview
Pernelle A Smits1,2* and Jean-Louis Denis1

Abstract
Background: Funding agencies constitute one essential pillar for policy makers, researchers and health service
delivery institutions. Such agencies are increasingly providing support for science implementation. In this paper, we
investigate health research funding agencies and how they support the integration of science into policy, and of
science into practice, and vice versa.
Methods: We selected six countries: Australia, The Netherlands, France, Canada, England and the United States. For
13 funding agencies, we compared their intentions to support, their actions related to science integration into
policy and practice, and the reported benefits of this integration. We did a qualitative content analysis of the
reports and information provided on the funding agencies’ websites.
Results: Most funding agencies emphasized the importance of science integration into policy and practice in their
strategic orientation, and stated how this integration was structured. Their funding activities were embedded in the
push, pull, or linkage/exchange knowledge transfer model. However, few program funding efforts were based on
all three models. The agencies reported more often on the benefits of integration on practice, rather than on
policy. External programs that were funded largely covered science integration into policy and practice at the end
of grant stage, while overlooking the initial stages. Finally, external funding actions were more prominent than
internally initiated bridging activities and training activities on such integration.
Conclusions: This paper contributes to research on science implementation because it goes beyond the two
community model of researchers versus end users, to include funding agencies. Users of knowledge may be end
users in health organizations like hospitals; civil servants assigned to decision making positions within funding
agencies; civil servants outside of the Ministry of Health, such as the Ministry of the Environment; politicians
deciding on health-related legislation; or even university researchers whose work builds on previous research. This
heterogeneous sample of users may require different user-specific mechanisms for research initiation, development
and dissemination. This paper builds the foundation for further discussion on science implementation from the
perspective of funding agencies in the health field. In general, case studies can help in identifying best practices for
evidence-informed decision making.
Keywords: Knowledge management system, Knowledge funding agencies, Knowledge use/utilization, International,
Science, Policy, Funding agency

* Correspondence: [email protected]
1
ENAP (École Nationale d’Administration Publique), 4750 Henri Julien,
Montréal, Québec H2T 3E5, Canada
2
Université de Montréal, C.P. 6128, succursale Centre-ville, Montréal,
Québec H3C 3J7, Canada

© 2014 Smits and Denis; licensee BioMed Central Ltd. This is an Open Access article distributed under the terms of the
Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use,
distribution, and reproduction in any medium, provided the original work is properly credited.
Smits and Denis Implementation Science 2014, 9:28 Page 2 of 12
http://www.implementationscience.com/content/9/1/28

Background production of scientific knowledge [15],’ funding agencies

A key challenge facing health services research and sci- have been analyzed [12,16,17].
ence policy is understanding and optimizing the integra- How science policies for SIPP are ingrained in the re-
tion of science into both policy and the practices of search agenda at the national level [18-20], how they are
professionals. This paper investigates how health re- adjusted to the educational sphere [21], how they are
search funding agencies support the integration of sci- formatted under external pressures [22], and how they
ence into policy and practice. are included in citizen and democratic procedures are all
First, there is confusion and misunderstanding about important issues. The dialogue around, and actions
terms such as knowledge transfer, knowledge translation, aimed at, producing stronger investigative SIPP research
implementation and research utilization, and even more highlights two elements: the need to consider other key
so, about such concepts as ‘moving knowledge into action’ players beyond just scientists and policy makers, and the
[1]. This has led to calls for an improved understanding of benefits of understanding the influence of scrutinizing
the uses of research [2]. Only a few fields of research— institutions on the role of science in policy making [23].
notably health service research [3-9] and knowledge trans- This paper identifies funding agencies as a key player and
fer research [10,11] —have investigated science integration contributor to SIPP. This premise means we need to study
into policy and practice (SIPP). In fact, health services re- funding agencies more closely. Not only do we need to see
search, especially research concerned with the use and im- them as a key player, as already suggested by Sutherland
plementation of research results in policy, has become so [23], we need to study their role as a central element, rather
intertwined with the field of knowledge transfer that calls than just one variable in the overall dynamics of SIPP.
for efforts have been made to compare and clarify terms This paper is innovative in several ways: it focuses on
used in the two fields. funding agencies as a key player in SIPP rather than just
Knowledge transfer spans a number of different fields, a component of the outside environment as previous au-
including communication with the public, communica- thors have done [12,16,17]; it compares funding agencies
tion with media, and collaborative approaches to science. on SIPP and examines the policy and practice achieve-
In this paper, the main focus is the direct integration of ments of program funding; it expands the traditional
science into decision making, both in policy and prac- conceptualization that knowledge transfer is centered
tice. We do not examine the indirect influences of sci- around practitioners, policy makers, and academics; it
ence on policy making, such as lobbying activities and adds to the growing trend for research focused on policy
public opinion. Rather, we focus on evidence-informed integration in the broader framework of health service
decision making aimed at modifying policy and practice, research; and it focuses on ‘the external social structures
such as SIPP and methods prior to SIPP in the process and processes that influence individual knowing and the
[12]. The line separating knowledge transfer and know- production of collective knowledge’ [24].
ledge translation is unclear, with some authors advocat- In Western countries, the agendas of funding agencies
ing that concepts like technology transfer, continuing prioritize national research and orchestrate research
education and commercialization not be included under funds to answer current and emerging health challenges
these terms [13]. We will not enter into this debate here; that have the potential to improve the health of their
rather, we will focus on the importance of research- populations. Funding agencies constitute operational pil-
based knowledge in the decision-making process at the lars for policy makers, researchers, and service delivery
level of both policy and practice. institutions like hospitals and health departments. This
In the 1970s, concern over knowledge-based decision paper investigates the role of funding agencies in SIPP.
making increased among researchers. Research utilization We provide an overview of the knowledge translation
[14] encompasses conceptual use, instrumental use and mechanisms of funding agencies as it relates to the
strategic purposeful use. Organizations—funding agencies health sector. There is a need to identify and discuss the
in particular—have been interested in licensing, acquiring, challenges to integrating research into policy and into
and commercializing biomedical discoveries for years. Re- practice in a timely manner, and to enhance the integra-
cently, stakeholders have championed the integration of tion of research findings into policy—at the macro, local
health-related sciences, human sciences, and biomedical and in practices at the micro levels.
sciences into policy and practice. Dedicated funding, the To date, a few empirical studies have provided details
identification of strategic objectives, and the implementa- about SIPP funding: most notably, a study that com-
tion of internal activities within funding agencies all foster pared European health services research at the national
SIPP. In the literature on science policy, or how the pol- level [5], and another that compared health research
icies of national governments, supranational organiza- funding at the agency level [25]. Both studies relied on a
tions, public science funding organizations, and large small number of informants. In the first study, 34 coun-
public research organizations aimed at influencing the tries were surveyed in 2007 to 2011, while in the second,
Smits and Denis Implementation Science 2014, 9:28 Page 3 of 12
http://www.implementationscience.com/content/9/1/28

members from 33 agencies were interviewed in 2003 Centre National de la Recherche Scientifique (CNRS)
and 2004. Both studies presented results from an in- Agence Nationale de la Recherche (ANR)
depth analysis of official funding agency documents. The
European comparisons mainly looked at both public and Canada
private funds, policy making, and the activities being Canadian Institutes of Health Research (CIHR)
promoted. In our study, we examine these aspects, but Natural Sciences and Engineering Research Council
also add the documented benefits of funding, both in (NSERC)
and outside of the policy domain. We include ‘internal’ Social Sciences and Humanities Research Council of
SIPP orientation strategies by funding agencies, along Canada (SSHRC)
with agency activities that support the innovation cycle
for beneficiaries of funds. We present a comparative, England
international analysis of the funding agencies’ require- National Institute for Health Research (NIHR)
ments, funding strategies, and the types of activities they
fund. Therefore, this paper continues the international USA
focus, while also examining the role that funding agen- Food and Drug Administration (FDA)
cies play in SIPP. It also contributes to the literature on Environmental Protection Agency (EPA)
SIPP by emphasizing health research funding agencies National Institutes of Health (NIH)
and inward strategic positioning, all while serving to
bring the current literature up to date. Data sources
The overall objective of this paper is to determine We selected published reports and information provided
what are the intentions, actions, and benefits of the inte- on the websites of the selected funding agencies (see
gration of science into policy and practice in health- Additional file 1). We extracted information from stra-
related funding agencies throughout the world. Our tegic plans, mission statements, organizational charts,
comparisons will help answer questions like: What kind reports on available funding for research and internal
of visibility do funding agencies give to SIPP? How do productions, success reports, and descriptions of agency
funding agencies promote SIPP? What impacts do fund- activities. Given that the interest in SIPP is fairly recent,
ing agencies have on practice and policy? we restricted our search of key resources to the period
of 2005 to 2011.
Methods The empirical secondary data that we gathered from
Selection of funding agencies these sources was limited: these data do not necessarily re-
We selected funding agencies from six countries: Australia, flect the most recent and emerging trends in the policies
The Netherlands, France, Canada, England, and the and decisions being made by funding agencies. To make up
Unites States. The agency had to be national or supra- for this limitation, we searched for the latest available and
national in scope, fund health-related research, and a accessible strategic plans and organizational documents.
major provider of research funds. We questioned pro- Such an approach may have introduced a positive bias and
fessors, researchers, members of funding agencies, lead to a more selective discourse disproportionately cen-
managers and health service providers who were mobi- tered around ongoing activities and the results obtained for
lized to identify the characteristics of the major provider these particular activities. However, we believe that this bias
of research funds category, identifying the ‘contrast or is less serious because we are interested in achievements ra-
continuum on knowledge translation engagement’ [25]. ther than failures and barriers. We included reports on the
Granting councils are: impacts of funding on policy or practices when the data
were detailed and relevant to the grant received. While
Australia source data analysis may not have allowed us to compile in-
National Health and Medical Research Council formation on all of the internal dynamics of funding, such
(NHMRC) dynamics were not our main point of interest.
Australian Research Council (ARC) Using secondary data has the advantages of providing
a level of depth similar from agency to agency, and
The Netherlands sometimes these secondary data have similar categories,
The Netherlands Organization for Health Research and thereby making the comparisons more accurate and
Development (ZonMW) straightforward. The reports and information published
on websites are produced after a review process and ap-
France proval by the department responsible for editing and
Institut National de la Santé et de la Recherche publishing official data, thereby ensuring a consistent,
Médicale (INSERM) shared view on topics of interest and, ultimately a
Smits and Denis Implementation Science 2014, 9:28 Page 4 of 12
http://www.implementationscience.com/content/9/1/28

comprehensive data set. It would have been difficult to Table 1 Summary of variables
obtain such a large and diverse dataset any other way: a Variable Dimensions
single representative from one institution simply could Intention Organizational visibility
not have provided such variety in content. We believe Purpose of SIPP
reports and official website information are a useful
Action Type of external actions
dataset for studying SIPP in funding agencies at this
point in the development of research about science inte- Type of internal actions
gration into policy and health. Steps of innovation
Note that our data collection relied on information Benefits Type of influences
available on the agencies’ websites. The uniformity of Specific influences on policy
the data published by a particular agency cannot be Specific influences on practice
guaranteed by our methodology. There is reason to be-
Specific influences on service delivery
lieve the selected agencies publish data that are not so
Policy refers to orientations taken by funding agencies on knowledge transfer.
different from one another. Indeed, the websites reveal Practice refers to the operational translation of programs and activities.
each agency’s formal representation of its own activ-
ities. Moreover, the information published by agencies innovation process (synthesis, initiation,
on their websites is directed at the public, researchers, development, end of grant, science, training).
policy makers, and practitioners. It is their public 3. Benefits: Funding agencies collect evidence of the
portrait. empirical benefits and return on investment for their
Our analysis of the websites generated secondary, ter- funding activities. Benefits are the consequences—
tiary, and endless snowball references as a result of navi- often positive outputs and outcomes—that
gating the agencies’ websites plus those of linked contribute to, or are generated by, specifically
agencies and ministries. We pulled documents that were targeted SIPP activities. We examined the following
directly posted on the agencies’ own websites. Relevant dimensions of benefits: type of influences (policy,
information was either easily accessible via the home- practice, service), specific influences on policy (local,
page, or it was less available, being stored in archives or national, international), specific influences on
identifiable by the title on webpages. We browsed the practice (improvement, new, modification of
whole website, including archives, paying particular at- regulations), and specific influences on service
tention to webpages related to strategy and funding. Of delivery (financial, human, material).
course, a key informant in each agency could have vali-
dated the documents we retrieved, but we sought only Results
to examine documentation. The agencies used a variety of terms and concepts when
talking about SIPP in their strategic and mission state-
Identification of variables ments: testing knowledge, applying knowledge (applica-
We investigated three main variables of SIPP and their tion), transfer, translate research/knowledge, valorization,
corresponding dimensions (Table 1): knowledge management, put to real-world use, integrate
science and technology, science coordination with prior-
1. Intentions: Health research funding agencies set ities, informed policy advice, use of knowledge/utilization
health as a priority and then devise ways to support of science, strategic management of licenses, innovation,
and foster the integration of science into policy and mobilizing knowledge, commercialization, evidence-based
practice. We examined the following dimensions of approach, and consider results in the decision making
intentions: organizational visibility of SIPP (absent, process.
external, internal) and the purpose of SIPP
(management, network, capacity building). Analysis of intentions
2. Actions: Health research funding agencies organize Organizational visibility
both internal and external SIPP-related activities. The importance and readiness of SIPP for funding agencies
Actions are any specific activities by funding is partially reflected in their organizational charts, mission
agencies to ensure the materialization of science into statements, and strategic objectives. The organizational
policy and practice, and to encourage evidence- charts sometimes attribute the responsibility of SIPP to dis-
based or evidence-informed policy or practices. We tinct directorates, departments and/or branches (Table 2).
examined the following dimensions of actions: type For example, SSHRC has a separate division for knowledge
of external actions (push, pull, linkage and exchange mobilization and program integration, unlike ARC, which
(L&E) funding), types of internal actions (reinforcing does not have any specific divisions. Most of the funding
capability, process, codification), and steps in the agencies manage their SIPP mission from within; usually it
Smits and Denis Implementation Science 2014, 9:28 Page 5 of 12
http://www.implementationscience.com/content/9/1/28

Table 2 Organizational visibility of SIPP in funding agencies

Visibility Funding agencies
Absent ARC. no specific directions
External INSERM. INSERM Transfert SA a private entity from INSERM
Internal CNRS Direction of innovation and relationships with companies (translation)
ANR. Partnership and competitiveness; investment in the future (translation)
EPA Office of science policy
NIH Office of science policy analysis
CIHR Knowledge translation branch; partnerships and citizen engagement branch; communications and public outreach branch
SSHRC Knowledge mobilization and program integration division
NHMRC Research investment direction/Prog Grants, Knowledge Translation & Capacity Building
NSERC Research partnerships program directorate; research grants and scholarship directorate
For ZonMw and NIHR, we found no data on organizational structure for SIPP. Please note that NIHR does commission research on SIPP through the NHS and the
Health Services and Delivery Research Programme [35].

is under the responsibility of one of their directorates Some of the funding agencies had a broader scope and
or branches. For example, SSHRC has a knowledge emphasized certain processes involved in SIPP. They
mobilization and program integration division, while favored a flow of information through contacts and ex-
NHMRC has research investment direction a /Know- changes between researchers and policy makers. For ex-
ledge Translation & Capacity Building. In France, ample, CIHR referred to knowledge translation and
INSERM manages some of its SIPP objectives through insisted on the importance of dynamic and iterative ex-
a private entity: INSERM Transfer SA. This group is in changes between the two communities:
charge of putting highly promising biomedical innova-
tions sponsored by INSERM funds into action; it diag- ‘Knowledge translation (KT) is defined as a dynamic
noses, accompanies, helps manage, protects, and and iterative process […] to improve the health of
supports innovations on the way to commercialization. Canadians, provide more effective health services and
One could say that INSERM is involved in translational products and strengthen the health care system. This
research rather than knowledge transfer per se, al- process takes place within a complex system of
though there needs to be some adoption of knowledge interactions between researchers and knowledge users
with partner companies and so this activity qualifies as which may vary in intensity, complexity and level of
knowledge transfer in our analysis. engagement depending on the nature of the research
and the findings, as well as the needs of the particular
Purpose of SIPP knowledge user’.
Strategic support for SIPP was embedded in various
strategies devoted to management, exchange, and/or Organizational chart: Knowledge translation branch;
capacity building. SIPP could be narrowed down to man- partnerships and citizen engagement branch;
agement objectives, in which knowledge informs the de- communications and public outreach branch
velopment or improvement of new practices and getting Source: CIHR. DPR 2009–2010.
a product out on the market. In such cases, the funding
agencies put an operational emphasis on SIPP. For ex- SIPP may build on the influence it has on policy and
ample, INSERM placed an emphasis on managing the policy makers to increase its influential capacity in the
transfer of technology: legislative branch. In Australia, both funding agencies
we studied sought to contribute to individual and
‘Its principal mission is to coordinate the valorization organizational learning: NHMRC by translating know-
of medical innovations from research laboratories’. ledge to a broad audience in order to improve policy
[our translation] and practice, and ARC by playing an advisory role in
policy making.
Organizational chart: INSERM Transfert SA a private
entity from INSERM ‘Policy: To provide informed high quality policy ad-
Source: translation. INSERM Transfert mission. vice to Government—through participation in policy
Website. forums and Government reviews, informed stakeholder
Smits and Denis Implementation Science 2014, 9:28 Page 6 of 12
http://www.implementationscience.com/content/9/1/28

consultations, evaluation and ongoing monitoring of and methods are available that could potentially
performance’. improve care for ethnic minorities in the
Netherlands. The aim is to spread this knowledge
Organizational chart: No specific directions. and these methods, and ensure they become part
Source: ARC. 2010. Strategic Plan 2010–11 to of mainstream care’.
2012–13.
NHMRC’s urgent research program reflects a pull
To summarize, our comparison of the intentions of model driven by policy needs:
funding agencies shows that:
‘Research that must be undertaken rapidly in response
1. Most funding agencies emphasized SIPP through to a threat to public health. The threat may be
specific organizational structures; generalised, or specific to a particular group of
2. Science was integrated into policy and practice individuals, and may be identified as either a current
through various operational means and by direct major problem, a potential major problem or a
support: through improvements to the management problem that is expected to increase in the future.
of SIPP, encouraging the flow of information The main catalysts for urgent research will be the fact
between actors in SIPP, and improving learning that a disease or illness, or its variant(s), is previously
related to knowledge; unknown or unidentified, and has a high morbidity
3. The internal structure of the funding agencies, and/or mortality rate, thus garners media coverage
combined with relevant funds, increased the and public and/or governmental concern. This
likelihood of SIPP. definition covers the range of possibilities at the
cellular level (the identification of the disease or
Analysis of actions illness, and its variants), to those at the public and
Types of external actions (pull, push and L&E models) population level, and then the level of risk to a nation.
Informed decision making was often conceptualized in For example, if a disease poses a probable or actual
terms of how to encourage mechanisms that would threat to the national and/or local economy by
bring science into the decision-making arena—that is in hindering exports, tourism, agriculture, and so on.
a push fashion. The push-pull model of knowledge The SRDC has outlined a process for considering
translation is famous for distinguishing between mecha- requests for urgent research based on this decision’.
nisms driven by science (push) and those driven by the
demands of practitioners or policy makers (pull). These ANR’s competitive pole funding program belongs to
models consider two communities in interaction: the sci- the linkage and exchange model:
entific community of researchers and the practice com-
munity of decision-makers, politicians, local community ‘A competitive pole is constituted by companies,
members, et al. A third model advocates for consider- research laboratories and training centers in a
ation of interactions that take place along the innovation geographic area in order to favor cooperation and
cycle and make science useful [26]. Such L&E [10], or exchange’. [our translation]
deliberate models, rely on the co-construction of applied
knowledge [27] and the relevance of applied research to In the agencies studied, the creation and development
both users and researchers. During research develop- of research in collaboration with users (L&E model) took
ment, data collection and interpretation tasks can be the form of meetings to explore prospective subjects of
shared. Most of the agencies we examined funded pro- interest, on-site training opportunities for practitioners to
grams based on one or two of these models, either push, come to research settings, and inviting researchers to join
pull, or L&E. Few provided funding based on all three decision making settings. The NIHR in England provided
types of SIPP models, with the exception of NHMRC, financial support for partnerships with health service pro-
ZonMW, NSERC and NIHR (Table 3, line 1). viders through its Service Delivery and Organization
ZonMW’s Ethnic Minorities and Health Care Program (SDO), where research results could be presented to prac-
belongs to the push model driven by science: titioners and practitioners could present their needs.
AHSC Academic Health Science Centers have brought to-
‘The Ethnic Minorities and Health Care Programme gether research units, health care services, and education
aims to promote the implementation of knowledge and institutions, thus promoting cross-fertilization between re-
skills available in this area. It targets both providers and search and practice.
users of care. The programme focuses on improving Funding based on the push model was for the transfer of
somatic curative care, an area in which knowledge existing research to users via publications in peer-reviewed
Smits and Denis Implementation Science 2014, 9:28 Page 7 of 12
http://www.implementationscience.com/content/9/1/28

Table 3 Funding agencies program funds

Line 1. Program funds in a push/pull/linkage and exchange (L&E) model
Push Pull L&E 3 modes
% of funding 69 62 62 23
agencies
Line 2. Eligible expenses per innovation cycle steps
Synthesis Initiation Development End of grant Science Training All steps
% of funding 38 54 69 77 31 62 23
agencies
Line 3. Internal activities related to SIPP
Capabilities Process Codification All 3
% of funding 85 54 46 46
agencies
Line 4. Benefits of funded projects
Policy Practice Service delivery 3 types
Local National International Improvement New Modification of Financial Human Material Policy, practice and
of practices practice regulations service delivery
% of funding 15 8 8 31 31 8 8 8 23 8
agencies
15 46 23 8

journals or practitioner-oriented journals, or through face-to- knowledge (capability). Some activities were more dy-
face dissemination in forums and conferences. For example, namic and helpful in building organizational knowledge,
the EPA established a Clean Diesel Emerging Technologies such as creating knowledge brokers, forums, and commu-
Program to help companies purchase innovative technolo- nities of practice. They put actors into situations where
gies to control fleet-related diesel emissions. Technologies they could discuss and/or solve problems or become more
existed. The aim was to make innovative, targeted technolo- comfortable with specific topics or issues; in short, they
gies available and financially accessible to companies. helped build organizational knowledge understood as a
In funding based on the pull model, research may be process. Other activities favored the provision of opera-
influenced by decision makers, either by focusing on pri- tionalized information through guides and handbooks.
orities, emergencies, threats, or an urgent need for data. These helped the organization build knowledge understood
For example, an infectious outbreak requires rapid as a codification. Capability, process, and codification are
knowledge and treatment solution, which requires that ways to categorize knowledge [28], and an organization can
research priorities become oriented by decision makers. use any of these as a basis of its actions.
Programs that targeted science included the develop-
ment of influenza vaccines (the NIVAREC center funded Activities that reinforce capabilities
by ZonMw), the development of specific, applied know- Most funding agencies offered services that facilitated the
ledge (e.g., ZonMw’s Chronic Fatigue Syndrome Pro- use of evidence-based knowledge in decision making.
gram, which encompasses the science of chronic fatigue Some resources, like online portals, assisted in identifying
syndrome, treatment and rehabilitation methods), pro- and tracking funding opportunities (NHMRC). Three-day
grams based on priority areas such as Aboriginal re- courses or a center for posting reviews allowed researchers
search (SSHERC), and strategic awards (NHMRC). to learn and share experiences (NIHR) and diffuse success
The funding agencies that we studied all funded either stories, to share ideas and good practices with other grant
push, pull, or L&E research (Table 3, line 1). Many agen- applicants and practitioners (ARC, NSERC, CIHR and
cies funded initiatives based on at least two models INSERM). Publishing lists of licensed innovations created
(push and pull), but these projects represented less than opportunities for companies to carry out discovery research
one-third of all documented funding (all models). (INSERM). And classes, summer schools (CIHR), and lec-
ture sessions for researchers, policy makers, or parliament
Types of internal actions members (CIHR, AR) served to train actors for SIPP.
Internal activities carried out by the agencies themselves
help develop organizational knowledge and SIPP. The in- Activities that reinforce processes
ternal activities that we documented varied in nature. Our documentation shows that not only resources, but
Some activities reinforced capabilities, such as classes and also active mechanisms favored SIPP. These mechanisms
SIPP training sessions, or helped build organizational put the utilization of science in the decision-making
Smits and Denis Implementation Science 2014, 9:28 Page 8 of 12
http://www.implementationscience.com/content/9/1/28

process in a dynamic context that enabled researchers to be more frequently funded than knowledge synthesis
and decision makers to interact through networks of or SIPP training projects, the funding agencies did
trainees (CIHR), encouraged journalistic interest in science sometimes have internal SIPP-related activities that
discoveries through journalism awards (CIHR), enabled were not related to funding competitions.
companies to consult groups of research expertise We may draw the following conclusions about the
(INSERM), and allowed professionals to interact through funding agencies’ involvement in the innovation process:
communities of practices (NHMRC). EPA had policy agree-
ments that allowed for formal liaisoning and the communi- 1. The push, pull and L&E models of knowledge
cation of scientific data to decision makers and official transfer were each supported by a slight majority of
regional coordinators for research and development. funding agencies
2. Few of the funding agencies relied on all three
Activities that reinforce codification models, meaning that all of the complementary
Making formalized knowledge useable relies on inform- dynamics of the knowledge processes were not
ative and descriptive documents. Summaries of research always covered in their funding efforts;
results can be translated into useful tools for practitioners, 3. Externally funded programs largely covered SIPP at
like the targeted summary of guidelines by NHMRC. the end-of-grant stage of the innovation process,
Among informative documents, we also found knowledge promoting diffusion in reviews, conferences,
transfer handbooks (CIHR). To translate research findings, etcetera;
the funding agencies also developed more prescriptive 4. Internally developed SIPP activities mainly focused
tools, such as targeted summaries of guidelines (NHMRC) on providing technical support, such as lectures via
or specific guidelines (NIH, USA, EPA, USA), and even an online portal, and such activities were less
rules (EPA, USA), agreements, and policies (NIHR). frequent than those aimed at bridging and
In summary, we found that the agencies and ministries translating knowledge.
developed a variety of activities, especially capabilities, to
make knowledge useful and useable (Table 3, line 3).
These activities enriched the organizations and enabled Analysis of benefits
them to use the science and knowledge produced. Tracking health service research utilization is not an easy
task [3,29]. Here, we report on SIPP-related success stor-
Steps of innovation ies, providing examples, individual cases, and positive re-
Funding for eligible expenses through funded programs, sults in which funded activities have led to changes in the
which may be based on a push, pull, or L&E model, may use of resources for service delivery, changes in the pro-
cover a number of different steps in the innovation cycle duction of service delivery, or in policy making.
[12]. The first step in innovation is initiation, in which
connections between decision makers and researchers are
established and topics of interest fine-tuned. The second Types of benefits
step is development, in which the research is conducted The funding agencies mentioned some benefits for prac-
and then data collected and analyzed. Then there is syn- tice and policy of their SIPP-related support. We did not
thesis (end of grant), which involves the presentation of find any references to the impact of SIPP-related internal
research results to the public through conferences and activities. The granting agencies reported the conse-
written diffusion methods like reports. Sometimes reports quences of SIPP under headings like ‘impact spotlights,’
synthesize previous results in the field. Research efforts ‘impact stories,’ ‘outcomes,’ and ‘results.’ The benefits
may center on either generic research topics, priority discussed in published reports took two forms: direct re-
areas, or SIPP itself. An eligible applicant, such as a fellow- sults of the funded research, such as a decrease in the
ship candidate, institution or group of actors, may, for ex- level of cholesterol in the intervention group, and the
ample, benefit from classes on SIPP or even do a impact of the funded research beyond the immediate
temporary stay at a federal institution in order to get a dir- scientific community, such as an influence on decision
ect feel of how and what is done inside public administra- making in the form of informing guidelines used to de-
tion (CIHR). The goal of such training is to help the velop a new policy. NHMRC’s Evaluation and Outcomes
applicant translate research into polity and practices. Working Committee defines direct results as the know-
The most common steps that were funded in the ledge creation outcome of research [30].
agencies we studied were end of grant, research devel- For the purposes of this research, we are interested in
opment, and training in SIPP (Table 3, line 2). Only the indirect outcomes and influences of SIPP benefits.
two agencies funded all steps in the innovation cycle. These benefits encompass both macro and micro issues.
While development and end-of-grant projects appeared They emerge at the policy level, at the practice/
Smits and Denis Implementation Science 2014, 9:28 Page 9 of 12
http://www.implementationscience.com/content/9/1/28

production level, and at the level of service delivery to consistently gathered through funded projects nor re-
clients and the population (Table 3, line 4). ported in funding agencies like CNRS.
To summarize, our analysis of funding agencies re-
Benefits at the policy level vealed the following:
The modifications of policies by SIPP funding could have
local, national, or international effects (Table 3, line 4). 1. There is evidence of benefits at the policy, practice
When the modifications were local, they affected the prac- and service delivery levels;
tices of groups or organizations. Sometimes national pol- 2. There is evidence that CIHR publicly documents
icies were modified through adjustments to regional or SIPP benefits in a more consistent way than do the
federal policies. The third type of SIPP influence other funding agencies studied;
was international, whereby the policies of international 3. There is evidence that the benefits of SIPP on
institutions or non-governmental organizations made practice are more common than reported.
changes to their policies based on practices elsewhere in
the world.
Discussion and conclusions
Benefits at the practice level Conclusions
Some of the SIPP funds supported operational modifica- The goal of this paper was to address the following
tions aimed at improving processes and procedures. questions: what visibility do funding agencies give to
These mechanisms were sometimes implemented at the SIPP? How do funding agencies promote SIPP? What
country level (Table 3, line 4). When regulations chan- impacts do funding agencies have on practice and pol-
ged under the impetus of research data, the result could icy? It is difficult to draw a complete and accurate por-
be the launch of a brand new process or the alteration trait of SIPP due to the diversity of terms being used to
of existing ones. Alterations could be accomplished designate knowledge transfer and evidence-informed de-
through the deletion of outdated information that re- cision making for policy and practice. This no doubt re-
quires modifications, or though the addition of informa- flects the fact that this is still a field under construction.
tion to improve current practices. The modification or Our findings support those of Tetroe et al. [25], who
introduction of a new practice or regulation on practices found that funding agencies from a variety of countries
should, theoretically, lead to improved service delivery. defined ‘knowledge transfer’ in many different ways [25].
For engineering projects that lead to conclusive out- Their results were based on interviews with health re-
comes, technologies could be used by other industries, search funding agencies, ministries, non-governmental
thereby allowing for the creation of new products, diag- organizations, researchers, and on an analysis of the
nostic tools or production lines. funding and internal activities of the agencies.
Apart from the variety of models, definitions, and
Benefits at the service delivery level measures that agencies use, there is a need to clarify in-
Eventually, research outcomes served as a basis for the dividual and organizational brokering interventions [31];
adjustment of services delivered to the population—to reduce inconsistency in publication dissemination meas-
patients, to clients, and to other countries (Table 3, line urement and guidance [32]; and adapt health research
4). Service delivery requires human resources, material systems to policy makers and to stakeholders’ policy-
resources and financial investment. We documented ad- making systems [5,33]. We need to take into consider-
justment in all of these types of resources based on evi- ation issues related to researcher autonomy (knowledge,
dence gathered in research. career based on productivity, employment stability); pol-
We traced the reported success stories to a number of icy motivation (agendas, urgent needs, career based on
different fields: policy making, practice organization, and publicity, re-election); and the closeness of funding
service delivery (Table 3, line 4). A common impact of agencies to researchers and policy makers. Based on our
the funding agencies funds was the adjustment of prac- results, we would also add such factors as scarcity of ac-
tical rather than policy aspects, and science delivery. tivities or funds to cover SIPP, from the early stages all
The funding agencies, especially those focused on med- the way up to the final steps of innovation.
ical development and technology, often reported the In our study, the importance attributed to SIPP was
commercialization of research ideas—for example, the revealed through an analysis of how present it was in
development of vaccines and medical technologies. the structure of the funding agencies. Some of the agen-
While success stories related to SIPP were often re- cies defined knowledge translation in their mission state-
ported under sections pertaining to the immediate re- ment and corporate objectives, while others did not.
sults of research, the influence of research on service Some agencies created a specific branch, directorate or
delivery, practice, and policy making may not be department to address SIPP.
Smits and Denis Implementation Science 2014, 9:28 Page 10 of 12
http://www.implementationscience.com/content/9/1/28

The funds available to cover SIPP were embedded in grant, science of SIPP, training in SIPP), and the actual
push, pull, and linkage and exchange models. Few of the results obtained have yet to be investigated. Additionally,
funding agencies that we studied offered funding based on the success stories purporting to demonstrate the actual
all three of these models. Comparing, between these three integration of science into decision making seemed to
models, how the results of a funded project on a particular depict the consequences of research results on practice,
topic is translated into knowledge is worth investigating and less on service delivery or even policy. Thus, our
further. Indeed, using a combination of all three models analysis probably under represents the extent of internal
may lead to more significant research utilization than does activities related to SIPP and underestimates the results
favoring one model over another. of funding. However, a quality verification across web-
While funding agencies emphasized the importance of sites and some internal documents reporting on the ac-
SIPP, much work still needs to be done to ensure cover- tivities of two Canadian institutions, Health Canada and
age of all models of the knowledge transfer process and CIHR, yielded the same general results.
of all steps in the SIPP process. This lack is seen in our
analysis of the programs being funded and in their in- Contributions
ternal activities. The benefits obtained from funding This paper makes an important contribution to research
SIPP and from SIPP activities are difficult to measure on SIPP because it goes beyond the two community
and compare because data reporting and collection is model, which is based on the conceptualization that re-
generally not systematic, as others have also noted [5]. searchers produce knowledge and end users utilize
Practice integration is largely predominant over policy knowledge, to include a variety of actors other than just
integration, which is in line with Kastrinos’ point that producers and users of science. Other authors have
there is ‘a marked trend away from mission-oriented pointed to the importance of having researchers frame
policies and towards diffusion-oriented policies’ [19]. the policy program [7], and to the major roles played by
Funding agencies are becoming more responsive to sci- funding, research and policy actors. We focus on such
ence than they are to policy [34]. issues in a detailed way, examining how science is inte-
grated into policy and practice by an important actor in
this process—funding agencies.
Study limitations One needs to keep in mind that while in this paper we
While some of the funding agencies studied showed a have conceptualized producers as researchers from the
broad range of internal activities aimed at promoting scientific community, it is not always easy to make a dis-
SIPP, the published documentation of internal activities, tinction between producers and users of science. For ex-
either through websites or official documentation, was ample, some producers of scientific knowledge work
not consistent across data sources. What is needed is inside national departments and agencies. Similarly,
more comparable data, and the integration of unpub- users of knowledge for decision making can be found in
lished internal SIPP activities in development. This a host of different entities: health organizations like hos-
paper certainly does not provide an exhaustive list of the pitals; civil servants assigned to decision making posi-
resources or mechanisms available for SIPP. Internal ac- tions within funding agencies, civil servants outside the
tivities created within the agencies, or across them, were Ministry of Health, such as the Ministry of the Environ-
not always documented in reports and webpages access- ment; politicians deciding on health-related legislation;
ible to the public, while pilot activities under develop- even university researchers whose work builds on previ-
ment were not yet available. Nonetheless, we are ous research. This heterogeneous sample of users may
confident that we have covered the sections considered require a host of user-specific mechanisms for the initi-
important by the funding agencies, as such sections are ation, development and dissemination of research.
generally the most clearly identified sections. This paper sets the foundation for further discussion
The empirical secondary data that we gathered from our on SIPP in funding agencies in the healthcare domain.
sources are limited: they do not necessarily reflect the We have detailed the complex relationships that exist
most recent and up-to-date decisions of funding agencies. between knowledge use in the process of decision mak-
We also need to consider that the benefits of research are ing at both the policy and practice levels. The other
mainly self reported by the researchers themselves, and element—in essence, the flip side—worth investigating is
that these benefits may be the result of either funding for knowledge development. This aspect, the integration of
the execution of research or funding aimed at encouraging policy and practice into science, may be essential if we
the use of research by stakeholders. are to achieve a meaningful dialogue and reciprocal in-
Furthermore, the relationship between the models fluence between research and policy, and between re-
(push, pull, and L&E models), or the different steps that search and practice [7]. Indeed, the production of new
are funded (synthesis, initiation, development, end of and useable knowledge to improve policy and practice
Smits and Denis Implementation Science 2014, 9:28 Page 11 of 12
http://www.implementationscience.com/content/9/1/28

might find interesting lessons in the study of how policy consolidated framework for advancing implementation science.
and practice integrate into science. Insights for the inte- Implementation Science 2009, 4:50.
9. Klazinga N, Fischer C, ten Asbroek A: Health services research related to
gration of policy into science can be learned from previ- performance indicators and benchmarking in Europe. J Health Serv Res
ous studies on the Rothschild initiative in England Policy 2011, 16:38–47.
[16,17]. Researchers conducted a seven-year formative 10. Lomas J: Using ‘linkage and exchange’ to move research into policy at a
Canadian foundation. Health Affairs 2000, 19:236–240.
evaluation of follow-up data on this attempt by the 11. Denis J-L, Lomas J: Convergent evolution: the academic and policy roots
English Department of Health in the 1970s to influence of collaborative research. J Health Serv Res Policy 2003, 8:1–6.
and set the agenda for healthcare research. Recent edito- 12. Graham I, Tetroe J: Getting Evidence into Policy and Practice: perspective
of a Health Research Funder. J Can Acad Child Adolesc Psychiatry 2009,
rials on collaborative research have highlighted the im- 18:46–50.
portance of maintaining an active relationship between 13. Straus SE, Tetroe J, Graham I: Defining knowledge translation. CMAJ 2009,
research and the policy and practice side [11]. 181:165–168.
14. Weiss C: The Many Meanings of Research Utilization. Public Administration
Case studies are very useful for identifying best practices Review 1979, 39:426–431.
in evidence-informed decision making. Of course, the in- 15. Leiden Universiy: 2013. http://www.leiden.edu/.
tegration of knowledge/science into practice and policy 16. Kogan M, Henkel M: Government and research: The Rothschid Experiment in a
government department. London: Heinemann; 1983.
does not depend on funding agencies alone [35]. Other or-
17. Kogan M, Henkel M, Hanney S: Government and research: thirty years of
ganizations like, for example, Canada’s Policy Research In- evolution. 2nd edition. Dordrecht: Springer; 2006.
stitute, inform policy, as do government entities like 18. Heinze T: How to sponsor ground-breaking research: a comparison of
deputy ministers. In countries like Germany, there are funding schemes. Sci Public Policy 2008, 35:302–318.
19. Kastrinos N: Policies for co-ordination in the European Research Area: a
mechanisms for such integration on advisory boards and view from the social sciences and humanities. Sci Public Policy 2010,
government task forces [5]. Future research could focus 37:297–310.
on whether establishing specific priority areas for funding 20. Stampfer M, Pichler R, Hofer R: The making of research funding in Austria:
transition politics and institutional development, 1945–2005. Sci Public
is more effective in informing decision making than is Policy 2010, 37:765–780.
funding based on pull, push, or L&E models. 21. Braun D: Organising the political coordination of knowledge and
innovation policies. Sci Public Policy 2008, 35:227–239.
22. Lepori B, van den Besselaar P, Dinges M, Pot B, Reale E, Slipers, ter S, Th ves
Additional file J, van der Meulen B: Comparing the evolution of national research
policies: what patterns of change? Sci Public Policy 2007, 34:372–388.
Additional file 1: List of documentation analyzed. 23. Sutherland WJ, Bellingan L, Bellingham JR, Blackstock JJ, Bloomfield RM,
Bravo M, Cadman VM, Cleevely DD, Clements A, Cohen AS, Cope DR,
Daemmrich AA, Devecchi C, Anadon LD, Denegri S, Doubleday R, Dusic NR,
Competing interests Evans RJ, Feng WY, Godfray HCJ, Harris P, Hartley SE, Hester AJ, Holmes J,
This study was funded in part by Health Canada. This funding agency did Hughes A, Hulme M, Irwin C, Jennings RC, Kass GS, Littlejohns P, et al:
not participate in the writing of this paper; nor did it read or approve it. A Collaboratively-Derived Science-Policy Research Agenda. PLoS ONE
2012, 7:e31824.
Authors’ contributions 24. Jacobson N: Social Epistemology. Science Communication 2007,
PS conceptualized and designed the study, and then collected and analyzed 29:116–127.
the data. Both PS and JLD interpreted the data. PS wrote the initial version 25. Tetroe JM, Graham ID, Foy R, Robinson N, Eccles MP, Wensing M, Durieux P,
of the paper, which was then revised by both PS and JLD. Both authors read Légaré F, Nielson CP, Adily A, Ward JE, Porter C, Shea B, Grimshaw JM:
and approved the final manuscript. Health research funding agencies’ support and promotion of knowledge
translation: an international study. Milbank Q 2008, 86(1):125–155.
Received: 15 April 2013 Accepted: 11 February 2014 26. Gholami J, Majdzadeh R, Nedjat S, Nedjat S, Maleki K, Ashoorkhani M,
Published: 24 February 2014 Yazdizadeh B: How should we assess knowledge translation in
research organizations; designing a knowledge translation self-
References assessment tool for research institutes (SATORI). Health Res Policy Syst
1. Graham ID, Logan J, Harrison MB, Straus SE, Tetroe J, Caswell W, Robinson 2011, 9:10.
N: Lost in knowledge translation: time for a map? J Contin Educ Health 27. Denis JL, Lehoux P, Champagne F: A knowledge utilization perspective on
Prof 2006, 26:13–24. fine-tuning dissemination and contextualisaing knowledge. In Using
2. Landry R, Amara N, Lamari M: Climbing the Ladder of Research Utilization. Knowledge and Evidence in Health Care: Multidisciplinary Perspectives. Edited
Science Communication 2001, 22:396–422. by Lemieux-charles L, Champagne F. Toronto, Canada: University of Toronto
3. Lavis JN, Ross SE, Hurley JE, Hohenadel JM, Stoddart GL, Woodward CA, Press; 2008:18–40.
Abelson J: Examining the Role of Health Services Research in Public 28. Denis J-L, Lehoux P: Organizational theories. In Knowledge
Policymaking. Milbank Q 2002, 80:125–154. Translation in Health Care. United Kingdom: John Wiley & Sons, Ltd;
4. Lavis JN, Robertson D, Woodside JM, McLeod CB, Abelson J: How Can 2013:308–319.
Research Organizations More Effectively Transfer Research Knowledge to 29. Lavis J, Ross S, McLeod C, Gildiner A: Measuring the impact of health
Decision Makers? Milbank Q 2003, 81:221–248. research. J Health Serv Res & Policy 2003, 8:165–170.
5. Ettelt S, Mays N: Health services research in Europe and its use for 30. Consan Consulting: Analysis of NHMRC funded research in 1992 and
informing policy. J Health Serv Res & Policy 2011, 16:48–60. 1997. Final report to EOWC and NHMRC. In Book Analysis of NHMRC
6. Gluckman P: Towards better use of evidence in policy formation : a discussion funded research in 1992 and 1997. Final report to EOWC and NHMRC. 2005.
paper. Office of the Prime Minister. Auckland, New Zealand: Science advisory 31. Ward V, House A, Hamer S: Developing a framework for transferring
committee; 2011:17. knowledge into action: a thematic analysis of the literature. J Health Serv
7. Gold M: Pathways to the Use of Health Services Research in Policy. Res & Policy 2009, 14:156–164.
Health Serv Res 2009, 44:1111–1136. 32. Wilson P, Petticrew M, Calnan M, Nazareth I: Does dissemination extend
8. Damschroder L, Aron D, Keith R, Kirsh S, Alexander J, Lowery J: Fostering beyond publication: a survey of a cross section of public funded
implementation of health services research findings into practice: a research in the UK. Implementation Science 2010, 5:61.
Smits and Denis Implementation Science 2014, 9:28 Page 12 of 12
http://www.implementationscience.com/content/9/1/28

33. Hanney S, Kuruvilla S, Soper B, Mays N: Who needs what from a national
health research system: lessons from reforms to the English Department
of Health’s R&D system. Health Res Policy Syst 2010, 8:11.
34. Slipersæter S, Lepori B, Dinges M: Between policy and science: Research
councils’ responsiveness in Austria, Norway and Switzerland. Sci Public
Policy 2007, 34:401–415.
35. Quevauviller P, Balabanis P, Fragakis C, Weydert M, Oliver M, Kaschl A,
Arnold G, Kroll A, Galbiati L, Zaldivar JM, Bidoglio G: Science-policy
integration needs in support of the implementation of the EU Water
Framework Directive. Environ Sci Policy 2005, 8:203–211.

doi:10.1186/1748-5908-9-28
Cite this article as: Smits and Denis: How research funding agencies
support science integration into policy and practice: An international
overview. Implementation Science 2014 9:28.

Submit your next manuscript to BioMed Central

and take full advantage of:

• Convenient online submission

• Thorough peer review
• No space constraints or color figure charges
• Immediate publication on acceptance
• Inclusion in PubMed, CAS, Scopus and Google Scholar
• Research which is freely available for redistribution

Submit your manuscript at

www.biomedcentral.com/submit