How to Conduct a Systematic


Review
 Introduction

 A systematic review is a type of literature review
that answers a specific research question
 A systematic review involves explicit methods to:
 Perform a comprehensive literature search
 Complete a critical appraisal of the individual
studies gathered
 Combine the valid studies using appropriate
statistical techniques
 Systematic reviews are considered one of the
best sources of evidence on a given topic
 Performing a Systematic
Review:

 Systematic reviews involve multiple steps and
normally take six to twelve months to
complete.
 Systematic reviews are performed by a small
team of individuals with various skills,
including searching experts, statistical experts
and subject experts.
 Key characteristics of a
systematic review:

 A clearly stated set of objectives with pre-defined
inclusion and exclusion criteria
 An explicit, reproducible methodology
 A systematic search that attempts to identify all
studies that would the eligibility criteria
 An assessment of the validity of the included
studies and their findings (i.e. assessing whether a
study may be biased)
 A systematic presentation and synthesis of the
characteristics and findings of the included studies
Steps to a Systematic Review

1. Formulate a question
2. Develop protocol
3. Conduct search
4. Select studies and assess study quality
5. Extract data and analyze/summarize and
synthesize relevant studies
6. Interpret results
Step 1: Formulate a Question

 What problem are you trying to address by
conducting the review? The research problem
should be a structured and unambiguous
question. The question should not be changed
once the review process has begun, as the
review protocol needs to be formed around
the question
 Con…

 The PICO Model
 The PICO model is a useful framework to break down the
research question for a systematic review. A good, answerable
question should follow the PICO model.
 Population – the group you are trying to study: the test
subjects or patients are the population
 Intervention – the treatment, exposure or experimental factor
to be studied
 Comparison – the alternative to compare to the intervention
 Outcome – is what the study hopes to measure or achieve
 Example question: Does cognitive behavioural therapy
prevent depression in individuals who have had a stroke?
 Population – post-stroke patients
 Intervention – cognitive behavioural therapy
 Comparison – no treatment
 Con…

 Alternate Models
 There are many alternate models that can be
used in place of the PICO model:
 Alternate models can work for non-clinical,
qualitative or social sciences research questions.
 The best model depends on personal preference
and the specific question you are trying to
answer.
 Two examples of alternate models are the
ECLIPSE and the SPICE models.


ECLIPSE Model
 Expectations
 Client Group
 Location
 Impact
 Professionals Involved
 Service
 E (stands for nothing)

 SPICE Model:
 Setting
 Perspective
 Intervention
 Comparison
 Evaluation
 Con…

 Step 2: Develop Protocol
 The protocol is extremely important in systematic
reviews. The protocol specifies the methods to be used
in the review with the aim of minimizing bias.
Transparency is key to a good systematic review, so the
protocol needs to be clearly stated.
 The protocol needs to include the following elements:
1. Inclusion/exclusion criteria
2. Searching strategy
3. Study selection criteria and process
4. Data extraction process
5. Assessment of study quality process
6. Data synthesis method(s)
7. Result dissemination plans


Inclusion/Exclusion Criteria
 The first part of the systematic review protocol is clearly defining the
inclusion/exclusion criteria. This is important for submitting to ethics
boards and funding bodies. It also helps readers understand the process
undertaken and evaluate the quality of your review
 Explicitly decide what groups to include in the systematic review.
 Decide how you want to limit your criteria before you start your literature
search and include this in the protocol.
 Inclusion Criteria: Types of Studies
 A systematic review should be based on the best quality evidence available
(i.e. the highest level of evidence)
 When possible, use controlled-trials. However, depending on your topic, this
might not be possible
 It may be beneficial to perform a scoping review prior to starting the
systematic review process to know what quality of studies are available.
Include this in your protocol
scoping review vs systematic
review

 
 Inclusion Criteria: Population
 What population do you want to include in your
systematic review?
 Decide what population you want to examine
before you run your searches
 Inclusion criteria may include: age, gender,
nationality
 Example: If you’re studying depression, it can
present differently in children and adults
 
 Search Strategy
 Outline the search strategy that will be used in
the systematic review:
 Terms – which search terms are you going to
use?
 Methods – what methods will you use? Are you
using a searching expert such as a librarian?
 Sources – what databases and sources will you
use?
 
 Study Selection Criteria and Process
 Determine how studies will be selected for
inclusion
 How many people are reviewing the studies?
What are their qualifications?
 
 Data Extraction
 How will data be extracted from the selected
studies?
 Who is performing this? What tools are they
using?
 Assessment of Study Quality Process
 Outline how the quality of included studies will
be assessed
 What tools or criteria are being used?
 
 Data Synthesis Method(s)
 How the results are being synthesized?
 The exact methods will depend on the data available
in the included studies
 Include the general plans, such as whether or not a
meta-analysis will be performed on the data.
 The protocol should also specify the outcomes of
interest and what effect measures will be used
 Result Dissemination Plans
 How will the findings will be disseminated to reach
the appropriate audience?
 Step 3: Conduct Search

 Conduct a search to find relevant articles for
the systematic review. The search strategy will
be outlined in the protocol. Once the protocol
is in place, the searching process can begin
 Use a structured search methodology when
conducting a search
 Con…


Steps to a Search:
 Define your topic/Ask a question (PICO)
 State the question you formulated in the first step of the systematic review
 Identify the individual concepts
 Pick out the key concepts in your question.
 The number of concepts depends on the nature of your question.
 Look at your PICO concepts for a place to start
 Search each concept separately
 For each concept, consider synonyms, alternate spellings and related terms
 Separate these with an OR in your search
 Do a different search for each concept
 Combine concept sets
 After searching separately, combine your concept searches with AND
 Review results and modify search
 After searching review the results. If they are too general, too specific
or irrelevant try a modified search
 Add limits/filters if necessary

 
 Subject Headings vs Keywords
 To ensure you are conducting a comprehensive
literature search, search using a combination of
keywords and subject headings
 Some databases search using subject headings
and some search using keywords
 Adapt your search strategy to each database
Subject Headings vs
Keywords

 Con…


Search Strategy

It may be beneficial to work with a librarian to help you design
comprehensive search strategies across a variety of databases
 The search should include scholarly articles as well as grey literature,
such as reports published by government or non-profit agencies.
 Search Log
 Keep a search log as you work. Keep track of what was searched in
which database. This will help you re-run searches or update your
systematic review. This is also important in providing transparency to
the process
 Citation Management
 Collect ALL of the retrieved records from each search into a reference
manager/citation management tool
 Citation management software can:
 Keep track of articles
 Organize and categorize records
 Share articles with other members of the systematic review team
 One Important feature of this software is that it can automatically
detect duplicate articles, saving time and effort. De-duplicate the
 Step 4: Select Studies and
Assess Study Quality

 Once all results are gathered and de-duplicated,
begin screening and assessing the studies
 First, screen studies for relevance
 Second, assess the quality of the remaining studies
 Screening Results
 Step 1: Title/abstract screening
 Scan just the titles and abstracts to remove studies
that are clearly not related to the topic
 Step 2: Full-text screening
 Use the inclusion/exclusion criteria to screen
the full-text of studies
 Con…

 Assessing Results
 When assessing studies for inclusion, standardized
assessment tools help eliminate bias and make for a
better systematic review
 The exact items on the checklist will depend on the type
of studies you are including
 There are pre-made checklists you can use such as the
Joanna Briggs Institute Critical Appraisal Tools checklist
for a randomized controlled trial
 Have more than one person review all the collected
articles
 State in the protocol the number of reviewers and how
discrepancies in reviewer selections will be resolved
 Con…

 PRISMA Flowchart
 Use the PRISMA flowchart to keep track of the
resources collected or selected in each step
 You will likely have hundreds, if not thousands,
of articles to screen
 This will help when writing up the systematic
review
 Step 5: Extract Data and Analyse, Summarize and
Synthesize Relevant Studies
 
Once the included studies are finalized, use a data
extraction form or systematic review software to extract
all relevant data from each study
 When the data has been extracted, analyze then
synthesize the results
 Extract data
 Have more than one person extract the data and check
for mismatches
 Record data exactly as reported
 Analyse Data
 The analysis will depend on the type of data extracted
 Meta-analysis involves using of statistical methods to
summarize the results of the chosen studies
 Whenever possible, perform a meta-analysis of the
results. This will help improve the quality of the review
 Step 6: Interpret Results

 The last step is to interpret the results of the systematic
review and disseminate them
 This step involves writing up the systematic review
 Presenting Results
 Clearly present your findings
 Include detailed methodology based on your protocol (search
strategies used, selection criteria, etc.)
 This ensures that your review can be easily updated in the future
with new research findings
 Propose future steps
 Provide recommendations for practice and policy-making if
sufficient, high quality evidence exists
 Indicate future directions for research to fill existing gaps in
knowledge or to strengthen the body of evidence
 Characteristics of a Good
Systematic Review:

 Explicit
 Comprehensive
 Reproducible
 Systematic
 Bias Minimized
 Ask yourself if your systematic review is
explicit, comprehensive, reproducible,
systematic and if bias has been minimized. If
you can answer yes, you have a good
systematic review
Best Practices for a Systematic Review


 Adopt a Reporting Standard
 Identify all sources consulted
 Use a search log to keep track of searches
 Manage references by a citation management tool
 Report the qualifications of searchers
 Visual presentation (i.e. Study Identification
Flowchart)
 Remember: transparency in the process and
reproducibility of the searches make for a good
systematic review