p47-56w51_LEARNING ZONE 23/08/2010 10:21 Page 47

learning zone
CONTINUING PROFESSIONAL DEVELOPMENT
Page 58 Page 59 Page 60
Systematic review: 2 Read Amy Causey’s Guidelines on how to
multiple choice questionnaire practice profile on write a practice profile
total hip replacement

Learning how to undertake

a systematic review: part 2
NS557 Bettany-Saltikov J (2010) Learning how to undertake a systematic review: part 2.
Nursing Standard. 24, 51, 47-56. Date of acceptance: February 8 2010.

4Select and appraise studies for inclusion in

Summary the review.
This article, the second in a two-part series, continues to describe 4Extract relevant information from the
the stages involved in conducting a systematic review. The article research articles.
discusses how to select and appraise studies for inclusion in the
4Demonstrate a basic understanding of how
review, and how to extract, synthesise and present the findings.
to synthesise and present the findings.
A summary of the key points for writing the results and discussion
sections of a report of a systematic review is also presented. 4Understand how to structure the discussion
Author that results from the systematic review.

Josette Bettany-Saltikov, senior lecturer in research methods,

School of Health and Social Care, University of Teesside, Introduction
Middlesbrough. Email: [email protected] The first of these two articles discussed the
Keywords importance of writing the protocol for a
systematic review. It also explained how to
Research methodology, systematic review formulate a review question and set out the
These keywords are based on subject headings from the British background, objectives and inclusion and
Nursing Index. All articles are subject to external double-blind peer exclusion criteria for a review. This article
review and checked for plagiarism using automated software. For continues by illustrating how to formulate a
author and research article guidelines visit the Nursing Standard search strategy and critically appraise the results
home page at www.nursing-standard.co.uk. For related articles of a literature search. These processes should be
visit our online archive and search using the keywords. described in the review protocol before the actual
search is conducted in order to minimise bias.
The time outs in this article will assist in
completing the protocol.
Aims and intended learning outcomes
In conducting a successful unbiased
This article, the second in a two-part series, aims systematic review it is important to access
to provide an overview of the stages involved in as many relevant studies as possible. This
conducting a systematic review, focusing on may involve searching a wide variety of
selecting and appraising articles for inclusion and databases and internet search engines as well
the presentation of data and findings. It is assumed as hand searching, which involves ‘a manual
that readers have a basic understanding of page-by-page examination of the entire
research terminology and the skills necessary contents of a journal issue to identify all eligible
to critically appraise a review. After reading this reports of trials, whether they appear in articles,
article and completing the time out activities you abstracts, news columns, editorials, letters or
should be able to: other text’ (The Cochrane Collaboration 2009).
Inclusion and exclusion criteria must be
4Know how to undertake a comprehensive and established because these will determine the
systematic search strategy. focus and limits of the review.

NURSING STANDARD august 25 :: vol 24 no 51 :: 2010 47

p47-56w51_LEARNING ZONE 23/08/2010 10:21 Page 48

databases tend to publish or include articles

learning zone research methods originating primarily from certain countries.
The Medline database, for example, includes
approximately 20 million references,
approximately half (52%) of which originate
Systematic search strategy
from the United States (Bruce et al 2008).
The aim of the literature search is to generate A number of key actions should be
a comprehensive list of primary studies – considered when undertaking a search for
published and unpublished – which may be relevant articles:
suitable for answering the proposed research
4Read reference lists to identify source
question. This is a crucial part of the review,
ideas and concepts that highlight the design
because the validity (truthfulness) of the review
of studies.
results is directly related to the thoroughness
of the search and its ability to identify all the 4Conduct a hand search, which may help
relevant studies (Centre for Reviews and to avoid possible bias in ‘keyword’ search
Dissemination 2009). systems, for example Medline. Keyword
Conducting a comprehensive literature search search systems are an easy way to identify
will also help to identify current knowledge published studies. However, ‘information
with regard to relevant concepts and contexts technology and the processes associated
and what is known and unknown in a particular with indexing are not infallible. Studies may
field (Petticrew and Roberts 2006). A not be marked by study design correctly,
comprehensive search strategy underlies the which may mean they are missed in the
quality of the literature search, which in turn electronic searching process. Hand searching
underlies the quality of the findings for the for evidence of intervention effectiveness
systematic review (Alderson and Green 2002). has therefore become a recognized tool in
Any conclusions made following a review are the systematic review process’ (Armstrong
only as good as the range and quality of the et al 2005).
literature obtained.
4Read grey literature, for example conference
It is important to search widely and
proceedings and PhD theses. Smaller and
thoroughly because not all research is
unpublished studies may still be robust enough
published in journals. Further, not all research
to provide valuable information.
published in journals is indexed in major
databases and, consequently, may not be easily 4Contact the authors of key articles. They may
retrievable (Bruce et al 2008). Other reasons be able to provide access to some of their
for searching widely include the fact that important, but unpublished work.
there may be a long wait before publication.
Publication gaps after conference presentations There are a number of steps involved in
are common as it takes authors a considerable converting the review question into a search.
amount of time to write up their findings, The first of these is to refer back to the
submit them, have them peer-reviewed and keywords that will form the basis of the search.
then amended as necessary. Finding an abstract Timmins and McCabe (2005) stated: ‘The use of
or poster presented at a conference or published appropriate key words is the cornerstone of an
in conference proceedings may therefore be effective search.’ It is possible to conduct
important as it will provide some, although searches using index terms and free text
limited, information. searching. Index terms include those used by
Problems with searching include publication electronic databases, which may not match the
and language bias (Dickersin et al 1987). terms in the research question precisely,
Publication bias means that positive results tend for example the Mesh – Medical Subject
to be published in journals more frequently then Headings – database in Medline. To ensure
negative results (Bruce et al 2008). Language that a search is comprehensive and both
bias refers to the fact that positive results are sensitive and specific, free text searching, also
more likely to be published in English. Egger known as ‘natural language’, should be used
et al (1997) found that authors were more likely in addition to, or instead of, index term
to publish randomised controlled trials in an searching (Lahlafi 2007).
English language journal if the results were As discussed in part 1 of this series, the first
statistically significant. Researchers and step is to write out the research question and
students are also more likely to look at research identify its PICO (population, intervention,
in their own language. Bias may also relate comparative intervention, outcomes) or PEO
to the geographical coverage of journals and (population, exposure, outcomes) components.
databases. For example, some journals and The next step is to identify any synonyms

48 august 25 :: vol 24 no 51 :: 2010 NURSING STANDARD

p47-56w51_LEARNING ZONE 23/08/2010 10:21 Page 49

(words of a similar meaning) for all the

TABLE 1
components of the question.
Any search needs to be both sensitive and Research question using the PICO formulation
specific. Sensitivity, in this context, refers to P I C 0
a search that picks up all research articles that (Population) (Intervention) (Comparative (Outcomes)
are potentially relevant. Specificity refers to intervention)
a search that selects only those research articles Women Advocacy General practice Quality of Life
that are directly relevant. It is also important who have programmes or routine (measured by
to identify all the synonyms relating to the experienced treatment the SF-36 scale)
question and then to combine them using domestic violence
specific words called Boolean operators.
These are words used in searches to combine
different keywords or phrases. A list of common TABLE 2
operators includes: Identifying synonyms and keywords for the search strategy
4OR – finds citations containing either of the Population AND Intervention AND Comparative AND
specified keywords or phrases (sensitivity). intervention Outcomes
4AND – finds citations containing all of the 1. Domestic violence 11. Treatment 21. General 28. Women’s
specified keywords or phrases (specificity). practice quality of life
4NOT – excludes citations containing specified 2. Wife abuse 12. Group support 22. GP
keywords or phrases. 3. Partner abuse 13. Individual 23. Routine
support treatment
Time out 1 4. Battered women 14. Advocacy 24. Doctor
programme
Identify synonyms for the
research question you developed 5. Spouse abuse 15. Counselling 25. Physician
when reading the first of 6. Rape 16. Community 26. Surgery
these articles.
7. Sexual abuse 17. Therapy
8. Coercion 18. Support
Case study: identifying synonyms The various
steps involved in performing a systematic 9. Murder 19. Advocacy
review will be identified using the case study 10. Combine 20. Combine 27. Combine
involving Isabelle from the first of these two 1-9 using ‘OR’ 11-19 using 21-26 using
articles. Isabelle is a staff nurse who intends ‘OR’ ‘OR’
to conduct a systematic review. Her quantitative The last step is to combine steps 10+20+27+28 using the term ‘AND’
research question is: For women who have
experienced domestic violence, how effective creates a list under the heading of Population
are advocacy programmes as compared with with each synonym in a new row and then
routine general practice treatment for numbers them from 1-9 (there happen to be nine
improving women’s quality of life as measured in this case). The words are numbered in the
by the Short Form 36 Health Survey (SF-36) order (or steps) that they will be inserted
scale? She now needs to convert the question individually into the database or search engine.
into a search strategy. This is repeated for the Intervention, ‘I’ (steps
Isabelle needs to identify synonyms for all 11-19), Comparative intervention, ‘C’ (steps
the PICO components of her question. She is 21-26), and Outcomes, ‘O’ (step 28), columns.
having difficulty thinking of synonyms, so she The example in Tables 1 and 2 and the text below
decides to use a thesaurus. To help her in the is worked using the CINAHL (Cumulative Index
task she uses a template to combine all the to Nursing and Allied Health Literature)
related terms of her question to try and obtain database. The list shows the words Isabelle types
as many relevant articles as possible. The into CINAHL and the order and combinations
template also optimises the sensitivity and of how she types them in. Using the template in
specificity of her search. Table 2, she first types the words from 1-9
To explain this further, look again at individually into the database. Once she has done
Isabelle’s question divided into PICO this she needs to combine (line number 10 in
components (Table 1), copied here from the Table 2) the words using the ‘OR’ operator; in
first article. To start with the population CINAHL this means pressing the ‘Search with
column, ‘P’, Isabelle first needs to find OR’ button.
synonyms for ‘women who have experienced 1. Domestic violence.
domestic violence’. In Table 2, Isabelle first 2. Wife abuse.

NURSING STANDARD august 25 :: vol 24 no 51 :: 2010 49

p47-56w51_LEARNING ZONE 23/08/2010 10:21 Page 50

learning zone research methods Sources of information

The researcher needs to undertake a
3. Partner abuse. comprehensive literature search on the research
4. Battered women. question using all relevant databases and other
5. Spouse abuse. information. Sources of information fall into
6. Rape. four main categories:
7. Sexual abuse.
4Online databases.
8. Coercion.
9. Murder. 4Journal articles.
10. 1 or 2 or 3 or 4 or 5 or 6 or 7 or 8 or 9
4Grey literature.
(Isabelle combines terms using ‘OR’).
This is then repeated for the remaining 4Books.
columns: Intervention (steps 11-20),
Comparative intervention (21-27) and Major databases include general resources such
Outcomes columns (28). Once she has done as CINAHL, Medline and Allied and
this, all the PICO synonyms need to be Complementary Medicine (AMED) among
combined using the term ‘AND’, which means others. The Centre for Reviews and
she will need to combine steps 10, 20, 27 and Dissemination publishes a comprehensive list
28 using the ‘Search with AND’ button in the (www.york.ac.uk/inst/crd/revs.htm).
‘CINAHL’ database. Specialist databases cover particular medical
Combining the terms in this way, for a search specialties and include the Health Management
performed in August 2010, resulted in a list Information Consortium (HMIC)
of two articles using all search terms and 131 (http://bit.ly/9IghRY).
articles using all search terms except ‘women’s Most search facilities will direct the researcher
quality of life’. to articles from up to date peer-reviewed
journals. However, as many research papers,
Time out 2 PhD theses and conference abstracts and
proceedings remain unpublished, it is essential
With your own review question to search the grey literature (unpublished sources
in mind, translate all your of evidence). The internet may be useful, but
keywords and synonyms into be careful to check that the information comes
a search strategy like the one from a trustworthy source.
developed by Isabelle. Add this to Search the internet in a number of ways:
the protocol you began in article 1.
4General purpose search engines, such as
Google or Yahoo.
Limiting the search It is preferable to apply
limits at the final stage of the literature search. 4A general subject gateway, such as Intute
Limits can include restricting the search to (www.intute.ac.uk).
English language articles, human studies,
4A specialist subject gateway, for example
research articles and possibly specifying a date
a disease-specific website such as The
range. There should be a rationale for this; it
National Cancer Institute website
should not be arbitrary. An example might be
(www.cancer.gov/cancertopics/pdq/
that, if there had been a significant change in
cancerdatabase).
advocacy programmes since 1990, it would be
wise to limit the search strategy to articles written 4Conference papers and proceedings can be
after this date. Different limits are available in found in the ISI Proceedings: Science and
different databases. If limiting a search to English Technology Edition. This contains details of
language articles only, it is important to approximately 10,000 conferences per year:
acknowledge that a language bias has been http://wok.mimas.ac.uk (requires subscription).
introduced into the search.
4Abstracts of dissertations may be found at:
www.theses.com (requires subscription).
Time out 3 4Clinical and non-clinical experts in the field of
Undertake a preliminary study may be found at the National Research
literature search, locating two Register (https://portal.nihr.ac.uk/Pages/
or three articles that are relevant NRRArchive.aspx).
to your question, using a database
such as CINAHL. Books are useful for identifying ‘stable’ sources
of information and developing background

50 august 25 :: vol 24 no 51 :: 2010 NURSING STANDARD

p47-56w51_LEARNING ZONE 23/08/2010 10:21 Page 51

knowledge of a research problem. However, articles (Torgerson 2003). It consists of two parts.
information in books can be rather dated – by The first involves sifting through the titles
the time the book is written and published up to and abstracts of all the articles retrieved from the
three years may have gone by since the research search, screening them systematically and
was carried out. selecting those that meet the pre-determined
It is important to record and save searches, inclusion criteria. This will result in a selection
as well as the results of the searches in an of papers that will a) definitely be included, b) may
electronic format, so that all the necessary or may not be included, and c) will be rejected. The
information will be available and easily accessible papers that will definitely be included and those
when it comes to writing up the review. The that may or may not be included must be
search strategy, including the database, the title examined closely. This means obtaining full copies
of the article, the abstract, the website or of each article, reading it and making a decision
database, and the date should be logged (Table 3). on whether it meets the inclusion criteria (The
As much detail as possible should be recorded so Cochrane Collaboration 2009).
that a colleague could replicate the review. This At this stage it is useful to develop an
will be useful if the search needs to be carried out appropriate form to help select articles that meet
again at a later date. Discussion of the ‘hits’ the pre-determined criteria. It is also important to
obtained and the selection process used to identify test the form on a number of articles to ensure that
articles for closer study will provide an audit trail. it is appropriate. Figure 1 demonstrates what such
a form might look like. This process should be
Time out 4 carried out by at least two reviewers to increase
the validity of the results (Torgerson 2003,
Create an appropriate form such Petticrew and Roberts 2006). Figure 1 indicates
as the example in Figure 1 and use that abstract 1 should be included, as all the
it to select, from your preliminary criteria have been met. Abstract 2 could be
search in time out 3, an article excluded, as two of the criteria have not been met.
appropriate for your own research question.
FIGURE 1
Selecting research papers (example data)
Appraising and extracting information
Abstract number 1 2 3
The methods of a systematic review involve
three phases: Population
Women? √ √ √
4Phase 1: selecting studies for inclusion or Over 18?
exclusion in the review. This involves screening
Intervention
the title and abstract of each article found in the
Advocacy √ √ ?
literature search, reading the full text of each
article that might usefully be included Comparative
in the analysis. intervention
Standard care √ X ?
4Phase 2: appraising the quality of articles.
Outcomes
4Phase 3: data extraction. Quality of life √ X √

Using the three phases approach should ensure the Type of study
appropriateness of the methods used can be easily Randomised
evaluated and duplicated. All three phases should controlled or controlled √ √ √
clinical trials
be discussed with a critical review panel and/or a
supervisor to ensure that the results obtained are Action Include Exclude Undecided –
as accurate and free from bias as possible. Include consult
Phase 1: selecting studies for inclusion or or exclude review
exclusion Phase 1 aims to filter out irrelevant panel

TABLE 3
An example of how to document the results of a search strategy
Database Dates Date Hits Full record/titles Notes
covered searched and abstracts
Medline 1990-2010 20/06/10 23 Include titles of all Give the search strategy a name,
articles that count for example ‘medline 1’, in case
as ‘hits’ the search needs to be repeated

NURSING STANDARD august 25 :: vol 24 no 51 :: 2010 51

p47-56w51_LEARNING ZONE 23/08/2010 10:21 Page 52

authenticity, rather than validity or reliability.

learning zone research methods However, appraising the quality of studies
is crucial. It allows the exploration of how
differences in quality might explain differences in
With reference to abstract 3, the researcher is the study results, and it guides the interpretation
undecided and should consult with her critical of findings and their value to practice.
review panel or supervisor. There are a number of practical issues to
consider when appraising a study (Centre for
Time out 5 Reviews and Dissemination 2009). These include
stating who will be assessing the quality of the
Select the quantitative or studies, how many reviewers will be involved,
qualitative critical appraisal what checklist or scale will be used for quality
form available on the McMaster assessment and how the reviewers will resolve
university site: http://bit.ly/bc7hlm. disagreements. Involvement of a critical colleague
Using the guidelines to help you, try panel or supervisor is important to ensure that all
to critique the article you chose in articles are critiqued appropriately.
time out 4. The quality of evidence and conclusions
generated by a systematic review depend on the
Phase 2: appraising the quality of articles quality of the primary studies that make up the
The term ‘assessing study quality’ is often used review. This quality assessment is one of the key
interchangeably with ‘assessing the internal features that sets a systematic review apart from
validity’ – that is, the extent to which a study is a literature review.
free from methodological biases’ (Petticrew and It is important to use appropriate checklists or
Roberts 2006) or ‘the degree to which the results scales for the type of study design to be evaluated.
of a study are likely to approximate the “truth”’ Box 1 contains a scale that can be used to evaluate
(The Cochrane Collaboration 2009). Jadad non-randomised studies as well as websites where
(1998) suggested that the following points critical appraisal forms for different study designs
should be considered when assessing the quality can be found.
of randomised controlled trials:
4Relevance of the research question. Time out 6
4Internal validity of the trial – the degree to Extract relevant data from the
which the trial design, conduct, analysis and results section of the research
presentation minimise bias. article used in time outs 4 and 5
and use this information to design
4External validity – the extent to which findings a data extraction form.
are generalisable.
4Appropriateness of the data analysis Phase 3: data extraction The data extraction
and presentation. phase is perhaps the most challenging aspect of
the methodology. Data extraction involves going
4Ethical implications.
back to the primary articles and highlighting the
In the context of systematic reviews, quality refers relevant information that will answer the research
to the methodological quality – the internal and question. This usually involves extracting data
external validity of quantitative studies.The relating to the PICO components. However, to
criteria for qualitative studies are different. standardise this process and improve the validity
These studies are often judged on the basis of of the results it is crucial to compile a data
extraction form. Again, as with the selection
form previously described, it is important to
BOX 1
pilot the form on one or two of the articles to
Resources for critical appraisal ensure it is useful and appropriate (The Cochrane
4Critical Appraisal Skills Programme Collaboration 2009). It is also usual for more
www.phru.nhs.uk/Pages/PHD/CASP.htm than one reviewer to extract data and then to
compare and discuss results.
4McMaster University Evidence-Based Practice Research Group In the case study example, Isabelle needs to look
critical appraisal forms
back at the PICO form she made when selecting
http://bit.ly/bc7hlm
her paper, discussed in the first of these two articles.
4The Newcastle-Ottawa Scale (NOS) for assessing the quality of She knows that all the articles included in the final
non-randomised studies in meta-analysis www.ohri.ca/programs/ selection are relevant to the research question
clinical_epidemiology/oxford.htm and have met the inclusion criteria. In the data
(Last accessed: July 6 2010)
extraction form, it is important that she extracts

52 august 25 :: vol 24 no 51 :: 2010 NURSING STANDARD

p47-56w51_LEARNING ZONE 23/08/2010 10:22 Page 53

all the relevant information to enable her to answer However, irrespective of the type of review
her question related to women’s quality of life. there will still need to be some form of summary
As well as collecting information on the population, or synthesis.
intervention and comparative intervention, Isabelle The results of the comprehensive search can
has to collect information on the outcomes. Box 2 be presented either textually or in a table. An
provides an example of what Isabelle’s data example of the latter appears in Table 4. It is
extraction form might look like. important that anyone reading the review can
ascertain how comprehensive, transparent and
Time out 7 replicable it is. When presenting results, it is usual
to include the databases searched with the dates
Complete the protocol you included in the search, the date of the search, the
began in the first of these articles, number of hits, the number of articles discarded
discussing your search, critical and the number of articles to be reviewed. Results
appraisal and data extraction strategy. presented in tables need to be explained fully.
The emphasis in this article is on how novice
researchers can conduct a systematic review.
Synthesising the evidence
‘The synthesis (at a minimum) is a summary BOX 2
of the current state of knowledge in relation to Example of a data extraction form
a particular review question’ (Popay et al 2006).
This is the stage of the process where one Date of data extraction: 20/07/10
attempts to find the answer to the review Reviewer: Isabelle Smith
question. In a quantitative review, if the results
Bibliographic details of study: Jones J (2008) The effect of advocacy
are similar – for example if the interventions, interventions compared to usual care on abused women’s quality of life.
designs and outcomes are all the same – then it Journal of Clinical Nursing. 10, 5, 345-352.*
may be possible to conduct a statistical
Purpose of study: To evaluate the effectiveness of a community advocacy
procedure, such as a meta-analysis, to combine
programme as compared with usual care on abused women’s quality of life.
the results.
Synthesis of evidence should address the Study design: Randomised controlled trial.
following: Population (sample): 60 women who were or had previously experienced
domestic violence were included in the study. The women were randomly
4Are the data sufficiently similar? allocated to either the intervention group (n=30) or the control group who
received usual care (n=30).
4Are there caveats that need to be
acknowledged? Intervention: Women attended an advocacy group once a week over 12
weeks. Group meetings provided support and help for women on all aspects
4Are there any particular trends or themes? relevant to domestic violence.
4Do the data seem to point in one direction Comparative intervention: The women in this group received usual care.
or several? Outcomes:
SF-36 Quality of Life Scale score
In some disciplines, such as nursing and the Pre-intervention advocacy group score: 30/50 (50 is the average rate for
social sciences, either the quantitative studies healthy individuals)
Post-intervention advocacy group score: 40/50
involved are significantly different or the
Pre-intervention usual care group score: 29/50
qualitative studies require different methods Post-intervention usual care group score: 30/50
of synthesis. Some reviews may also include
*Fictional article.
studies with different designs (mixed methods).

TABLE 4
An example of a way to present the results of a systematic search
Database Search Number of hits Number of articles Number of articles Number of articles to
with dates date retrieved from discarded because duplicated from be reviewed by title
the search of irrelevant titles another database and abstract
CINAHL 20/6/10 1,569 1,456 79 34
(2000-2010)
Medline 21/6/10 1,847 1,346 244 284
(1963-2010)
EMBASE 23/6/10 2,485 1,567 600 318
(1996-2010)

NURSING STANDARD august 25 :: vol 24 no 51 :: 2010 53

p47-56w51_LEARNING ZONE 23/08/2010 10:22 Page 54

recommendations for presenting results

learning zone research methods (see www.cochrane.org/information-researchers
-and-authors).
The way data extracted from studies is
For more experienced researchers, the Preferred presented depends on the type of data being
Reporting Items for Systematic Reviews and handled. If they are quantitative data, the usual
Meta-Analysis (PRISMA) (transparent reporting method is to present them in either tabular format
of systematic reviews and meta-analyses) or as a chart. If they are qualitative data, then it is
document, outlines how reviews should be usual to present them categorised by themes and
reported in academic journals (Moher et al 2009). sub-themes. Table 7 shows an example of how
It is important to present a description of all Isabelle could present the number of participants
the included studies. Ideally, the details of what in each study. The types of interventions used in
is presented should be the same for each study.
Table 5 shows an example of how information
could be presented in tabular format, with a TABLE 6
fictional example of the details that could be How the critique of studies can be presented
included for one study, using Isabelle’s case using Isabelle’s case study as an example
study as an example. Article 1 (Jones 2008)
It is worthwhile critiquing each study
Study purpose Yes.
individually and then presenting a shortened Was the purpose
version of the answers in a table. This will be stated clearly?
useful when discussing the results later in the
Literature Highly relevant literature
review. An example of the first four questions
Was relevant background was discussed in
and answers for Isabelle’s article, using the
literature reviewed? appropriate depth, which
McMaster’s critical quantitative appraisal tool and the authors clearly demonstrated
(Law et al 1998), is presented in Table 6. a gap in the literature.
Study design The study design used
Presenting, reporting and discussing results What type of design used? was a randomised
controlled trial.
Depending on whether the systematic review
is being conducted for a dissertation or is being Sample The sample of women
written for journal publication, different results Was the sample described was described in detail.
in detail? Was the sample A sample size calculation
will be presented. This may depend on
size justified? was carried out to
submission requirements for a journal. For
justify the sample size.
example, The Cochrane Library has its own
TABLE 5
A template for describing studies to be included in a review, based on the PICO structure
Study Population Intervention Comparative Outcomes
intervention
1. Jones J (2008) Effect Sample selection: 50% in a domestic 50% usual care. Quality of life (QOL) scales.
of advocacy compared Volunteers recruited violence support group.
to usual care on women’s from advertisements Advocacy group
quality of life. Journal posted in various Pre-intervention
of Clinical Nursing. community agencies. QOL score: 30/50 (50 is
10, 5, 345-352.* the average figure for QOL
Number: 60. for healthy individuals).
Post-intervention
Mean age: 24 years old, QOL score: 40/50.
range 21-51 years.
Usual care group
Abusive relationship Pre-intervention
status: QOL score: 29/50.
45% were currently Post-intervention QOL
in abusive relationships score: 30/50 (no difference
with no intention of in QOL following treatment).
leaving, 35% were
trying to leave abusive
relationships.
*Fictional article.

54 august 25 :: vol 24 no 51 :: 2010 NURSING STANDARD

p47-56w51_LEARNING ZONE 23/08/2010 10:22 Page 55

the included studies could also be combined to

TABLE 7
produce a pie chart or they could be presented
in a table. Table 8 illustrates how the percentages Number of participants in each study
of the types of interventions that were delivered Article Number
to the participants in the intervention and
Davies (1992) 40
control groups could be presented for publication
in a journal. The pie chart (Figure 2) displays Smith (2005) 70
an alternative method that could be used if Bettany (2008) 80
the information is intended for presentation in
a dissertation. Jones (2008) 60
Outcomes Outcome measures help determine
the answer to the research question. As with the TABLE 8
population and intervention data, outcome Types of intervention
measures such as quality of life scores can be
presented in a table or a graph. Table 9 shows Types of intervention Percentage of participants
examples of how Isabelle could present quality Individual sessions 10%
of life scores in her articles both before and after
Group advocacy 30%
the interventions.
Discussion of the review Having reported the Both 10%
findings from the studies clearly and concisely, Usual care 50%
the next step is to discuss the findings in relation
to the relevant background literature and theory,
as well as the specific aims and objectives. The FIGURE 2
discussion section should begin with a summary Percentage of participants receiving each
of the major findings. It is important to discuss type of intervention
findings by comparing and contrasting the
results, as well as relating the discussion to the
background literature, and not simply to 30%
repeat the information in the results section. 50%
Depending on the type of review (qualitative or
quantitative), the theoretical frameworks are
usually discussed in the discussion section
(mainly for quantitative reviews) while some 10%
authors choose to combine the two (writing up
the results and discussion together in the same 10%
section). This is done more frequently for
qualitative reviews. Some suggested issues Usual care Individual sessions
that could be discussed in the section are Group advocacy Both
highlighted in Box 3.

Time out 8 Conclusion

Write a plan of how you could
This article has provided an overview of the stages
present your results from,
involved in undertaking a systematic review.
and structure the discussion of,
A case study has been used to illustrate the search
a systematic review
and appraisal strategy and the organisation and

TABLE 9
Mean quality of life (SF-36) scores before and after advocacy intervention and usual care
Advocacy group Usual care group
Before After Before After
Davies (1992) 25/50 38/50 24/50 26/50
Smith (2005) 23/50 41/50 23/50 24/50
Bettany (2008) 21/50 44/50 18/50 17/50
Jones (2008) 30/50 40/50 29/50 30/50

NURSING STANDARD august 25 :: vol 24 no 51 :: 2010 55

p47-56w51_LEARNING ZONE 23/08/2010 10:22 Page 56

it is also important to write up the results and

learning zone research methods publish them so that other nurses can benefit
from the findings NS

presentation of data. Nurses who wish to carry

out a systematic review need to know how to
search, select, appraise and synthesise relevant Time out 9
literature. Conclusions and findings are
Now that you have completed
important as they may generate new evidence
the article, you might like to
and enhance practice.
write a practice profile. Guidelines
Although the primary aim of a review is to
to help you are on page 60.
answer a clinical question and improve practice,

BOX 3
Issues to be addressed in the discussion section of a report of a systematic review
4Summarise the findings in words.
4Develop and/or discuss the theory as to how the intervention works (if this is what is being investigated).
4Compare and contrast the findings of the study and compare them to the work of other authors.
4Discuss the quality of the included studies. Does it affect the outcome of the results?
4Relate the findings to the initial area of interest.
4Interpret the findings in relation to the literature reviewed in the background section.
4Support a particular theory or model that might be dominant in the area.
4Discuss any methodological shortcomings or flaws in the systematic review and state how these may affect the
interpretation of the results. Recommendations on how these shortcomings may be rectified in future studies
would be beneficial.
4Suggest any implications for existing theory or research.
4Discuss the findings with respect to practice.
4Discuss the ethical aspects of the included studies.
4Discuss whether or not practice should change as a result of the review, giving the rationale.
4Suggest ideas for future research on the topic.
4Identify the overall conclusions of the study.

References
Alderson P, Green S (2002) Reviews: CRD’s Guidance for Lahlafi A (2007) Conducting a Sciences: A Practical Guide.
Cochrane Collaboration Open Undertaking Reviews in Health Care. literature review: how to carry out Blackwell Publishing, Oxford.
Learning Material for Reviewers. www.york.ac.uk/inst/crd/ bibliographical database searches.
Popay J, Roberts H Sowden A
www.cochrane-net.org/openlearning systematic_reviews_book.htm British Journal of Cardiac Nursing.
et al (2006) Guidance on the
(Last accessed: August 5 2010.) (Last accessed: August 5 2010.) 2, 12, 566-569.
Conduct of Narrative Synthesis
Armstrong R, Jackson N, Dickersin K, Chan S, Law M, Stewart D, Pollock N, in Systematic Reviews: a produce
Doyle J, Waters E, Howes N Chalmers TC, Sacks HS, Letts L, Bosch J, Westmorland M of The ESRC Methods Programme.
(2005) It’s in your hands: the value Smith H Jr (1987) Publication bias (1998) Guidelines for Critical Review Version 1. Institute of Health
of handsearching in conducting and clinical trials. Controlled Clinical Form – Quantitative Studies. Research, Lancaster.
systematic reviews of public health Trials. 8, 4, 343-353. www.srs-mcmaster.ca/Portals/ The Cochrane Collaboration
interventions. Journal of Public 20/pdf/ebp/quanguidelines.pdf (2009) Cochrane Handbook for
Egger M, Zellweger-Zähner T,
Health. 27, 4, 388-391. (Last accessed: July 27 2010.) Systematic Reviews of Interventions.
Schneider M, Junker C,
Bruce N, Pope D, Stanistreet D Lengeler C, Antes G (1997), Moher D, Liberati A, Tetzlaff J, www.cochrane-handbook.org
(2008) Quantitative Methods Language bias in randomised Altman DG (2009) Preferred (Last accessed: August 5 2010.)
for Health Research: A Practical controlled trials published in English Reporting Items for Systematic Timmins F, McCabe C (2005)
Interactive Guide to Epidemiology and German. The Lancet. 350, 9074, Reviews and Meta-Analyses: How to conduct an effective
and Statistics. First edition. 326-329. The PRISMA Statement. literature search. Nursing Standard.
Wiley, London. www.prisma-statement.org 20, 11, 41-47.
Jadad A (1998) Randomised
Centre for Reviews and Controlled Trials: A Users Guide. Petticrew M, Roberts H (2006) Torgerson C (2003) Systematic
Dissemination (2009) Systematic BMJ Books, London. Systematic Reviews in the Social Reviews. Continuum, London.

56 august 25 :: vol 24 no 51 :: 2010 NURSING STANDARD

p58w51_Assessment 20/08/2010 13:21 Page 58

learning zone assessment

c) Ethical implications ❏
Systematic review: 2 d) All of the above

9. Which of the following is not

❏

TEST YOUR KNOWLEDGE AND WIN A £50 BOOK TOKEN a Boolean operator?
a) WITH ❏
HOW TO USE THIS ASSESSMENT b) AND ❏
c) NOT ❏
This self-assessment questionnaire (SAQ) 59-65 Lowlands Road, Harrow-on-the-Hill, d) OR ❏
will help you to test your knowledge. Each Middlesex HA1 3AW, or send them by email
week you will find ten multiple-choice to [email protected]. 10. External validity is the
questions that are broadly linked to the Subscribers can complete the assessment
extent to which:
learning zone article. Note: There is only
one correct answer for each question.
at www.nursing-standard.co.uk by clicking
on the CPD link on the left of the homepage.
a) Trial design minimises bias ❏
b) Trial analysis minimises
Ways to use this assessment Ensure you include your name and address bias ❏
4 You could test your subject knowledge by and the SAQ number. This is SAQ no. 557. c) Findings are generalisable ❏
attempting the questions before reading Entries must be received by 10am on d) Trial conduct increases bias ❏
the article, and then go back over them to Tuesday September 7 2010.
see if you would answer any differently. This self-assessment questionnaire
When you have completed your
4 You might like to read the article to update self-assessment, cut out this page and add it was compiled by Tanya Fernandes
yourself before attempting the questions. to your professional portfolio. You can record
the amount of time it has taken. Space has The answers to this questionnaire
Prize draw
been provided for comments. will be published on September 8
Each week there is a draw for correct entries.
Please send your answers on a postcard to You might like to consider writing a practice
Zena Latcham, Nursing Standard, The Heights, profile, see page 60.

Report back
1. To conduct a successful unbiased c) 52% ❏
systematic review it is important to: d) 73% ❏ This activity has taken me ____ hours to
a) Access as many relevant complete.
studies as possible ❏ 5. A comprehensive search Other comments:
b) Search a variety of databases ❏ strategy should include:
c) Establish inclusion and a) Books ❏
exclusion criteria ❏ b) Journal articles ❏
d) All of the above ❏ c) Grey literature ❏
d) All of the above ❏ Now that I have read this article and
2. Which of the following completed this assessment, I think
statements is correct? 6. What does the letter ‘C’ in the my knowledge is:
a) All research is easily accessible ❏ acronym PICO stand for? Excellent ❑
b) Not all research is published a) Comparative intervention ❏ Good ❑
in journals ❏ b) Community ❏ Satisfactory ❑
c) All research published in journals c) Conclusion ❏ Unsatisfactory ❑
is indexed in major databases ❏ d) Criteria ❏ Poor ❑
d) All research is reliable ❏ As a result of this I intend to:
7. Phase 3 of a systematic
3. Why declare that a search includes review involves:
English language articles only? a) Selection of studies ❏
a) To minimise bias ❏ b) Data extraction ❏
b) To encourage transparency ❏ c) Appraisal of studies ❏
c) To acknowledge language bias ❏ d) Reading the full text of
Answers to acutely ill patients
d) To decrease numbers of articles ❏ each study ❏ questions
The answers to SAQ no. 555 on
4. What percentage of references 8. Which of the following factors are acutely ill patients, which appeared
included in the Medline database important when assessing the quality in the August 11 issue, are:
originates from the United States? of randomised controlled trials?
1. d 2. b 3. a 4. c 5. d
a) 15% ❏ a) Research question’s relevance ❏ 6. a 7. d 8. c 9. b 10. d
b) 38% ❏ b) Internal validity of the trial ❏
58 august 25 :: vol 24 no 51 :: 2010 NURSING STANDARD
Copyright of Nursing Standard is the property of RCN Publishing Company and its content may not be copied
or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission.
However, users may print, download, or email articles for individual use.