Report
on search method
Assessment type;
Report
Word limit/length;
1500 Words (all inclusive of in-text citations, reference list, any tables/figures, and appendices)
Overview
Report on the methods section of the literature search including databases, search strategy, eligibility criteria used to select included studies, and flow chart.
Learning Outcomes
This assignment task is aligned with the following learning outcomes:
1. Develop a research question and problem statement to guide critical examination of an area of personal interest in Mental Health, located within your discipline.
2. Discern between and critique the evidence supporting further research and practice improvement.
Assessment details
The aim of Assignment 2 is to produce a transparent, reproducible Method Section for your literature review. You need to provide sufficient information to allow the reader to replicate the entire search process. Furthermore, in replicating the search process the reader should also be able to arrive at the same number of studies that met the inclusion criteria.
Instructions
Identify the databases, search strategy, and eligibility criteria used to select included studies. You must also provide some rationale for excluding studies. The three templates that follow have been designed to provide the guidance and structure to assist the production of a transparent and reproducible Methods section.
Each template: the Method template, Database search template (Table 1), and PRISMA graphic (Figure 1), should each be filled in and included in your report. There should be
NO introduction and NO conclusion as the assignment is merely the Method section of the critical literature review you will complete for Assignment 3. Further guidance is provided in the checklist below,
Assignment 2 Checklist
Use this checklist to assist completion of a transparent, reproducible method section for your critical literature review. Note, that the three templates that follow make up all of your report of the method section.
1. Include your REVISED research question from Assignment 1.
2. Identify all the databases searched (minimum of
four databases). Please use PubMed and Medline as alternatives, due to high similarities.
· Identify all search terms in ‘Table 1’ (i.e. using the Database search template that follows). Search terms can be: MeSH; Database thesaurus compatible and/or Free text
3. Identify all limiters in ‘Table 1’, including the time period of search, based on your justification.
4. Define and justify the eligibility criteria:
· the study designs (e.g., specify which quantitative/qualitative/mixed method study designs)
· the type of interventions and/or outcomes (for experimental studies)
· participant characteristics (e.g., the population, age range, gender) and the study location
· the publication type and language (e.g., original peer-reviewed English articles)
5. Run and revise the search. Undertake first and second cull, as below.
6. Explain the process of the first cull of articles. Identify the number of duplicates. Identify the number of articles rejected based reading on the titles and abstracts; give example reasons.
7. Explain the process of the second cull of articles. Select three or more reasons, based on inclusion/exclusion criteria. This cull should be based on reading the full-text paper.
8. Identify the number of records after database searching and after duplicates are removed; these numbers should be the same in Table 1 and Figure 1.
9. Fill in the templates that follow: The Method template, Database search template (Table 1), and modified PRISMA flow diagram (Figure 1). The flow diagram should demonstrate the processes used to arrive at the number of studies you will use in your review.
10. Provide References in APA 7th style
Method Template
ADOPT THIS METHOD TEMPLATE for the second assignment. A revised version of the search strategy, using this template, should also be included in Assignment 3. Guidance is provided in the
italicized text contained in the [square brackets]; this text is
only the instructions and must be removed when you submit the method section. Please ignore the similarity in parentage related to the shared material.
Research question: [
State the research question, if you have revised it, use that question]? The format [
insert, e.g. PICO] was used to structure the question and guide the search. [
List the P, I, C, O – or relevant – according to your chosen format].
Search strategy: The [
insert names of Databases used] electronic databases were used to conduct systematic literature searches. These databases were chosen as they are [
insert one/two sentences about the database characteristics (e.g. no pre-set filters, approx. number of citations and relevance) to justify use]. The following key search terms were selected and combined with Boolean operators: [
insert your key search terms here]. [
Add one sentence about the delimiters – e.g. were they similar, different for each database?]
. Secondary and grey sources of literature were excluded from this review.
The search strategy, including the databases, key search terms, and the number of articles found using this method, is shown in Table 1. The search was undertaken during [
enter the month and year]; it was limited to articles published between [
insert period and reason]. The following inclusion criteria were applied during article selection: [
list inclusion criteria, for example, as follows; justify these criteria as necessary]
· Study designs and aim/s
· Intervention and outcome
· Participants and setting
· Publication type and language
This literature search method yielded XXX articles. Once duplicates (n=XX articles) were removed, the abstract of XXX articles was screened for inclusion in this review. The majority [
only state this if it is indeed true and applies to your search process] of the rejected articles (n=XXX) were [
give a few examples of types of studies/articles rejected]. These articles were excluded. [
This is the first cull of articles.]
The remaining XX full-text articles were screened for inclusion in this review. [
This is the second cull of articles]. Further XX articles were excluded for the following reasons: [
Give 3 or more reasons for exclusion]
· Reason ……………………………………………
· Reason ……………………………………………
· Reason ……………………………………………
See
Figure 1
for the modified PRISMA flow diagram
Database search Template
ADOPT THIS DATABASE SEARCH TEMPLATE to report your database search strategy. Table 1 provided here, for guidance, is an example from an actual literature review by Brownie and Nancarrow (2013, p. 3).
You will need to substitute with your information i.e. your search terms and no. of articles discovered.
Table 1
Databases and search terms used to identify
literature for review
(Adopted from Brownie, S., & Nancarrow, S. (2013). Effects of person-centered care on residents and staff in aged-care facilities: A systematic review.
Clinical Interventions in Aging, 8, 1-10)
Database
Search terms
No. articles
Medline
patient-centered care [MeSH] OR “person-cent#red care” OR “patient-focus#ed care” OR “resident-cent#red care” OR “relationship-cent#red care” OR “individuali#ed care” OR “resident-oriented care”
N =
AND
homes for the aged [MesH] OR nursing homes [MesH] OR residential facilities [MesH] OR long-term care [MesH] OR assisted living facilities [MesH]
OR
“Eden Alternative” OR “Green House Model” OR “Wellspring Model”
Limiters:
1995-2012; English
Cinahl
patient-centered care [Heading] OR “person-cent#red care” OR “patient-focus#ed care” OR “resident-cent#red care” OR “relationship-cent#red care” OR “individuali#ed care” OR “resident-oriented care”
N =
AND
nursing homes [Heading] OR residential care [Heading] OR residential facilities [Heading] OR housing for the elderly [Heading] OR “long-term care”
OR
“Eden Alternative” OR “Green House Model” OR “Wellspring Model”
Limiters:
1995-2012; English, Research article [Publication type]
Scopus
“person-cent?red care” OR “patient-focus?ed care” OR “resident-cent?red care” OR “relationship-cent?red care” OR “individuali?ed care” OR “resident-oriented care”
N =
AND
“nursing home*” OR “residential care” OR “residential facilities” OR “housing for the elderly” OR “long-term care” OR “long-term care facilities”
OR
“Eden Alternative” OR “Green House Model” OR “Wellspring Model”
Limiters:
1995-2012, Article [Publication type], article title, abstract, and keywords
ProQuest
person-centered care [Search term] OR patient-centered care [Search term] OR “person-centered care” OR “patient-centered care” OR “patient-focus?ed care” OR “resident-cent?red care” OR “relationship-cent?red care” OR “individuali?ed care” OR “resident-oriented care”
N =
AND
residential aged care [Search term] OR aged care homes [Search term] OR homes for the aged [Search term] OR nursing homes for the elderly [Search term] OR nursing home* [Search term]
OR
“Eden Alternative” OR “Green House Model” OR “Wellspring Model”
Limiters:
1995-2012, English, peer-reviewed, article [Document type]
Total records identified after database searching
N =
Total records after duplicates removed
N =
Modified PRISMA flow diagram
ADOPT THIS Modified PRISMA flow diagram TEMPLATE to report the screening and selection of articles to include in your Critical review. The diagram is modified from Figure 1 in Brownie and Nancarrow (2013, p.4).
Please follow the instruction in the square bracket and use your data to populate the modified PRISMA flow diagram.
Figure 1
Modified PRISMA flow diagram of article screening and selection
Identification
Potentially relevant articles identified through database searching (n =)
Articles after duplicates removed (n =)
Screening
Articles excluded after assessing titles/abstract, based on inclusion/exclusion criteria (n=)
Eligibility
Full-text articles assessed for eligibility (n = 21)
Articles excluded after full-text assessment, based on inclusion/exclusion criteria (n=)
Additional studies identified through
citation chaining (n =)
Articles excluded after full-text assessment, based on inclusion/exclusion criteria (n=)
Included
Total papers included (n =)
Type of the paper included [e.g., Cluster-randomized trial (n =)]
Type of the paper included [e.g., Quasi experimental studies (n =)]
Assignment 2 Rubric
Criterion
HD (>26)
PA (15-19)
Fail (<15) Research Question (2%) · Included with formatting (e.g., PICO) · Suitably focused and structured · Provides a structured research question for a worthwhile critical literature review All criteria were met and clearly described. Some criteria were met and described. Criteria not met or described. Database and search strategy as per templates (11%): · At least four databases with justification / · Search terms/MeSH, database thesauri compatible terms and free text/Boolean operators, wildcards & truncations/Use of search period and reason · Table correctly labeled with the title above the table · Accurate search strategy indicating transparency/reproducibility of the method All criteria were met and clearly described. Some criteria were met and described. Criteria not met or described. Definition & justification of inclusion criteria (4%) · Study designs · Nature/type of intervention/outcome (where applicable) · Participants and settings · Publication type and language All criteria were met and clearly described. Some criteria were met and described. Criteria not met or described. Explanation and rationale for the first cull, based on titles/abstracts (3%) · Number of duplicates · Number of articles rejected All criteria were met and clearly described. Some criteria were met and described. Criteria not met or described. Explanation and rationale of a second cull (3%) · Number of excluded papers · Excluded studies grouped logically by reason for exclusion All criteria were met and clearly described. Some criteria were met and described. Criteria not met or described. Modified PRISMA flow diagram (3%) · A logical, clear, and accurate diagram · Figure correctly labeled above the diagram All criteria were met and clearly described. Some criteria were met and described. Criteria not met or described. Writing, structure, format (2%), and APA 7th edition referencing (2%) · Word limit met, considering clear and succinct writing based on APA 7th format · Grammatically correct writing · Adoption of all three templates · All needed citations are included and all references are based on APA 7th format All criteria were met and clearly described. Some criteria were met and described. Criteria not met or described.
14
Best Practice & Research Clinical Rheumatology 27 (2013) 295–306
Contents lists available at SciVerse ScienceDirect
Best Practice & Research Clinical
Rheumatology
journal homepage: www.elsevierheal th.com/berh
10
Else Marie Bartels, PhD, DSc, Research Librarian DB *
The Parker Institute, Department of Rheumatology, Copenhagen University Hospital Frederiksberg and
Bispebjerg, Ndr. Fasanvej 57, 2000 Frederiksberg, Denmark
Keywords:
Bibliographic databases
Evidence-based medicine
Information literacy
Information services
Internet
Literature
* Tel.: þ45 38164168; fax: þ45 38164159.
E-mail address: else.marie.bartels@regionh.dk.
1521-6942/$ – see front matter � 2013 Elsevier Lt
http://dx.doi.org/10.1016/j.berh.2013.02.001
All medical practice and research must be evidence-based, as far as
this is possible. With medical knowledge constantly growing, it has
become necessary to possess a high level of information literacy to
stay competent and professional. Furthermore, as patients can now
search information on the Internet, clinicians must be able to
respond to this type of information in a professional way, when
needed. Here, the development of viable systematic search stra-
tegies for journal articles, books, book chapters and other sources,
selection of appropriate databases, search tools and selection
methods are described and illustrated with examples from rheu-
matology. The up-keep of skills over time, and the acquisition of
localised information sources, are discussed.
� 2013 Elsevier Lt
d. All rights reserved.
Introduction
Medical information, mainly in the form of scientific papers but also as books and other types of
resources (mostly as Internet sites), is growing at a remarkable rate. One result of this is that a high
level of information literacy is required by all who wish to keep up-to-date in their field. Another part
of information literacy is to be able to trace the information patients have found on the Internet, and to
assess this in a professional way, in order to keep a good patient–doctor relationship, where the patient
has confidence in the doctor’s knowledge and skills.
Although general-purpose searching via a search engine (e.g., using Google [1]) or a search-engine-
type search inMedline via PubMed [2] (see later) may cover your information need to some degree, it is
important to be disciplined and focussed and to know the available information sources if you wish to
practise your daily work in an evidence-based way [3,4]. It is only when you wish to find “something
about a subject” that you might try your luck with a general-purpose search; and in this case you still
d. All rights reserved.
mailto:else.marie.bartels@regionh.dk
http://crossmark.dyndns.org/dialog/?doi=10.1016/j.berh.2013.02.001&domain=pdf
www.sciencedirect.com/science/journal/15216942
http://www.elsevierhealth.com/berh
http://dx.doi.org/10.1016/j.berh.2013.02.001
http://dx.doi.org/10.1016/j.berh.2013.02.001
E.M. Bartels / Best Practice & Research Clinical Rheumatology 27 (2013) 295–306296
have to make sure that you have found valid information, at least in the form of a review article from a
peer-reviewed journal or a textbook chapter of an acceptable standard and level. For efficient up-keep
of the needed level of professional knowledge at any given time, you need to be able to carry out a
proper systematic search and make a correct choice of information sources.
There are four steps towards reaching a level of information literacy that will make keeping up with
medical literature manageable: (1) learn to define your questions in a meaningful way; (2) get to grips
with the ins and outs of literature searching; (3) make a time schedule for necessary searches; and (4)
update yourself in new information sources at least once a year.
Literature search
Scientific papers in peer-reviewed journals
Most of the new medical literature appears as papers in peer-reviewed journals. To keep up with
this part of the information flow, you have to follow steps 1–8, below:
1. Define your problem.
2. Create a search strategy.
3. Select the right bibliographic databases.
4. Search.
5. Select suitable references from those that have been retrieved.
6. Assess whether the search was satisfactory.
7. Redesign the search strategy and/or choose other databases/search tools, where needed.
8. Repeat steps 2–6, if necessary.
Define your problem
A successful search is based on looking for the key issues, but how do you ensure that you do exactly
that? As an illustration, suppose that you wish to be updated on the effects of biologics on rheumatoid
arthritis (RA). This is not exactly awell-defined problem in search terms. There are three important key
issues: effects, biologics and RA.
Starting with RA, this might very well be as fully defined as it should be, but ask yourself whether a
further specification is needed. Is it a particular patient group in terms of age, gender, genetics or
similar? It could be that the group concerned is ‘young women with RA during pregnancy’ or another
specific group of RA patients.
Effects must also be specified – effects measured in what manner and compared to what? An
example of the question that has to be addressed could be: What is the effect of treatment with bi-
ologics compared to non-steroidal anti-inflammatory drugs (NSAIDs) treatment or to steroids?
Furthermore, is it the effect of a specific type of biologics compared with a specific type of NSAIDs? The
other part of the question is which effect am I looking at? Is it DAS28, pain reduction, joint destruction,
function or quality of life? There are many types of outcome measures, and usually you will have a fair
idea of the important ones for a particular patient group or for a specific treatment.
The last step towards a clear definition of the problem in question is to define biologics. Howbroad a
definition is allowed and which biologics are the most important ones to include. Is it really a com-
parison between biologics in general with another defined type of treatment, or is it a specified biologic
treatment, you have in mind?
The definition of the problem about which you want to find information is the base on which the
whole procedure is built, and more experienced practitioners and researchers will have a great
advantage here, being able to write out the problem of interest quickly. Often, it will be necessary to
break down the problem into sub-questions to create clear search strategies, which will lead to better
results. You must also decide if your question asks for an epidemiological approach, where you are
looking at effects of the past in awhole population and therefore cannot ask for randomised controlled
trials (RCTs), or if you aremainly interested in looking for designed studies in controlled and, if possible,
randomised studies. In all this, you also have to think clearly, and make the best use of the material
E.M. Bartels / Best Practice & Research Clinical Rheumatology 27 (2013) 295–306 297
available. Nearly all high-powered RCTs started as pilot studies. Many large epidemiological studies
began as more humble studies of smaller groups, which provided the ideas for the full-scale studies. It
is important to understand what type of studies you are looking for to get a valid answer to your
question [5].
Create a search strategy
Having delineated a well-defined question, it is possible to create a search strategy based on a
search table. Instead of rushing into a search by typing in the first words that come to mind, it is
worthwhile working out a search table. If the question is effect of exercise on physical function in
juvenile idiopathic arthritis, a search table could look like Fig. 1.
Juvenile idiopathic arthritis has several names, and it is wise to check with the MeSH database
in PubMed [2] or with keywords in EMBASE (Exerpta medica) [6] (see below) to get further ideas
concerning the various names used for the same condition by different authors.
For the juvenile idiopathic arthritis patients exercise will be limited, and it might be carried out
under the supervision of a physiotherapist. Keeping this in mind whenworking out the search table, a
list of suggestions covering exercise, again using some known keywords from the medical databases, is
given, and more possibilities may be added. For the last term, physical function, a set of known
outcome measures are given. There are more of these, and the choices here are scales that specifically
are used for children, because this is a child-specific disease. In each column, each representing a key
issue, at least one of the given terms has to be found in a reference to include this in the retrieved
references. The terms in each column are therefore combined with ‘OR’ when searching. This will give
rise to three sets of results, one from each column.
As all of the threemain issues in this example (other problems could havemoremain issues) have to
be included in the total search, the results of the three searches, one for each column, have to be
combined with ‘AND’, when searched. The end search will be (juvenile idiopathic arthritis OR juvenile
rheumatoid arthritis OR juvenile chronic arthritis OR juvenile onset Still’s disease) AND (exercise*
OR
physical therapy OR jogging OR swimming OR pool therapy OR dancing) AND (physical function OR
exercise test OR CHAQ OR JASI OR JAFAS OR joint range of motion).
The above is a simple example. The search can usefully be extended much further, and the given list
here is a mixture of ‘free text words’, words appearing anywhere in a reference in the searched
database, and defined keywords (MeSH words), which might be specific for the particular database. It
is necessary to search both terms that are given keywords in the database, if a keyword covering the
term exists, and other words covering the term in question. Although it is important to search the
keywords/MeSH words when available, they should in general be searched as both keywords and as
Juvenile idiopathic arthritis Exercise* Physical function
Juvenile rheumatoid arthritis Physical therapy Exercise test
Juvenile chronic arthritis Jogging CHAQ
Juvenile-onset Still’s disease Running JASI
SAFAJgnimmiwS
Aquatic therapy Joint range motion
gnicnaD
gnilcyC
AND
OR
Fig. 1. A possible search table for the question ‘Effect of exercise on physical function in juvenile idiopathic arthritis’. The table may
be developed further, especially in the Exercise and in the Physical function column.
E.M. Bartels / Best Practice & Research Clinical Rheumatology 27 (2013) 295–306298
free text words. This will be the case if all search words are searched as ‘free text words’. The reason for
this is that a keyword may only have been in use for some of the years covered by a database and not
always been used to index a relevant reference during these years. In the example, it should be noted
that exercise has an * after the word. This causes the search to include any term that starts with ‘ex-
ercise’. This is called truncation of or adding a wild card to the search term.
Truncation is also used to account for different types of English language spelling, for instance
*edema for edema or oedema (American and British English spellings).
The broader you search in terms of keeping each term less defined, the more ‘noise’ (useless/un-
wanted references) will come out of the search. As an example, you could search osteoarthritis without
limiting your search to a joint such as the knee. You will get several studies on other joints than the
knee. However, you will also catch some studies of knee osteoarthritis that did not appear in your
narrower search. You will therefore achieve a higher sensitivity (a better coverage of the literature)
with the broader search, but the cost is the high number of retrieved references you have to assess for
inclusion, where several do not give an answer to your request.
The alternative is a very specific, narrow definition of the problem in question in your designed
search strategy. This will give a high specificity (more or less all of the retrieved references will be
relevant), but the coverage of the problem will most likely not be optimal. Depending on your infor-
mation needs, you should aim at a search strategy that will give ‘enough information’ for your purpose.
For updating, a more specific searchmay be preferred, whereas youmay need several broader searches
for research purposes.
Select the right bibliographic databases
When the question is clearly defined and a search strategy created, it is time to choose the right
database(s) or other search tools. For medical literature, there is a good range of bibliographic
databases.
Bibliographic databases cover only a certain chosen number of journals, and the selection varies
from database to database. This is why you have to consider which databases are the ones to search for
your particular subject area. Furthermore, it is important to remember that although databases may
seem alike, they are not. Two databases may have the same structure but their keywords and names for
the various fields (‘tags’ such as author, address and abstract) are very often different. The way one
database searches may also differ from all the others. When designing a search strategy, it is important
to look for possible keywords that define your search terms andmake sure the meaning of these words
really is the same as your understanding of the words. You have to understand that a term used in your
local clinical or laboratory setting is not necessarily used for the same notion in a database and that a
term used in a database might have a different meaning than the one you normally would expect.
Further, the terminology and spelling will vary between countries and languages (e.g., between
American English and British English). You will, as mentioned, take this into account by using trun-
cation/wild cards.
In rheumatology there will be a need to search a set of bibliographic databases, depending on the
area of interest. For a quick update, you will get by with a search in Medline [2] and/or EMBASE [6], but
if you want to get a complete update of your field, you will probably need to search more than these
two databases.
Table 1 shows a selection of bibliographic databases of interest, with a short explanation and a
suggestion of what to search where. It is worth knowing that new research areas first will appear as
meeting abstracts, and these are mainly found in Web of Science [7], Biosis Previews [8] and EMBASE
[6]. For special areas, a set of smaller databases is also shown in Table 1.
EMBASE [6] and – to a certain degree –Medline [2] cover some journals written in languages other
than English, but if you wish to search articles in other languages you may find that the country
speaking the language provides a bibliographic database inwhich you can search in the given language.
Bibliographic databases come in various wrappings, depending on who delivers them to the user.
The delivery firm is called the ‘database host’, and one host can give access to a wide variety of da-
tabases. What the host provides is the design of the search page. PubMed is really the host for several
databases, apart from Medline. Other examples of hosts are STN, OVID and EBSCO. A database will
Table 1
List of databases of interest for rheumatologists.
Name [Ref.] Description Seen from the rheumatologist’s point of
view, good for searching:
Medline/PubMed [2] Medicine, human biology, general physiology,
cell biology. Medline is provided from most
database hosts; the PubMed version is the
hosting service from US National Library of
Medicine, which is the creator of the database
Medline.
All clinical and physiological questions.
Good coverage of accreditation and
management.
EMBASE [6] Medicine, human biology, general physiology,
cell biology. Abstracts from some larger
medical conferences.
All clinical and physiological questions.
Very good coverage of musculoskeletal
diseases and their treatment.
Strong in the area of Pharmacology.
Broader coverage of European journals
than Medline.
Psycinfo [9] Psychology. Includes books and book chapters,
as well as journal articles.
References include reference lists.
Human psychology such as patient–
doctor relationships and cognitive therapy.
Useful references from reference lists
of found references.
Cochrane Library [10] The database consists of Cochrane Database of
Systematic Reviews (CDSR), Database of
Abstracts of Reviews of Effectiveness (DARE)
and Cochrane Controlled Trials Register (CCTR).
A small limited database of high quality.
Systematic reviews with meta-analyses,
and RCTs, concerning clinical treatment
approaches as a base for evidence-based
treatment.
Web of Science [7] Science, technology, health sciences, sociology,
and humanities. Abstracts from some larger
medical conferences.
Coverage of a wide variety of subjects.
Not so strong on a specific search of a
subject area, but excellent for catching
references from interdisciplinary areas,
as well as conference abstracts.
Biosis Previews [8] Microbiology, genetics, cell biology, general
physiology and biochemistry, behaviour,
botany, ecology. Some conference abstracts.
Cell biology, genetics, and other basic
bio-medical areas. Includes a good
selection of conference abstracts.
Chemical Abstracts [11] Chemistry and biochemistry. Drugs and drug treatment.
Search at very high level due to use of
CAS numbers which will relate to any
name given to a particular drug.
Toxnet [12] NLMs special entrance to free databases
concerning toxicology.
LactMed for drug effects during breast
feeding.
TOXLINE for adverse effects of drugs.
PEDro [13] PEDro covers physiotherapy and includes
references to systematic reviews, RCTs, and
clinical practice guidelines. Small, limited,
high-quality database. Very basic search system.
Evidence-based physiotherapy.
Only very basic searches can be
performed
CINAHL [14] Nursing and allied health research database.
References include reference lists.
Useful when searching for nursing and
physiotherapy information.
Reference lists of found references may
give some useful guidelines etc.
AMED [15] Allied and complementary medicine. Physiotherapy, palliative care,
occupational therapy
MANTIS� [16] Manual, alternative, and natural therapy
index system (earlier chirolars)
Manual medicine, chiropractic,
osteopathy. Generally strong on
rehabilitation.
Clinical Trials.gov [17] A registry and results database of publicly and
privately supported clinical studies of human
participants conducted around the world.
Clinical trials which have happened or
are on the way.
PROSPERO (International
Prospective Register of
Systematic Reviews) [18]
Prospective register of systematic reviews. Place for registering protocols for
systematic reviews and meta-analyses.
(continued on next page)
E.M. Bartels / Best Practice & Research Clinical Rheumatology 27 (2013) 295–306 299
Table 1 (continued )
Name [Ref.] Description Seen from the rheumatologist’s point of
view, good for searching:
CrossRef [19] CrossRef’s classifies itself as the citation
linking backbone for all scholarly information
in electronic form. CrossRef links via CrossRef
Digital Object Identifiers (CrossRef DOI) a
reference to the electronic full-text reference,
such that the DOI-number is unique for the
particular reference.
Searching references via DOI number,
or finding DOI-number for a reference.
Derwent Innovation
Index [20]
Patents. Useful if one wishes to patent a
treatment and need to find what is
already patented in the area.
Journal Citation
Reports [21]
Gives impact at the journal and category
levels, as well as presenting the relationship
between citing and cited journals.
Useful for finding impact factors
(remember these change every year),
as well as cited and citing half life.
E.M. Bartels / Best Practice & Research Clinical Rheumatology 27 (2013) 295–306300
therefore look different on different hosts, but – despite this – the database behind will be exactly the
same, the search system will be the same and you should not search the same database twice by
searching the same database via two hosts. For example, it is only necessary to search Medline either
via PubMed or via OVID, despite the different appearances of this database in the two hosting systems.
Another problem can occur when searching several databases provided from the same host in one
search. This will not give the best and most professional results because the databases behind the
common search face are different in structure, and the full benefit of your selection of keywords,
publication types, etc. will not be obtained. It is valuable to search each database separately. This will
also allow you to download chosen references from your searches for import to reference handling
systems (Reference Manager [22], Endnote [23], Procite [24] and others), where knowledge of both
database and host is demanded for successful import into your own reference databases.
Search
There is a continuous development in the ways search systems search databases.
Artificial intelligence is part of many search systems, but its quality varies. Occasionally you will get
great benefits from the artificial intelligence and get a better search, but at other times you will find
some search results that seem very far from your intended search. It all depends on the way you
approach your database in the form it is made available to you. In Medline searched via PubMed [2],
which uses artificial intelligence, you can search by introducing the whole search string (the end
search). You can see how the database has been searched by looking at ‘search details’ (the box at the
right-hand side of the screen), which explains why your search gives the references retrieved. Instead
of relying on the artificial-intelligence approach, you could choose to search each term alone. When
you have carried out the single-term searches, you can then go to ‘Advanced’where from ‘History’ you
combine the terms with AND or OR as appropriate, where each searched termwill be represented with
#1 or #2, etc. Your further search will look something like this: (#1 OR #3) AND (#4 OR #5).
In the PubMed version of Medline, it is easier to learn to handle the total search string created from
your search strategy as soon as you feel confidentwith thewhole search technique and have designed a
good search strategy for updating. The advantage in searching one term at a time to start with is, on the
other hand, that you are able to understand if your choice of search terms is satisfactory. If you have
only two hits on one of your search terms, and you know the area is well described, you know you have
to find another word for your term.
Apart from the operators AND and OR, you can use NOT, although this operator should be used
carefully and only in situations inwhich you are completely certain about what to exclude. For instance,
you can use NOT when you have carried out two different searches with different search strategies and
you want to exclude the references in the second set of results that have already appeared in the first
set. If your first set of results come from search #15 and your second set of results come from search
#26, you will get the results only appearing in #26 and not in #15 by searching: #26 NOT #15.
E.M. Bartels / Best Practice & Research Clinical Rheumatology 27 (2013) 295–306 301
The consequences of using the Boolean operators AND, OR or NOT is shown Fig. 2.
The main medical databases such as Medline and EMBASE have keywords organised as a tree
structure, withmain headings and subheadings. In PubMed [2], you can find these in theMeSHDatabase
(choose MeSH in the search field next to the PubMed logo). If you search the term ‘osteoarthritis’ you
will get an explanation of the use of this term in Medline. By going one step deeper, you will get sub-
headings and the tree structure (where this term appears in the Medline hierarchy of keywords):
All MeSH categories
Disease category
Musculoskeletal diseases
Joint diseases
Arthritis
Osteoarthritis
Osteoarthritis, hip
Osteoarthritis, knee
Osteoarthritis, spine
All MeSH categories
Disease category
Musculoskeletal diseases
Rheumatic diseases
Osteoarthritis
Osteoarthritis, hip
Osteoarthritis, knee
Osteoarthritis, spine
You can see that osteoarthritis appears in two parts of the tree: rheumatic diseases and joint
diseases.
InMedline youwill search the term in all parts of the tree, and youwill also search all narrower terms
belonging to the particular MeSH. If you search for ‘osteoarthritis’, you will automatically search ref-
erences with osteoarthritis; osteoarthritis, hip; osteoarthritis, knee; and osteoarthritis, spine. However,
if you wish to limit your search to, for instance, diagnosis, you may in the PubMedMeSH database do so
by choosing the subheading Diagnosis and ‘send to box with AND’, search and see the results.
In Medline searches via other hosts than PubMed, and in EMBASE, there might be a choice of
‘explode search’ or not to do so. This will achieve exactlywhat it describes: ‘explode search’will include
all references with the search term or with any of the subheadings; by not exploding, you will limit
yourself to the chosen term.
Select suitable references from the retrieved ones
Depending on howmuch you choose to limit your search, you are likely to find several (and perhaps
many) references which are not relevant for your purpose. As mentioned above, you have to find the
Fig. 2. The Boolean operators.
E.M. Bartels / Best Practice & Research Clinical Rheumatology 27 (2013) 295–306302
right balance between sensitivity and specificity in the given situation. Around 200 or less is a
reasonable number to skim through by looking at titles and abstracts, where these are available. Often,
the title will help you decide whether you read further or not. Many databases have the possibility of
looking at ‘related articles’ and, if you go just one layer into the ‘related articles’ search for the chosen
references, you will get a useful supplement and a more complete coverage of the literature with your
search.
The last step is to go through the reference lists of your chosen references to determine whether
there are other references you have missed, either because they have not been included in the data-
bases searched or because your search has not been able to catch them. You need to be aware that the
electronic versions of the bibliographic databases start at different times; some go back a long way
whereas others start around 1980. Reference lists are good sources for covering important, older
literature, as well as covering important book chapters and conference proceedings.
Assess whether the search was acceptable
Whenyour initial set of relevant references has been selected, it is time to checkwhether something
is missing. A person who is new to a particular field will have a problem in assessing the expected
number of references, because it may not be easy to see whether one should expect five studies or
1000. If the subject is a ‘hot topic’, a lot of studies ought to appear in that area. However, if the subject is
brand new, there might not be more than – say – three conference abstracts. If you already know about
one or two important studies, these ought to appear as part of your retrieved references. If they do not,
your search strategies are not good enough, or you have not chosen the right combination of databases,
or the references in question are not included in any of the available databases and have to be found
elsewhere.
Redesign the search strategy and/or choose other databases/search tools
If your coverage of the literature was found to be unsatisfactory, you have to go back to step 1 and
start again. You may get some help by looking at the references you knowwere missing to see whether
you can get a clue about which additional search terms could usefully be added. In addition, look at
Table 1 to see whether a search in another database would be the answer. You will then have to repeat
steps 2–6. Otherwise, try other search tools.
Keeping up with the peer-reviewed journal articles
When you are satisfied with your search result, save your search strategy for use at later up-dates.
Every time you update your knowledge in a field, you have to consider whether it is necessary to
improve your search strategies. This is because your field develops all the time and you have to follow
the various ways diseases and treatments change names over time to be able to include the new terms
in your future searches. You will also find that keywords develop over time. As an example, fibro-
myalgia as a MeSH word appeared in 1989. If you want to search a term as a keyword earlier than the
keyword appears in a database, you can try to search the keyword word above in the hierarchy/tree.
Most rheumatologists will have access to a variety of bibliographic databases via their workplace or
research library. The licence fees for many of these databases are high, and you cannot expect to have
access to all databases shown in Table 1. For anybody who has no access to databases paid by their
study or workplace, the US National Library of Medicine (NLM) provides free access to Medline [2] and
some other databases via PubMed, and PEDro [13] is also free.
A major part of the medical profession will have access to the full-text versions of the journals
through their hospital or research libraries. For those who do not have access to all licenced scientific
journals, it is important to be aware of that more and more journals are becoming ‘open access’. This
means that it is free to access the papers published and that the author or the author’s employer have
paid a fee to make their paper freely available. Today many grant-giving bodies demand that the
results of the research paid by their grants must be published as an open-access paper. Several of the
old journals will therefore now provide an open-access option, if the authors or the grant-giving body
E.M. Bartels / Best Practice & Research Clinical Rheumatology 27 (2013) 295–306 303
will pay for this. This has suddenly given a much wider free access to electronic journals. A large
selection of open-access papers in health science can be searched in PubMed Central, which is a part
of PubMed. To get to PubMed Central, choose PMC in the box for choosing database at the top of the
front page next to the PubMed logo. When you open PubMed, this box shows ‘PubMed’, which is the
entrance to Medline via PubMed. There is a choice of several databases you can search via the search
host PubMed.
Some open-access journals, which may be too new to fulfil the entrance criteria for bibliographic
databases such as Medline or EMBASE, will be acceptable if they at least are searchable via CrossRef [19]
and have a digital object identifier (DOI) number. This will guarantee peer review and some scientific
standard. All electronic scientific papers should have a DOI registration number, which is presented at
the front page of the paper.
Other search tools
Bibliographic databases are the ‘safe’ places to search because the references included in these
databases are from peer-reviewed journals or, in cases where books or book chapters are included,
from recognised scientific publishers. However, it should be recognised that there are other limits to
this ‘safe strategy’, because a useful part of the available information is published in journals that are
not included in the described databases and these journals may also be peer reviewed and of perfectly
acceptable quality. The major bibliographic databases concentrate on covering publications from
publishers in Europe and the United States, and to some degree Japan, Australia and New Zealand, but
valuable information could still be missed by searching these major bibliographic databases. The other
issue is that many patients, and their relatives, nowget their information about illnesses and treatment
from the Internet. It is therefore also important to keep up with what the layman’s sources of
knowledge are communicating in your field in order to be ahead of your patients and to supplement
your more scientific sources of medical information, before you are asked questions about a certain
condition or therapy or drug treatment.
Search engines
For scientific search via Internet search engines, it is important to understand the difference in
search techniques in these engines from search in bibliographic databases.When using a search engine
such as Google [1] or Ixquick [25], you have to think of the main words and put them in the order of
importance in the search field. Different search engines search in different ways, but most place a high
importance to words appearing in headings and in the first paragraph of the text. The word placed as
number one in a search will be counted as more important than the second or the third. Although it is
possible to perform an ‘advanced search’, this does not make a great difference to the result and you
cannot combine searches in the way described for bibliographic databases.
For scientific questions, it is recommended to search either Google Scholar [26] or Scirus [27], which
aim to provide high-quality answers. Another choice could be a multi-search engine such as Ixquick
[25] (this type of tool harvests results from searches in several search engines and presents them in one
list) due to its large coverage.
Through search engines you will find many homepages on rheumatology subjects, including
everything from stories of individual patients or from a next of kin to the information pages of patient
organisations, homepages of learnt societies and lectures from university courses. You can also find
online encyclopaedias such as Wikipedia [28]. Wikipedia is not peer reviewed but because it is free on
the Internet and anybody can make corrections in the written articles, you can often find solid and up-
to-date information. However, it is also possible for the information to be incomplete or even some-
times incorrect, and you therefore should consider yourself responsible for evaluating any information
that you take from Wikipedia or similar sources.
There are some important sites on the Internet if you need health statistics. The main ones are the
World Health Organization’s (WHO) extended homepage [29] and free pages of useful statistics from
various countries’ National Board of Health. These sources can be considered reliable and need no
evaluation.
E.M. Bartels / Best Practice & Research Clinical Rheumatology 27 (2013) 295–306304
Another important issue, well covered by the Internet, is bioethics. If you need up-to-date advice on
bioethics, you can find the legal documents or interpretations of these under NLM’s Bioethics Infor-
mation Resources [30] or under Council of Bioethics Europe Division [31].
Evaluation of Internet resources where peer-review is not applied
It is necessary to evaluate any found useful resource before the information provided can be taken
into account. The evaluation is simple and builds on common sense:
1. Does the page cover the information you are looking for at an acceptable level?
2. What is the URL (Uniform Resource Locator) address?
3. Is the information given sufficiently complete, correct and precise?
4. Who is the creator of this page?
5. Has the page an acceptable structure?
6. If there are links, what quality are these?
If you find that a page gives you the desired information, you can check whether the address in-
dicates commercial interests or a private person, or whether it takes you to a well-known university,
hospital site or a similar acceptable institution. Although the medical industry has commercial in-
terests, research laboratories may post acceptable information on the Internet. You just have to use
your judgement about the firms’ main marketing objectives when assessing the information given.
If a page gives any information you know is incorrect, you should not use any of the information
given there. You should also require some recognisable expertise of the creator of a page that you will
accept as an information source. Today, it is fairly easy to create a well-structured homepage, so you
may ask yourself if you can trust a pagewhere it is very difficult to find the information. Assessing links
is usually easy at sites of interest for rheumatologists. There will most often be only a couple of ref-
erences to peer-reviewed journals or to health-care institutions, and therewill be very few others. Only
commercial pages will have advertising and similar andmust be looked uponwith some apprehension.
Useful sites are homepages from patients’ organisations who, in many cases, will have professional
staff providing the information on their homepages.
Books and book chapters
During your professional life, it is important to keep up with the development of the subjects in
your areas, and this is often most easily done at any given time by reading the newest textbooks in the
field. Another source of useful books is the many doctoral theses. To follow the books published in your
field over time, you can search any medical research library’s catalogue. You may not find all published
literature of interest, but you will, if you search at yearly intervals, be made aware when it is time to
bring your knowledge up-to-date. If there are a couple of theses and two new textbooks published
during the year, it is time to read up on the subject.
Open access has also reached the book market. You may therefore find several new subject-specific
books in monograph series such as for instance InTech Press [32] or Future Medicine [33], where all
books are edited by specialists and all are free to access. Older handbooks may also appear on the
Internet for free. With these you must be aware that the freely available version often may be an older
edition, or for book chapters, perhaps a pre-proof version with yet-to-be-corrected mistakes.
Searching for a systematic review
The most advanced type of literature search is the search necessary to be able to write a systematic
review. A systematic review has to qualify for being ‘systematic’ and not being an editorial or similar,
whichmeans that it has to cover ‘all’ relevant published studies in the area. The reviewhas to be based on
a protocol, and this protocol has to be published via sites such as PROSPERO [34] prior to starting the
search. The Cochrane Handbook [35] will guide through how such a review – and a possible planned
E.M. Bartels / Best Practice & Research Clinical Rheumatology 27 (2013) 295–306 305
meta-analysis–has to be structured, and the (PRISMA) Preferred Reporting Items for Systematic Reviews
and Meta-Analyses statement [36] must also be considered, but this is outside the scope of this paper.
The main message when preparing a search for a systematic review is:
– Define the aim and objectives of the systematic review clearly, you are not just writing an essay.
– Work hard on the search strategy, it has to be good and comprehensive, as you are bound to follow
it after publishing your protocol.
– Choose all bibliographic databases you can imagine.
– Search meeting abstracts, as well as databases covering protocols for studies such as Clinical
Trials.gov [17].
– Scrutinise reference lists of chosen studies and major reviews.
Summary
Keeping up with the literature is necessary in all medical practice and research to provide up-to-
date diagnosis and treatment. Possessing the latest knowledge is essential if you are to find the best
way of achieving the highest level of competence in your field. Defining a subject for the purpose of
carrying out a literature search helps to clarify how everybody sees the problem, both broadly and in
detail, and this is useful in itself. All soundly based research and innovation projects start with an
information search, supplemented with further searches when new aspects appear.
The virtual library, with its many electronic search tools, may look as if it changes all the time and
may deter a busy professional from carrying out thorough searches. As a general rule, there should be
no worries in terms of searching. The basic design remains the same. You just have to find where
everything is located.Whatever the appearance, the aim seen from the user’s point of view is the same:
“Find me the relevant information in my subject over a defined period of time.” No search tool is so
complicated that a personwith basic information-literacy skills and an education in the health sciences
cannot work out how to use it. However, courses in information literacy are available and important, if
you wish to be highly competent in this area, especially if you intend to carry out a systematic review.
To keep up with the literature, you must build searching into your work routines, and it is good
practice to search at least every 6 months to keep up with your field.
Practice points
– Search the literature on a regular basis, at least every 6 months.
– Define your subject using a search table.
– Choose relevant databases.
– Create a search strategy adjusted to the database(s) you need to search.
– Search each database separately.
– Search the Internet for supplementary information.
– Remember to evaluate all resources acquired via an Internet search.
Conflict of interest statement
The author had no conflicts of interest concerning this work.
Acknowledgement
This work was supported by the OAK Foundation.
References
[1] Google. http://www.google.com [accessed 17.01.2013].
http://www.google.com
E.M. Bartels / Best Practice & Research Clinical Rheumatology 27 (2013) 295–306306
*[2] PubMed. http://www.ncbi.nlm.nih.gov/pubmed [accessed 17.01.2013].
*[3] Grant MJ. How does your searching grow? A survey of search preferences and the use of optimal search strategies in the
identification of qualitative research. Health Information & Libraries Journal 2004;21:21–32.
*[4] Glasziou P, Vandenbroucke J, Chalmers I. Assessing the quality of research. British Medical Journal 2004;328:39–41.
[5] Hacksaw A. A concise guide to clinical trials. Oxford, UK: Wiley-Blackwell; 2009.
*[6] EMBASE. http://www.embase.com [accessed 17.01.2013].
[7] Web of Science. http://thomsonreuters.com/products_services/science/science_products/a-z/web_of_science/ [accessed
17.01.2013].
[8] BiosisPreviews. http://thomsonreuters.com/products_services/science/science_products/a-z/biosis_previews/ [accessed
17.01.2013].
[9] Psycinfo. http://www.apa.org/pubs/databases/psycinfo/index.aspx [accessed 17.01.2013].
[10] The Cochrane Library. http://www.cochranelibrary.com/view/o/index.html [accessed 17.01.2013].
[11] Chemical Abstracts. http://cas.org [accessed 17.01.2013].
[12] TOXNET. http://toxnet.nlm.nih.gov/ [accessed 19.07.2008].
[13] PEDro. http://www.pedro.org.au [accessed 17.01.2013].
[14] CINAHL. http://www.ebscohost.com/academic/cinahl-plus-with-full-text/ [accessed 17.01.2013].
[15] AMED. http://www.library.nhs.uk/help/resource/amed [accessed 17.01.2013].
[16] MANTIS�. http://healthindex.com [accessed 17.01.2013].
*[17] Clinical Trials.gov. http://www.clinicaltrials.gov/ [accessed 17.01.2013].
*[18] Prospero. http://www.crd.york.ac.uk/PROSPERO/ [accessed 17.01.2013].
[19] CrossRef. http://www.crossref.org [accessed 17.01.2013].
[20] Derwent Innovations Index. http://thomsonreuters.com/products_services/legal/legal_products/a-z/derwent_
innovations_index/ [accessed 17.01.2013].
[21] Journal Citation Reports. http://thomsonreuters.com/products_services/science/science_products/a-z/journal_citation_
reports/ [accessed 17.01.2013].
[22] Reference Manager. http://www.refman.com/ [accessed 17.01.2013].
[23] Endnote. http://www.endnote.com/ [accessed 17.01.2013].
[24] Procite. http://www.procite.com/ [accessed 17.01.2013].
[25] Ixquick. http://www.ixquick.com/ [accessed 17.01.2013].
[26] Google Scholar. http://scholar.google.com [accessed 17.01.2013].
*[27] Scirus. http://www.scirus.com/ [accessed 17.01.2013].
[28] Wikipedia. http://www.wikipedia.org/ [accessed 17.01.2013].
*[29] WHO. http://www.who.int/en/ [accessed 17.01.2013].
*[30] NLM’s Bioethics Information Resources. http://www.nlm.nih.gov/bsd/bioethics.html [accessed 17.01.2013].
*[31] Council of Bioethics Europe Division. http://hub.coe.int/what-we-do/health/bioethics [accessed 17.01.2013].
[32] InTech Press. http://www.intechopen.com/ [accessed 17.01.2013].
[33] Future Science Group. http://www.future-science-group.com/ [accessed 17.01.2013].
[34] PROSPERO. http://www.crd.york.ac.uk/PROSPERO/ [accessed 17.01.2013].
[35] The Cochrane Handbook for Systematic Reviews. http://www.cochrane.org/training/cochrane-handbook [accessed 17.01.
2013].
[36] Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gøtzsche PC, Ioannidis JPA, et al. The PRISMA statement for reporting sys-
tematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration.
Journal of Clinical Epidemiology 2009;62(10):e1–34.
http://www.ncbi.nlm.nih.gov/pubmed
http://www.embase.com
http://thomsonreuters.com/products_services/science/science_products/a-z/web_of_science/
http://thomsonreuters.com/products_services/science/science_products/a-z/biosis_previews/
http://www.apa.org/pubs/databases/psycinfo/index.aspx
http://www.cochranelibrary.com/view/o/index.html
http://cas.org
http://toxnet.nlm.nih.gov/
http://www.ebscohost.com/academic/cinahl-plus-with-full-text/
http://www.library.nhs.uk/help/resource/amed
http://healthindex.com
http://www.clinicaltrials.gov/
http://www.crd.york.ac.uk/PROSPERO/
http://www.crossref.org
http://thomsonreuters.com/products_services/legal/legal_products/a-z/derwent_innovations_index/
http://thomsonreuters.com/products_services/legal/legal_products/a-z/derwent_innovations_index/
http://thomsonreuters.com/products_services/science/science_products/a-z/journal_citation_reports/
http://thomsonreuters.com/products_services/science/science_products/a-z/journal_citation_reports/
http://www.refman.com/
http://www.endnote.com/
http://www.procite.com/
http://www.ixquick.com/
http://scholar.google.com
http://www.scirus.com/
http://www.wikipedia.org/
http://www.who.int/en/
http://www.nlm.nih.gov/bsd/bioethics.html
http://hub.coe.int/what-we-do/health/bioethics
http://www.intechopen.com/
http://www.future-science-group.com/
http://www.crd.york.ac.uk/PROSPERO/
http://www.cochrane.org/training/cochrane-handbook
-
How to perform a systematic search
Introduction
Literature search
Scientific papers in peer-reviewed journals
Define your problem
Create a search strategy
Select the right bibliographic databases
Search
Select suitable references from the retrieved ones
Assess whether the search was acceptable
Redesign the search strategy and/or choose other databases/search tools
Keeping up with the peer-reviewed journal articles
Other search tools
Search engines
Evaluation of Internet resources where peer-review is not applied
Books and book chapters
Searching for a systematic review
Summary
Conflict of interest statement
Acknowledgement
References
Original Article
A Guide to Writing a Qualitative Systematic
Review Protocol to Enhance Evidence-Based
Practice in Nursing and Health Care
Ashleigh Butler, MNurs, BNurs, RN • Helen Hall, PhD, MMid, RN, ND •
Beverley Copnell, PhD, RN
Keywords
systematic review
protocol,
qualitative,
meta synthesis,
guidelines
ABSTRACT
Background: The qualitative systematic review is a rapidly developing area of nursing research.
In order to present trustworthy, high-quality recommendations, such reviews should be based on
a review protocol to minimize bias and enhance transparency and reproducibility. Although there
are a number of resources available to guide researchers in developing a quantitative review
protocol, very few resources exist for qualitative reviews.
Aims: To guide researchers through the process of developing a qualitative systematic review
protocol, using an example review question.
Methodology: The key elements required in a systematic review protocol are discussed, with
a focus on application to qualitative reviews: Development of a research question; formulation
of key search terms and strategies; designing a multistage review process; critical appraisal
of qualitative literature; development of data extraction techniques; and data synthesis. The
paper highlights important considerations during the protocol development process, and uses a
previously developed review question as a working example.
Implications for Research: This paper will assist novice researchers in developing a qualitative
systematic review protocol. By providing a worked example of a protocol, the paper encourages
the development of review protocols, enhancing the trustworthiness and value of the completed
qualitative systematic review findings.
Linking Evidence to Action: Qualitative systematic reviews should be based on well planned,
peer reviewed protocols to enhance the trustworthiness of results and thus their usefulness in
clinical practice. Protocols should outline, in detail, the processes which will be used to undertake
the review, including key search terms, inclusion and exclusion criteria, and the methods used for
critical appraisal, data extraction and data analysis to facilitate transparency of the review process.
Additionally, journals should encourage and support the publication of review protocols, and
should require reference to a protocol prior to publication of the review results.
INTRODUCTION
The qualitative systematic review is a newly emerging area of
health care research. Qualitative reviews differ from their quan-
titative counterparts in that they aim to present a comprehen-
sive understanding of participant experiences and perceptions,
rather than assess the effectiveness of an intervention (Stern,
Jordan, & McArthur, 2014). However, their goal remains the
same: to produce high-quality recommendations for patient
care based on a scrupulous review of the best available evi-
dence at the time (Aromataris & Pearson, 2014; Risenberg &
Justice, 2014a). In order to achieve this, the review process
must be well developed and preplanned to reduce researcher
bias and eliminate irrelevant or low quality studies. Typically,
a systematic review is planned by developing a protocol, which
forms the foundation of the entire process.
Developing the protocol before undertaking the review en-
sures that all methodological decisions, from identifying search
terms to data extraction and synthesis processes, are carefully
considered and justified, enhancing the integrity and trustwor-
thiness of the results (Moher et al., 2015; Risenberg & Jus-
tice, 2014a). Additionally, it encourages consistency between
reviewers, reduces the ambiguity of what constitutes “data,ˮ
and ensures the data extraction and synthesis processes are
not arbitrary (Moher et al., 2015).
Although the processes used in quantitative systematic re-
views are well developed, with many guidelines available to
assist novice researchers, there are very few examples of a qual-
itative systematic review protocol available. This paper aims to
guide readers through the process of developing a qualitative
systematic review protocol, using a meta synthesis protocol
Worldviews on Evidence-Based Nursing, 2016; 13:3, 241–249. 241
C© 2016 Sigma Theta Tau International
17416787, 2016, 3, D
ow
nloaded from
https://sigm
apubs.onlinelibrary.w
iley.com
/doi/10.1111/w
vn.12134 by Southern C
ross U
niversity, W
iley O
nline L
ibrary on [17/01/2023]. See the T
erm
s and C
onditions (https://onlinelibrary.w
iley.com
/term
s-and-conditions) on W
iley O
nline L
ibrary for rules of use; O
A
articles are governed by the applicable C
reative C
om
m
ons L
icense
The Qualitative Systematic Review Protocol
Table 1. Example SR: Modified PICO
Population Parents, family, siblings (deceased child)
Context Death of a child in PICU
Outcome Family experiences
entitled “The family experience of the death of a child in the
Pediatric Intensive Care Unit (PICU)” as an example.
Where to Start: Choose a Topic and Aim
Systematic reviews aim to answer a specific question, rather
than provide a simple overview of the evidence (Aromataris
& Pearson, 2014). It is important to have a well-developed
question from the outset, as it will form the basis for the entire
review protocol, guiding the formation of the search strategy,
inclusion criteria, and data extraction (Bettany-Saltikov, 2012).
However, developing a focused, answerable question for a
review can be challenging for novice researchers. There are
numerous frameworks to aid in designing a question for quali-
tative studies: Population, Exposure, Outcomes (PEO); Sample,
Phenomena of Interest, Design, Evaluation, Research type
(SPIDER); and Setting, Perspective, Intervention, Comparison,
Evaluation (SPICE). The acronym PICO, (Population, Interven-
tion, Comparison, Outcome) developed for quantitative review
questions, (Bettany-Saltikov, 2012; Risenberg & Justice, 2014a;
Stern et al., 2014) can also be modified to Population, Context,
Outcome (PCO) or Population, Interest, Context (PICo), to
more appropriately suit a qualitative methodology (Risenberg
& Justice, 2014a; Stern et al., 2014). For example, the question
“What is the experience of the family when a child dies in the
PICU?ˮwas designed using the modified PCO framework (see
Table 1).
The review question is used to design the overall study aim.
The aim should be a clear statement of the intention of the
review, and is typically phrased as a statement. For the above
example, the aim would be stated as follows: “The aim of this
review is to synthesize the best available evidence exploring
the experiences of the death of a child in the PICU, from the
perspective of the child’s family.ˮ
Locating the Literature
Once a focused question has been developed and the aim writ-
ten, the search strategy must be designed. This is one of the
most important parts of the systematic review protocol, because
it outlines a priori the strategies reviewers will use to find, se-
lect, appraise and utilize the data. It is advisable to conduct
a brief search of the literature before planning the review, to
ensure it has not previously been done. Consulting an expert
librarian at this stage may also provide valuable assistance in
identifying keywords and appropriate databases, and develop-
ing a robust search strategy.
Stage One: Developing a Search Strategy
Keywords and search terms. The next step in writing a
qualitative systematic review protocol is developing the key-
words and search terms. The PICO framework can be used
to identify the keywords in the review question. The example
from Table 1 outlines five main keywords: Population-Family,
Context-Death, Context-Child, Context-PICU, and Outcome-
Experiences. Once the keywords are ascertained, a table listing
all of the synonyms can be developed to guide the search, such
as in Table 2. This table of synonyms will then form the ba-
sis of the search strategy. Examining some of the key studies
on the topic can help to uncover commonly used synonyms
and keywords in the literature and help to focus the search
terms. Familiarity with the truncation or wildcard operators for
each database will enable searching for all alternative spellings
or endings to a word, ensuring all possibilities are captured.
Plans to use relevant MeSH headings or similar should also be
documented.
Determining inclusion and exclusion criteria. The inclusion
criteria provide boundaries for the review, defining which stud-
ies will be potentially included, and which ones are irrelevant
to the topic (Stern et al., 2014). Additionally, inclusion criteria
help to mitigate any personal bias of the reviewer; they ensure
that studies are selected only on the basis of predefined, jus-
tified criteria, rather than because they are of interest to the
reviewer, fit into a preconceived framework, or match emerg-
ing findings (Aromataris & Pearson, 2014). The researcher
must negotiate the fine balance between having too narrow
or specific inclusion criteria, where there is a risk of eliminat-
ing relevant papers, and having too few or too broad criteria,
capturing a large number of irrelevant papers. Commonly, in-
clusion criteria consist of aspects such as type of study, type
of data (qualitative or quantitative), phenomena under study,
date of study and age or sex of participants (Stern et al., 2014).
Excluding papers based on language may introduce a language
bias into the review, limiting the transferability of the results;
however, this may be difficult to avoid as translating papers
is often not possible. Whatever the inclusion criteria, they
should be justifiable based on the requirements of the review,
and clearly documented in the protocol. The inclusion criteria
used for the example question are outlined in Table 3, and pro-
vide an illustration of the typical types of justifications used in
a qualitative systematic review protocol.
Designing the search strategy. A systematic review requires
a comprehensive search of multiple databases, using the
same search strategy for each database. It is important that
the protocol clearly outlines the planned search strategy; it
ensures the search is undertaken in exactly the same way
each time, and also allows the search to be replicated by other
researchers in the future with the same results (Aromataris &
Riitano, 2014). Ideally, the search will contain three parts: the
242 Worldviews on Evidence-Based Nursing, 2016; 13:3, 241–249.
C© 2016 Sigma Theta Tau International
17416787, 2016, 3, D
ow
nloaded from
https://sigm
apubs.onlinelibrary.w
iley.com
/doi/10.1111/w
vn.12134 by Southern C
ross U
niversity, W
iley O
nline L
ibrary on [17/01/2023]. See the T
erm
s and C
onditions (https://onlinelibrary.w
iley.com
/term
s-and-conditions) on W
iley O
nline L
ibrary for rules of use; O
A
articles are governed by the applicable C
reative C
om
m
ons L
icense
Original Article
Table 2. Example SR PICO Search Terms
Population Context-death Context-child Context-PICU Outcome
Mother Death Child* PICU Experience
Father Die Daughter P*ediatric ICU Perception
Grandparent Dead Son P*ediatric Intensive Care Perspective
Grandmother Deceased P*ediatric P*ediatric critical care View
Grandfather Dying P*ediatric Intensive therapy unit Need
Sibling Loss
Brother ‘Passed away’
Sister Bereav*
Famil* ‘End of life’
Parent*
Note. The * is used as a truncation indicator.
Table 3. Example SR Question: Inclusion Criteria
Criteria Justification
Conducted between 1990 and 2014 The development of a formal definition of family centred care in 1987 (Shelton, Jeppson, &
Johnson, 1987) led to a change in the way pediatric departments recognize and incorporate
parents and family members into a child’s care delivery. Studies published before 1990 will be
excluded, to ensure the review examines current practice and philosophical standpoints.
Examines family member
experiences, perspectives or
needs as a primary aim
Family experiences and needs surrounding child death in PICU must be a primary aim of each
study. Studies examining family experiences of organ donation, bereavement follow up or
family presence during resuscitation will be excluded, owing to the expansive number of
reviews on each topic.
Relates to the death of a child aged
less than 18 years in a PICU setting
The child’s death must have occurred in a PICU setting. Any studies which focus on the death of a
child in the neonatal ICU (NICU) will be excluded, due to the difference in the philosophy of
care delivery. Studies which examine data from both NICU and PICU settings will be included if
the data from PICU parents is reported separately.
Original qualitative data The review will focus on the experiences, needs or perspectives of family members, which is most
appropriately answered through qualitative research. Any study which utilizes survey data or
statistical reporting of results will be excluded, as will commentaries or discussions on the
subject. Qualitative data from a mixed methods study will be included.
Published in the English Language Due to limited resources, studies published in languages other than English are unable to be
translated and included into the review.
databases, the reference lists and hand searching, and the grey
literature sources.
Identifying the most appropriate databases for the review
topic is crucial. Searching inappropriate databases leads to
inappropriate results, which may impact on the overall review
findings. Librarians are often well positioned to identify
the most useful databases for the area under study. Typical
nursing databases include CINAHL Plus, PubMed, OVID
Medline, and Scopus. These databases, alongside PsychINFO
and EMBASE, were proposed in the example review protocol,
due to their relevance to the review question.
Once the databases are identified, the search strategy should
be developed. The protocol should document who will under-
take the search, how the search terms will be combined and
used, and whether any limits will be applied.
The search strategy used to answer the example question is
outlined in Figure 1, and was based on the recommendations
given by Bettany-Saltikov (2012) and Aromataris and Riitano
Worldviews on Evidence-Based Nursing, 2016; 13:3, 241–249. 243
C© 2016 Sigma Theta Tau International
17416787, 2016, 3, D
ow
nloaded from
https://sigm
apubs.onlinelibrary.w
iley.com
/doi/10.1111/w
vn.12134 by Southern C
ross U
niversity, W
iley O
nline L
ibrary on [17/01/2023]. See the T
erm
s and C
onditions (https://onlinelibrary.w
iley.com
/term
s-and-conditions) on W
iley O
nline L
ibrary for rules of use; O
A
articles are governed by the applicable C
reative C
om
m
ons L
icense
The Qualitative Systematic Review Protocol
Each database will be searched by the research student, in consulta�on with an expert librarian,
based on the following strategy. Each column in Table 2 contains a set of synonyms for the key
search terms. Each term in the column will be entered into the database and will be truncated
where appropriate. All individual searches for that column will be combined using the “OR” Boolean
operator into a single group. Each overall group will then be combined using the “AND” func�on to
produce a final list of cita�ons, which will be saved into Endnote, and screened for duplicates.
Records of all searches in each database will be maintained.
Figure 1. Example SR question: search strategy.
(2014). It provides a systematic way to search each database,
minimizing the impact of the researcher on the outcome of the
search.
It is important that thorough records of all searches are
maintained for future reference, as this provides an audit trail
and enhances trustworthiness of the review findings. Addition-
ally, use of a PRISMA flowchart is recommended as a pictorial
representation of the search process (Moher, Liberati, Tetzlaff,
& Altman, 2009).
Another common search strategy is examination of refer-
ence lists, or hand searching key journals in the area of interest.
The reference lists of relevant papers, especially other literature
reviews on the topic, may identify citations which did not ap-
pear during a database search. The protocol should outline
whether this type of search will be undertaken, and if key jour-
nals will be manually searched for potentially relevant articles,
these should be identified as well.
Lastly, the protocol should also outline whether or not
grey literature will be sourced, and which databases will be
searched. Grey literature is the term given to unpublished
studies, theses, conference proceedings, presentations, gov-
ernment documents, or any other relevant documents that are
not published in journals and will not appear in a database
search (Aromataris & Riitano, 2014; Bellefontaine & Lee,
2014). The inclusion of grey literature helps to reduce publi-
cation bias—the notion that studies with limited, negative, or
neutral outcomes are less likely to be published (Aromataris
& Riitano, 2014; Pappas & Williams, 2011). Grey literature
can be obtained from government websites, Google scholar,
these databases (such as trove.nla.gov.au; worldcat.org), or
grey literature data bases (such as opengrey.eu; greylit.org).
Stage Two: Reviewing the Literature
In order to uncover the studies most relevant to the review, a
multistage process for reviewing and selecting citations must
be developed. The protocol should stipulate how many review-
ers will undertake the review, how many stages there are, and
what each stage will encompass.
How many reviewers? A systematic review requires at least
two independent reviewers (Aromataris & Pearson, 2014;
Porritt, Gomersall, & Lockwood, 2014; Risenberg & Justice,
2014b). Having more than one reviewer at each stage increases
the trustworthiness of the review findings by removing per-
sonal bias from the review process, and minimizing the poten-
tial for error. The protocol should clearly stipulate what each
reviewer’s role will be in each stage of the review, such as in
Figure 2.
How many stages? Typically, the review process is under-
taken in a series of stages, with articles moving through
screening based on title and abstract, and then full text review.
Only those with titles and abstracts that meet inclusion criteria
are retrieved and included for full text review (Aromataris &
Pearson, 2014; Porritt et al., 2014). The protocol should outline
how many review stages each article will undergo, what each
stage involves, and how many reviewers will be included at
each stage. The protocol should also clearly document what
will occur if reviewers disagree. Generally, most reviewers tend
to err on the side of caution and include any citations that are
unclear when screening based on title and abstract, and then
utilize a third reviewer if reviewers disagree during full text
review (Porritt et al., 2014). The protocol should also discuss
what will occur if there is insufficient or unclear information
in an article. Many reviewers will attempt to contact the author
for clarification; however, the protocol should stipulate a
timeframe for reply before the article is excluded on the basis
of insufficient information. An outline of the review process
for the example SR question can be viewed in Figure 3.
The Critical Appraisal
The aim of critical appraisal in a systematic review is to as-
sess the potential studies for rigour, and ensure they are
free from significant methodological issues which may impact
on the quality of the review findings (Bettany-Saltikov, 2012;
Korhonen, Hakulinen-Viitanen, Jylha, & Holopainen, 2013).
Whilst the more traditional qualitative literature provides am-
ple guidance on what constitutes rigor in the various qualita-
tive methodologies (Charmaz, 2006, 2014; Corbin & Stauss,
2008; Holloway & Wheeler, 2010; Lincoln & Guba, 1985; Polit
& Beck, 2010; Sandelowski, 1986; Thomas & Magilvy, 2011;
Whittemore, Chase, & Mandle, 2001), very few of these guide-
lines have been incorporated into critical appraisal tools. Thus,
critical appraisal of qualitative studies remains a contentious is-
sue, with little consensus on what makes a good study, whether
critical appraisal should be undertaken at all, and if so, what
244 Worldviews on Evidence-Based Nursing, 2016; 13:3, 241–249.
C© 2016 Sigma Theta Tau International
17416787, 2016, 3, D
ow
nloaded from
https://sigm
apubs.onlinelibrary.w
iley.com
/doi/10.1111/w
vn.12134 by Southern C
ross U
niversity, W
iley O
nline L
ibrary on [17/01/2023]. See the T
erm
s and C
onditions (https://onlinelibrary.w
iley.com
/term
s-and-conditions) on W
iley O
nline L
ibrary for rules of use; O
A
articles are governed by the applicable C
reative C
om
m
ons L
icense
Original Article
The review process will use four reviewers – one research student, and three supervisors. Ar�cles will
be distributed across the four reviewers in such as way that the research student reviews each cita�on,
and the three supervisors independently review one third of the total cita�ons at each stage.
Figure 2. Example SR question: reviewer roles.
All poten�al ar�cles will undergo a two stage screening process based on the inclusion criteria, and
undertaken by four reviewers, as outlined in Figure 2.
Stage 1: All cita�ons will be screened based on �tle and abstract. Reviewers will meet to discuss
results. All uncertain cita�ons will be included for full text review.
Stage 2: Full text of each included cita�on will be obtained. Each study will be read in full and
assessed for inclusion. Any discrepancies which cannot be resolved through discussion will be sent to
a third reviewer for a decision. Authors will be contacted for missing or incomplete informa�on. If
there is no response within 2 weeks, the ar�cle may be excluded on the basis of missing informa�on.
Figure 3. Example SR question: screening and review.
should be done with the findings (Dixon-Woods et al., 2006;
Downe, 2008; Porritt et al., 2014; Thomas & Harden, 2008;
Toye et al., 2014). To further complicate the issue, there are
a number of different tools available to aid in the critical ap-
praisal of qualitative research, with ongoing debate over which
is most suitable for use in systematic reviews (Dixon-Woods
et al., 2006; Downe, 2008; Toye et al., 2014).
In light of these issues, there are a number of aspects
the protocol must consider and discuss in relation to critical
appraisal:
� Whether critical appraisal will be carried out, and by
whom. The protocol should provide justification if no
appraisal will occur.
� Which appraisal tool will be used, and why. The pro-
tocol should also outline any information or instruc-
tions for reviewers when using the tool.
� Whether the papers will be scored or ranked, and how
this will occur. Generally, most critical appraisal tools
provide a checklist for reviewers, but do not provide
any guidance as to what constitutes a high or low
quality study. The protocol should therefore clearly
document any scoring system which will be imple-
mented, and what will happen if reviewers disagree
during this process.
� How the results of the appraisal will be used. This
decision will depend largely on the purpose of the
review: those which aim to present an overview of
findings may opt to include all studies, whilst those
reviews which aim to inform practice or policy may
omit lower quality studies to enhance trustworthiness.
The protocol should outline the definition of a low- or
high-quality article, and discuss whether any studies
will be excluded and why. It is wise to trial the tool
and scoring system on a small sample of papers from
the initial scoping literature review during this stage
of protocol design, to examine the scores provided
and inform development of an appropriate ranking
system and cut-off point.
For the example systematic review, the researchers took
the view that the use of critical appraisal was necessary to
assess the extent to which the authors’ findings represent the
participants’ experiences or views, and decided that studies
would be excluded based on quality. The Critical Appraisal
Skills Programme (CASP; CASP International Network, 2013)
qualitative checklist was used for critical appraisal, which had
been widely used in recent similar reviews. The tool allows for
appraisal of all types of qualitative data, and the tool contains
only 10 questions, facilitating rapid evaluation; however, it does
not provide a scoring system. Based on previous experience, the
scoring system outlined in Table 4 was designed, and was used
without issue.
Data Extraction
The next step in developing a systematic review protocol is data
extraction. Designing this stage of a qualitative review is often
more difficult than for a quantitative review, because what con-
stitutes data is often unclear. The protocol should clearly outline
what “dataˮ is before outlining how it will be extracted. Com-
monly, qualitative reviews define data as first order constructs
(participants’ quotes), or second order constructs (researcher
interpretation, statements, assumptions and ideas; Toye et al.,
2014). Extracting both forms of data allows the reviewers to
Worldviews on Evidence-Based Nursing, 2016; 13:3, 241–249. 245
C© 2016 Sigma Theta Tau International
17416787, 2016, 3, D
ow
nloaded from
https://sigm
apubs.onlinelibrary.w
iley.com
/doi/10.1111/w
vn.12134 by Southern C
ross U
niversity, W
iley O
nline L
ibrary on [17/01/2023]. See the T
erm
s and C
onditions (https://onlinelibrary.w
iley.com
/term
s-and-conditions) on W
iley O
nline L
ibrary for rules of use; O
A
articles are governed by the applicable C
reative C
om
m
ons L
icense
The Qualitative Systematic Review Protocol
Table 4. Example SR Question: Reviewer Guidelines for Using the CASP Checklist
Item Guidelines
Question 2: Appropriate for qualitative methodology Exclude if inappropriate
Question 3: Research design Yes- Specifically states research design, with justification
Unsure- Outline of research design only
No- Not discussed or inappropriate to research question
Question 5: Data collection Yes- Addresses 4 or more items listed on the CASP checklist
Unsure- Addresses 2–3 items listed on the CASP checklist
No- Addresses less than 2 items
Question 7: Ethical considerations Exclude if unclear or unstated ethical approval
Question 10: Recommendations Yes- The following must be discussed: Contributions to existing knowledge,
identifies areas for future research, makes recommendations based on results
Unsure- only 2 items discussed
No- only 1 item discussed
Scoring system:
Yes: 1 point High-quality paper: Scores 9–10
Unsure: 0.5 points Moderate-quality paper: Scores 7.5-9
No: 0 points Low-quality paper: Less than 7.5
Exclude: Less than 6
view and work with the raw data (quotes) as well as the au-
thors’ interpretations, which we argue helps ensure the review
findings are thoroughly grounded in the original experiences
of the participants.
After the concept of data is well defined, the protocol should
outline how it will be extracted, whether any other informa-
tion will be gathered during the extraction process, and how
many reviewers will be involved, similarly to the example pro-
vided in Figure 4. Generally, data is extracted using a data
extraction tool, which also facilitates the extraction of bibli-
ographic and methodological information about each study,
and ensures that data extraction is consistent amongst all re-
viewers and across all studies (Aromataris & Pearson, 2014;
Bettany-Saltikov, 2012; Risenberg & Justice, 2014b). The ex-
traction tool should be designed by the reviewers based on the
needs of the study, and should be attached as an appendix in
the protocol. Additionally, the protocol should outline whether
the tool will be piloted before use, and how any modifications
will be managed and reported.
Data Synthesis
Developing a plan for data analysis is the final stage of writing a
systematic review protocol. Generally speaking, the aim of data
synthesis or analysis is to assemble the collective findings into
a meaning statement or set of statements which represent and
explain the phenomena under study (Munn, Tufanaru, & Aro-
mataris, 2014). The meta synthesis of qualitative data has long
been a contentious issue. Many scholars argue that by inter-
preting an interpretation, qualitative synthesis risks losing the
essence of the original studies (Korhonen et al., 2013; Thomas
& Harden, 2008; Toye et al., 2014). However, a well-planned
data synthesis process can help to ensure that the review find-
ings remain firmly grounded in the original data, ensuring the
results reflect the original participants’ experiences.
Several methods exist to guide the synthesis and analysis of
qualitative systematic review data, each with its own strengths
and limitations (Dixon-Woods, Agarwal, Jones, Young, &
Sutton, 2005). The chosen method will depend largely on the
type and purpose of the review being undertaken; for example,
a meta synthesis typically requires reviewers reinterpret the
qualitative data into a higher level of abstraction and may use
similar thematic analysis techniques to those used in original
studies, whereas a meta summary may only require content
analysis to provide an aggregation of the overall findings
(Dixon-Woods et al., 2005; Korhonen et al., 2013; Sandelowski,
2006). Whatever the chosen method, each step should be
clearly outlined in the protocol (see Figure 5 for an example),
alongside who will undertake the analysis and whether the
246 Worldviews on Evidence-Based Nursing, 2016; 13:3, 241–249.
C© 2016 Sigma Theta Tau International
17416787, 2016, 3, D
ow
nloaded from
https://sigm
apubs.onlinelibrary.w
iley.com
/doi/10.1111/w
vn.12134 by Southern C
ross U
niversity, W
iley O
nline L
ibrary on [17/01/2023]. See the T
erm
s and C
onditions (https://onlinelibrary.w
iley.com
/term
s-and-conditions) on W
iley O
nline L
ibrary for rules of use; O
A
articles are governed by the applicable C
reative C
om
m
ons L
icense
Original Article
A data extrac�on tool has been developed for the purpose of this review. The tool will be piloted on
2-4 ar�cles prior to use, and will then be modified as required. Data extrac�on will be undertaken by
4 reviewers as per cita�on screening.
The following informa�on will be extracted from each ar�cle: Bibliographic informa�on; study aims;
study design: methodological underpinnings; sample: strategy, size, inclusion/exclusion criteria and
par�cipant characteris�cs; data collec�on methods; data analysis techniques; ethical considera�ons
and issues; results: themes, quotes, author interpreta�ons or explana�ons; strengths and
limita�ons; and reviewer comments.
Figure 4. Example SR question: data extraction.
The extracted data will be analyzed u�lising thema�c analysis techniques, allowing clear
iden�fica�on of themes arising from the data, and facilita�ng higher order abstrac�on and theory
development. The thema�c analysis and meta synthesis processes outlined by J. Thomas and
Harden (2008) are outlined below, and will be used to enhance transparency in the review process.
Data analysis will primarily be undertaken by the student reviewer, with findings con�nually
discussed in team mee�ngs to ensure they appropriately reflect the original data.
Stage 1: Coding text: Free line by line coding of the findings from the primary studies will occur.
Data will be examined for meaning and content during the coding. The codes will then be entered
into a code book. This process will allow the transla�on of codes and concepts between studies.
Stage 2: Developing descrip�ve themes: The codes will then be examined and analysed for their
meanings, and reorganized into related categories. Each category will be analyzed for its proper�es.
Stage 3: Genera�ng analy�cal themes: Each category will then be examined and compared to other
categories, specifically looking for similari�es and differences. Similar categories will be merged into
higher level constructs and then themes, going beyond the findings of the original studies into a
higher order abstrac�on of the phenomena.
Figure 5. Example SR question: data synthesis.
findings will be discussed with other reviewers. This not only
allows the results to be reproduced by other researchers, but
also enhances the transparency and overall trustworthiness of
the review findings.
Publishing the Protocol
Once completed, the protocol should be made available to
other researchers. Most commonly, this is achieved by regis-
tering the protocol with review databases such as the Joanna
Briggs Institute, The Cochrane Collaboration, or PROSPERO,
although there are also a limited number of nursing journals
which will publish a review protocol (Booth et al., 2011; Moher
et al., 2015). Publication encourages transparency of the
review methodology and enables peer review and feedback
prior to the review being undertaken, improving the quality
and trustworthiness of the subsequent review findings and
recommendations (Aromataris & Pearson, 2014; Booth et al.,
2011; Moher et al., 2015). It also ensures that reviewers adhere
to the predefined review processes, as deviation from the
protocol is easily identifiable and requires justification during
publication of the review findings (Booth et al., 2011; Moher
et al., 2015). Additionally, publication of the review protocol
ensures other researchers are aware that the review is being
undertaken, minimizing the amount of time and resources
wasted on duplicate reviews (Booth et al., 2011). Overall,
the publication or registration of review protocols increases
the trustworthiness of the review findings, ensuring that the
recommendations are based on high-quality review of the best
available evidence at the time.
CONCLUSIONS
The qualitative systematic review remains relatively new to the
discipline of nursing, providing greater insight into the needs
Worldviews on Evidence-Based Nursing, 2016; 13:3, 241–249. 247
C© 2016 Sigma Theta Tau International
17416787, 2016, 3, D
ow
nloaded from
https://sigm
apubs.onlinelibrary.w
iley.com
/doi/10.1111/w
vn.12134 by Southern C
ross U
niversity, W
iley O
nline L
ibrary on [17/01/2023]. See the T
erm
s and C
onditions (https://onlinelibrary.w
iley.com
/term
s-and-conditions) on W
iley O
nline L
ibrary for rules of use; O
A
articles are governed by the applicable C
reative C
om
m
ons L
icense
The Qualitative Systematic Review Protocol
of participants than any one single study. The systematic review
should be based on a predeveloped protocol which outlines the
methods and processes which will be used in the review before
it is undertaken, enhancing transparency and trustworthiness
of the review findings. However, given that the techniques
used to design and undertake the qualitative review itself are
still developing, there are very few resources available to guide
nurse researchers through the process of developing a review
protocol. This paper highlights the importance of developing
a systematic review protocol for qualitative reviews, and uses
an example review question to guide researchers through the
protocol development process. By learning to design and im-
plement a systematic review protocol, researchers can help to
ensure that their findings and recommendations are based on
trustworthy, high-quality evidence, improving care delivery to
patients and their families. WVN
LINKING EVIDENCE TO ACTION
� Develop a review protocol prior to undertaking the
review to enhance rigor.
� Utilize a framework (such as PICO) to design an
appropriate and answerable review question.
� Consult an expert librarian for assistance in
developing keywords, identifying appropriate
databases, and designing the search strategy.
� Use two or more reviewers at each stage of the
review to reduce personal bias and minimize po-
tential for error.
� Publish the protocol before undertaking the review
to enhance transparency of the review process and
trustworthiness of the findings.
Author information
Ashleigh Butler, PhD candidate, School of Nursing and Mid-
wifey, Monash University, and Clinical Nurse Specialist, Adult
and Pediatric Intensive Care Unit, Monash Health, Melbourne,
Victoria, Australia; Helen Hall, Lecturer, School of Nursing and
Midwifery, Monash University, Melbourne, Victoria, Australia;
Beverley Copnell, Senior Lecturer, School of Nursing and Mid-
wifery, Monash University, Melbourne, Victoria, Australia
Address correspondence to Ashleigh Butler, C/O Intensive
Care Unit, Monash Medical Centre, 246 Clayton Rd., Clayton,
Victoria, Australia, 3168; aebut2@student.monash.edu
Accepted 14 June 2015
Copyright C© 2016, Sigma Theta Tau International
REFERENCES
Aromataris, E., & Pearson, A. (2014). The systematic review: An
overview. Synthesizing research evidence to inform nursing
practice. American Journal of Nursing, 114(3), 53–58.
Aromataris, E., & Riitano, D. (2014). Constructing a search strategy
and searching the evidence: A guide to the literature search for
systematic review. American Journal of Nursing, 114(5), 49–56.
Bellefontaine, S., & Lee, C. (2014). Between black and white: Exam-
ining grey literature in meta-analyses of psychological research.
Journal of Child & Family Studies, 23(8), 1378–1388.
Bettany-Saltikov, J. (2012). How to do a systematic literature review
in nursing: A step-by-step guide. Berkshire, England: McGraw-Hill
Education.
Booth, A., Clarke, M., Ghersi, D., Moher, D., Petticrew, M., &
Stewart, L. (2011). An international registry of systematic-review
protocols. The Lancet, 377(9760), 108–109.
CASP International Network. (2013). 10 questions to help you
make sense of qualitative research. Retrieved from http://www.
caspinternational.org/mod_product/uploads/CASP%20Quali-
tative%20Research%20Checklist%2031.05.13
Charmaz, K. (2006). Constructing grounded theory: A practical guide
through qualitative analysis. London, England: SAGE Publica-
tions.
Charmaz, K. (2014). Constructing grounded theory (2nd ed.). Lon-
don, England: SAGE Publications.
Corbin, J., & Stauss, A. (2008). Basics of qualitative research: Tech-
niques and procedures for developing grounded theory (3rd ed.). Los
Angeles, CA: SAGE Publications.
Dixon-Woods, M., Agarwal, S., Jones, D., Young, B., & Sutton,
A. (2005). Synthesising qualitative and quantitative evidence: A
review of possible methods. Journal of Health Services Research &
Policy, 10(1), 45–53.
Dixon-Woods, M., Bonas, S., Booth, A., Jones, D. R., Miller, T.,
Sutton, A. J., . . . Young, B. (2006). How can systematic reviews
incorporate qualitative research? A critical perspective. Qualita-
tive Research, 6(1), 27–44.
Downe, S. (2008). Metasynthesis: A guide to knitting smoke.
Evidence-Based Midwifery, 6, 4–8.
Holloway, I., & Wheeler, S. (2010). Qualitative research in nursing
and healthcare (3rd ed.). Oxford, England: Wiley-Blackwell.
Korhonen, A., Hakulinen-Viitanen, T., Jylha, V., & Holopainen,
A. (2013). Meta-synethesis and evidence-based health care—A
method for systematic review. Scandinavian Journal of Caring
Sciences, 27(4), 1027–1034.
Lincoln, Y., & Guba, E. (1985). Naturalistic inquiry. Beverly Hills,
CA: SAGE Publications.
Moher, D., Liberati, A., Tetzlaff, J., & Altman, D. G. (2009). Pre-
ferred reporting items for systematic reviews and meta-analyses:
The PRISMA statement. BMJ, 339(7716), 332–336.
Moher, D., Shamseer, L., Clarke, M., Ghersi, D., Liberati, A., Petti-
crew, M., & . . . Group, P.-P. (2015). Preferred reporting items for
systematic review and meta-analysis protocols (PRISMA-P) 2015
statement. Systematic Reviews, 4(1). doi: 10.1186/2046-4053-4-1
Munn, Z., Tufanaru, C., & Aromataris, E. (2014). Data extraction
and synthesis: The steps following study selection in a systematic
review. American Journal of Nursing, 114(7), 49–54.
Pappas, C., & Williams, I. (2011). Grey literature: Its emerging
importance. Journal of Hospital Librarianship, 11(3), 228–234.
248 Worldviews on Evidence-Based Nursing, 2016; 13:3, 241–249.
C© 2016 Sigma Theta Tau International
17416787, 2016, 3, D
ow
nloaded from
https://sigm
apubs.onlinelibrary.w
iley.com
/doi/10.1111/w
vn.12134 by Southern C
ross U
niversity, W
iley O
nline L
ibrary on [17/01/2023]. See the T
erm
s and C
onditions (https://onlinelibrary.w
iley.com
/term
s-and-conditions) on W
iley O
nline L
ibrary for rules of use; O
A
articles are governed by the applicable C
reative C
om
m
ons L
icense
Original Article
Polit, D., & Beck, C. (2010). Essentials of nursing research: Appraising
evidence for nursing practice (7th ed.). Philadelphia, PA: Lippincott
Williams and Wilkins.
Porritt, K., Gomersall, J., & Lockwood, C. (2014). Study selection
and critical appraisal: The steps following the literature search
in a systematic review. American Journal of Nursing, 114(6), 47–
52.
Risenberg, L., & Justice, E. (2014). Conducting a successful sys-
tematic review of the literature, part 1. Nursing, 44(4), 13–17.
Risenberg, L., & Justice, E. (2014). Conducting a successful sys-
tematic review of the literature, part 2. Nursing, 44(6), 22–26.
Sandelowski, M. (1986). The problem of rigor in qualitative re-
search. Advances in Nursing Science, 8(3), 27–37.
Sandelowski, M. (2006). ‘Meta-jeopardy’: The crisis of representa-
tion in qualitative metasynthesis. Nursing Outlook, 54(1), 10–16.
Shelton, T., Jeppson, E., & Johnson, B. (1987). Family-centered care
for children with special health needs. Washington, D.C.: Associa-
tion for the Care of Children’s Health.
Stern, C., Jordan, Z., & McArthur, A. (2014). Developing the review
question and inclusion criteria: The first steps in conducting a
systematic review. American Journal of Nursing, 114(4), 53–56.
Thomas, E., & Magilvy, J. K. (2011). Qualitative rigor or research
validity in qualitative research. Journal for Specialists in Pediatric
Nursing, 16(2), 151–155.
Thomas, J., & Harden, A. (2008). Methods for the thematic synthe-
sis of qualitative research in systematic reviews. BMC Medical
Research Methodology, 8, 45–45.
Toye, F., Seers, K., Allcock, N., Briggs, M., Carr, E., & Barker, K.
(2014). Meta-ethnography 25 years on: Challenges and insights
for synthesising a large number of qualitative studies. BMC
Medical Research Methodology, 14, 80–80.
Whittemore, R., Chase, S. K., & Mandle, C. L. (2001). Validity in
qualitative research. Qualitative Health Research, 11(4), 522–537.
doi 10.1111/wvn.12134
WVN 2016;13:241–249
Worldviews on Evidence-Based Nursing, 2016; 13:3, 241–249. 249
C© 2016 Sigma Theta Tau International
17416787, 2016, 3, D
ow
nloaded from
https://sigm
apubs.onlinelibrary.w
iley.com
/doi/10.1111/w
vn.12134 by Southern C
ross U
niversity, W
iley O
nline L
ibrary on [17/01/2023]. See the T
erm
s and C
onditions (https://onlinelibrary.w
iley.com
/term
s-and-conditions) on W
iley O
nline L
ibrary for rules of use; O
A
articles are governed by the applicable C
reative C
om
m
ons L
icense
Order an Essay Now & Get These Features For Free:
Turnitin Report
Formatting
Title Page
Citation
Outline
