Assessment: Evidence and Practice: A Review.
Assessment type; Discussion
Word limit/length; 750
MINIMUM OF 7 References; Please use the APA 7th edition referencing style for all assessment tasks in this unit.
Overview
There is often a crucial difference in the evidence of best practice and the care delivered in mental health settings, particularly in relation to change.
This assessment requires you to review and discuss an area where there is a difference and discuss why this is so.
This brief discussion requires you to provide a synopsis of your findings.
.
Learning outcomes
This assessment task is aligned to the following learning outcomes:
1. Critically contrast the findings from the literature with that of current practice and /or policy in clinical settings.
Assessment details
Participate in the following discussion board topic by performing the following steps:
Choose an area where practice may lag behind evidence. For example:
• Identification and treatment of addiction in mental health contexts
• Lived experience of mental ill health in the workforce
• Management of the effects of Psychological Trauma experienced as a child in clinical practice
Briefly identify the evidence and critically debate why there is a practice gap between evidence and clinical practice
Submission format
Please prepare by way of a report. Please be mindful of the wordcount
Assessment: Rubric
1-
CRITERIA;
Presents an overview of the evidence related to the chosen topic from current literature.
MARKS (45%)
Clearly and concisely describes the evidence. Synopsis is very well written. Uses extremely relevant available evidence to support their work.
2-
CRITERIA;
Discussion and analysis of evidence related to the identified area. Reasoned debate around translation of the evidence into the health care setting
MARKS; (45%)
Issue is very clearly described and supported. Excellent in depth discussion on why translation of research into practice may not take place. Utilises relevant available evidence to support their writing extremely well
3-
CRITERIA;
Writing • Spelling and grammar • Referencing
MARKS (10%)
Clear, crisp and coherent style.
Very well organised.
Free of grammar and spelling errors.
All citations follow required style.
SYSTEMATIC REVIEW Open Access
Navigating the sustainability landscape: a
systematic review of sustainability
approaches in healthcare
L. Lennox1,2*, L. Maher3 and J. Reed1
: Improvement initiatives offer a valuable mechanism for delivering and testing innovations in healthcare
settings. Many of these initiatives deliver meaningful and necessary changes to patient care and outcomes. However,
many improvement initiatives fail to sustain to a point where their full benefits can be realised. This has led many
researchers and healthcare practitioners to develop frameworks, models and tools to support and monitor
sustainability. This work aimed to identify what approaches are available to assess and influence sustainability
in healthcare and to describe the different perspectives, applications and constructs within these approaches
to guide their future use.
: A systematic review was carried out following PRISMA guidelines to identify publications that reported
approaches to support or influence sustainability in healthcare. Eligibility criteria were defined through an iterative
process in which two reviewers independently assessed 20% of articles to test the objectivity of the selection criteria.
Data were extracted from the identified articles, and a template analysis was undertaken to identify and assess the
sustainability constructs within each reported approach.
: The search strategy identified 1748 publications with 227 articles retrieved in full text for full documentary
analysis. In total, 62 publications identifying a sustainability approach were included in this review (32 frameworks, 16
models, 8 tools, 4 strategies, 1 checklist and 1 process). Constructs across approaches were compared and 40 individual
constructs for sustainability were found. Comparison across approaches demonstrated consistent constructs were seen
regardless of proposed interventions, setting or level of application with 6 constructs included in 75% of the
approaches. Although similarities were found, no approaches contained the same combination of the constructs nor
did any single approach capture all identified constructs. From these results, a consolidated framework for sustainability
constructs in healthcare was developed.
Conclusions: Choosing a sustainability method can pose a challenge because of the diverse approaches reported in
the literature. This review provides a valuable resource to researchers, healthcare professionals and improvement
practitioners by providing a summary of available sustainability approaches and their characteristics.
Trial registration: This review was registered on the PROSPERO database: CRD42016040081 in June 2016.
Keywords: Sustainability, Method, Tool, Model, Framework, Assessment, Quality improvement
* Correspondence: l.lennox@imperial.ac.uk
1NIHR CLAHRC North West London, 369 Fulham Road, London SW10 9NH,
United Kingdom
2Department of Primary Care and Public Health, Imperial College London,
369 Fulham Road, London, United Kingdom
Full list of author information is available at the end of the article
© The Author(s). 2018 Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0
International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and
reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to
the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver
(http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
Lennox et al. Implementation Science (2018) 13:27
DOI 10.1186/s13012-017-0707-4
http://crossmark.crossref.org/dialog/?doi=10.1186/s13012-017-0707-4&domain=pdf
https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=40081
mailto:l.lennox@imperial.ac.uk
http://creativecommons.org/licenses/by/4.0/
http://creativecommons.org/publicdomain/zero/1.0/
Background
Internationally, there is a need to continually improve
health and care services. To support this, many healthcare
organisations are engaged in a wide range of improvement
initiatives. Despite the significant investment of staff time
and other resources, many promising initiatives fail to
sustain and do not produce long term benefits [1–6]. Sus-
taining worthwhile changes poses a challenge to those
undertaking an improvement initiative. A systematic re-
view of 125 studies of improvements made in healthcare
found that the projects do not maintain all aspects origin-
ally implemented with fewer than half continuing inter-
ventions at high levels of fidelity [1]. Similar results were
found in a review on the continuation of programme ac-
tivities where only 60% of sites reported sustaining at least
one programme component [5].
Initiatives that fail to sustain are extremely wasteful of
human and monetary investments [7, 8]. Large variation
in the practices and care can be seen across similar
services when initiatives which initially demonstrate im-
proved patient outcomes fail to maintain their gains [7, 8].
This has also been shown to be detrimental to improve-
ment efforts in general as staff, patient and public opinion
of improvement initiatives declines and enthusiasm for
engaging in future programmes is lost [9, 10]. In the
current climate of rising demands, shifting priorities and
competition for resources, there is a need to understand
how sustainability of implemented initiatives can be influ-
enced as health planners and other stakeholders want to
ensure the long-term impact of their investments [1, 11].
Despite recognition of this challenge and considerable re-
search conducted in this area, relatively little is known
about how to translate this evidence into action to support
the long-term impact of improvement efforts [12].
Further complicating this research area is the lack of
consensus on how to define sustainability. This has led to
contradictory recommendations for influencing sustain-
ability and debate on what qualifies as a sustained im-
provement [10, 13]. Sustainability is often viewed as an
‘outcome’ where health benefits, activities or workforce
capacity are maintained [8]. Some have cautioned against
this linear perspective on sustainability as it ‘does not take
account of the recursive or reflexive character of sustain-
ability and learning or of the continuous adjustments that
shape the sustainability process’ [13]. More recently, the
ability to adapt and continuously improve has also been
recognised as a potential definition of sustainability [14].
This concept of sustainability as a ‘process’ rather than an
‘outcome’, incorporates concepts of adaptation, learning
and continuous development [15]. This lens allows
sustainability to be viewed as a change process that can be
influenced by individuals throughout initiatives by
continuing to develop and adapt in response to the needs
of the system [15–17]. For the purposes of this work,
‘sustainability’ will refer to the general continuation and
maintenance of a desirable feature of an initiative and its
associated outcomes as well as the process taken to adapt
and develop in response to emerging needs of the system.
This definition as well as any additional domains found
will be explored in the review.
With no clear consensus on how to define or influence
sustainability, many researchers and healthcare practi-
tioners have developed frameworks, models and tools to
support and monitor sustainability in healthcare settings
[12, 18]. With little overarching direction for this area of
research, new definitions, factors and methods for asses-
sing sustainability have been produced by individual
studies [18]. Some work has been undertaken to review
frameworks for sustainability in specific settings and
programmes, but little has been done to comprehen-
sively review available approaches for sustainability
across healthcare settings [15, 18, 19]. It is recognised
that diverse healthcare settings ‘use similar processes to
achieve adoption, implementation, and sustainability’
which indicates general learning and lessons can be
gathered from across settings to inform sustainability re-
search [18]. This provides an opportunity to draw from
the current literature to investigate available sustainabil-
ity approaches and develop a sustainability knowledge
base that is useful beyond specific settings or interven-
tions [18, 20]. This paper offers a review of sustainability
approaches to support healthcare teams and researchers
to understand the different perspectives, applications,
and constructs within approaches to guide their use in
healthcare improvement initiatives.
This review addressed the following research
questions:
1. What approaches have been proposed to influence
or assess sustainability in healthcare?
2. Where have they come from and how have they
been developed?
3. What are their key characteristics?
4. What sustainability constructs are examined in each
approach?
Methods
Search and information sources
A systematic review was undertaken guided by the Pre-
ferred Reporting Items for Systematic Reviews and Meta-
Analysis (PRISMA) reporting standards [21]. The selection
of databases, search terms and search strategy was sup-
ported by a medical librarian to ensure an overall quality
and coverage of the systematic review. The search was car-
ried out on Embase, HMIC Health Management Informa-
tion Consortium, and Ovid MEDLINE in January 2017,
and a follow-up search was conducted prior to submission
in September 2017. Key words included a combination of
Lennox et al. Implementation Science (2018) 13:27 Page 2 of 17
sustainability terms (sustain*, institutionali#ation, routi-
ni#ation, maintenance, integration, normali#ation, embed*)
and method terms (model, framework, tool, plan, checklist,
scale, strategy, theory, conceptuali#ation) along with health
or healthcare. A snowballing approach was also taken; ref-
erences from included papers were analysed and retrieved
if deemed relevant.
Data collection process and study selection
We sought approaches (for the purposes of this work, the
term approaches refers to published models, checklists,
tools, processes, strategies, conceptualisations and frame-
works) that aim to influence and/or assess sustainability
within healthcare settings. The level or type of influence
was not specified but could include assessment, planning,
evaluation, monitoring, prediction or testing. Papers pub-
lished in peer-reviewed journals introducing a tangible
and clear approach for sustainability were included. Papers
published in languages other than English were excluded.
Approaches used within a larger system process or staged
process (for example an implementation model including
sustainability as the final stage) were excluded. Commen-
tary, posters, protocols, conference proceedings, editorials
and perspectives were excluded. Papers only defining or
constructing concepts of sustainability were excluded.
Two authors independently screened the first 20% of the
full-text articles for inclusion. Any differences in selected
articles were discussed, and inclusion and exclusion cri-
teria were refined to reflect these discussions. One author
(LL) then screened the remaining papers for inclusion.
Quality assessment and data extraction strategy
A quality assessment and data extraction form was devel-
oped for identified articles. Existing quality assessments
were explored, but it has been noted that available quality
assessment approaches often fail to consider the rationale
and context of studies [22, 23]. Their use to determine the
inclusion of qualitative studies is often not recommended
as many existing tools do not capture the multiple mean-
ings of “good quality” and “rightness”; therefore, studies
should often not be excluded based on this quality assess-
ment [22, 24]. The available assessments were not sensi-
tive to the aim of our study which was exploratory in
nature. We sought to provide an overview of available
approaches for sustainability and designed our data ex-
traction form to identify and describe the included articles.
The aim of the data extraction was to report descriptions
and study information not to ascertain validity of the ap-
proaches or their constructs. To ensure the studies met
the baseline quality expected, each article was assessed
with the structured data extraction form. Data extraction
included strategy name, purpose of use, healthcare setting,
level of healthcare use, description of use, sustainability
constructs, scoring mechanism, target user, definition of
sustainability, theoretical underpinning, sustainability per-
spective and method development details. One author
(LL) extracted the data from the articles. This information
was then independently checked against the full-text arti-
cles by the second author (LM). Any missing data or
discrepancies were discussed between authors and were
resolved by consensus. Agreement was reached for accur-
acy of all studies.
Data synthesis and presentation
To examine the sustainability constructs within each
method, articles were uploaded to Nvivo 10 software for
analysis. Template analysis was conducted using prede-
fined codes to guide the analysis process [25]. Constructs
within one method (Shediac-Rizkallah and Bone’s concep-
tual sustainability framework) served as the baseline
template for coding sustainability constructs [8]. This
technique allowed each approach’s constructs to be com-
pared and contrasted and additional constructs to be iden-
tified. The preliminary coding structure was iteratively
developed with new constructs integrated and refined as
further sustainability approaches were added to the data-
set. One author conducted the initial coding with input
from other authors on coding structure and construct la-
bels. To assess coding clarity and reliability, a second
coder independently coded 25% of the articles and an
inter-rater reliability score (kappa coefficient) was calcu-
lated. Discrepancies between coders were used to refine
codes and revise the definitions and inclusion criteria for
each of the constructs. Results have been summarised
using ratios and narrative summaries.
Risk of bias in individual studies and across studies
This review aimed to explore the creation and introduc-
tion of sustainability approaches; therefore, results other
than the description of the sustainability method in indi-
vidual studies were not analysed. As this review focused
on published sustainability approaches, publication bias
may have affected the results of this study. Approaches
available in the grey literature were identified but not
included in this review.
Registration
This systematic review was registered on the PROS-
PERO database under the registration number: CRD
42016040081 in June 2016 [26].
Results
The search strategy resulted in 2889 publications from the
databases. Snowballing and electronic citation tracking
identified 121 further papers for potential inclusion. Titles
and abstracts were examined, and 229 articles were re-
trieved in full text for full documentary analysis. In total,
Lennox et al. Implementation Science (2018) 13:27 Page 3 of 17
62 papers which identified sustainability approaches were
identified for inclusion in this review Fig. 1.
Sustainability approaches
The 62 papers identifying sustainability approaches are
outlined in Table 1. Full data extraction details for each
approach are available in Additional file 1. Sustainability
approaches have been consistently developed and
adapted since the late 1980s with an average of two
created every year Fig. 2.
Theoretical perspectives, definitions and development
details
Theoretical perspectives
Exploring the theoretical underpinnings of the ap-
proaches revealed diverse theoretical grounding. Al-
though 37% (23/62) did not have an explicit link to
theory, 15 different theories were identified within the
other approaches. While numerous theories were found,
4 theories were common across multiple sustainability
approaches covering 45% of papers: diffusion of
innovations theory, complexity theory, ecological theory
and open systems theory. Theoretical perspectives guided
how sustainability was defined within approaches and
how it was viewed within healthcare systems. A brief de-
scription of the most common perspectives and their
links to the sustainability approaches are outlined in
Table 2.
Definitions of sustainability
Definitions for sustainability were explicitly stated in 76%
(47/62) of approaches and implicitly deduced from the
remaining 24%. Multiple definitions were found across ap-
proaches, but 5 distinct definitions for sustainability were
identified:
1. Continued programme activities (included in 86%
(53/62) of the approaches)
Fig. 1 PRISMA diagram. Description of search strategy and article retrieval
Lennox et al. Implementation Science (2018) 13:27 Page 4 of 17
Table 1 Papers included in review
Author Year Name Purpose
1. Alexander, J.A.
et al. [62]
2003 The model for community health partnership
sustainability
To provide practical guidelines for partnership sustainability
2. Amaya, A.
et al. [65]
2014 Conceptual framework for sustainability To identify themes and relationships emerging from data to
identify recommendations to inform decision-makers on
priorities
3. Ament, S.
et al. [80]
2014 Strategies to sustain improvements in hospital
practice
To suggest post-implementation strategies which are valuable in
sustaining implementation successes
4. Atun, R.
et al. [53]
2010 A conceptual framework for analysing integration
of health interventions into health systems
To analyse and map the nature
and extent of integration in different settings, along with the
factors that influence the integration process
5. Azeredo, B.T.
et al. [45]
2017 Framework for investigating the sustainability of
ARV provision
To structure data collection and analysis
6. Blackford, J. and
Street, A [69]
2012 The Advance Care Planning-Service Evaluation Tool
(ACP-SET)
To assist community-based palliative care services to establish a
sustainable system-wide model relevant to their local context
7. Blanchet, K. and
Girois, S [57]
2013 The Sustainability Analysis Process (SAP) To conceptualise and measure sustainability of health systems in
low-income countries and fragile states
8. Bray, P. et al. [81] 2009 Sustainability Pyramid Model To propose a series of practice characteristics that constitute
critical elements for QI sustainability activities
9. Brinkerhoff, D.
and Goldsmith,
A. [20]
1992 The analytical framework for Institutional
sustainability
To analyse the generic conditions for sustaining institutions in
general and provide suggested strategies
10. Chambers, D.
et al. [11]
2013 The Dynamic Sustainability Framework To maximise the fit between interventions, practice settings and
the broader ecological system over time
11. Dauphinee, W.
and Reznick, R [63]
2011 Framework for guiding change and managing
and monitoring a successful multicentered
network.
To identify success factors that can facilitate the adoption of a
national simulation network
12. Dominick, G.M.
et al. [82]
2016 ENRICH Sustainability Survey To identify residential children’s homes (RCHs) that sustained
PA-promoting environments.
13. Dorsey, S.
et al. [46]
2014 NINR Logic Model for Center Sustainability To provide guidance for those who wish to develop and sustain
a centre or plan for sustainability
14. Edwards, J. C.
et al. [42]
2007 Catholic Healthcare partners HF-GAP Sustainability
Assessment (AHRQ)
To trigger planning for sustainability early in a project’s design
15. Feldstein, A.C.
and Glasgow,
R.E [83]
2008 Practical, Robust Implementation and Sustainability
Model (PRISM)
To enhance implementation and sustainability and to help
conceptualise, implement and evaluate health care improvement
programmes
16. Finch, T.L.
et al. [84]
2012 Technology Adoption Readiness Scale (TARS) To contribute to the successful normalisation of e-health, either
as a ‘diagnostic’ tool or for evaluation purposes
17. Fleiszer
et al. [58]
2015 Framework for the sustainability of healthcare
innovations
To guide data collection and content analysis
18. Ford, J.H.
et al. [47]
2015 Strategies to Sustain Use of A-CHESS To suggest strategies to be used to sustain the use a mobile app
19. Fox, A.
et al. [79]
2015 The sustainability of innovation theoretical
framework
To guide research, determine variables, influence data analysis
20. Goodman
et al. [85]
1993 Level of Institutionalisation (LoIn) Scale To measure the extent of programme integration into an
organisation
21. Goodman, R.
and Steckler, A [86]
1989 Model for Program Institutionalisation To demonstrate how health promotion programmes may
become institutionalised to guide programme design and
evaluation
22. Gruen, R.L.
et al. [7]
2008 Model of health-programme sustainability To provide a model of health-programme sustainability based on
context and resource availability
23. Hanson, D.
et al. [43]
2005 A systematic ecological framework to design
sustainable interventions
To design sustainable, community-based, safety promotion
interventions
24. Hodge L.M.
and Turn, K [54]
2016 A Conceptual Framework of Supporting Factors To guide and evaluate capacity building in EBP implementation
and sustainment in low-resource community settings
2013 Framework for e-medicine sustainability
Lennox et al. Implementation Science (2018) 13:27 Page 5 of 17
Table 1 Papers included in review (Continued)
Author Year Name Purpose
25. Isabalija, S.R.
et al. [87]
To facilitate the development, implementation, and sustainability
of e-medicine by providing professionals with information on
which to build their sustainability efforts
26. Iwelunmor, J.
et al. [68]
2016 A conceptual framework To bring attention to sustainability as a core component
embedded within the overall life cycle of an intervention that
evolves through time
27. Johnson
et al. [19]
2004 A Sustainability Planning Model To address two sets of sustainability factors known to be
associated with success in sustaining an innovation
28. Knight, T.
et al. [59]
2001 A framework for evaluating the sustainability of
collaborative working
To provide formative evaluation of future collaborative initiatives
and analysis of collaborative working
29. Leffers, J. and
Mitchell, E [88]
2011 Conceptual Framework for Partnership and
Sustainability in Global Health Nursing.
To offer guidance and a framework for partnership and
sustainability for nurses who participate in global efforts
30. Lennox
et al. [56]
2017 The Long Term Success Tool (LTST) To support those implementing improvements reflect on 12 key
factors to identify risks and prompt actions to increase chances
of sustainability over time
31. Luke, D.A. [36] 2014 Program Sustainability Assessment Tool (PSAT) To assess and plan for sustainability risks and develop an action
plans
32. Maher, L.
et al. [61]
2010 NHS III Sustainability Model To predict the likelihood of sustainability and guide teams to
things they could do to increase the chances that changes will
be sustained
33. Mancini, J.A.
and Marek, L.I
[37]
2004 Model of community-based program sustainability/
Program Sustainability Index (PSI)
To evaluate community-based programme sustainability
34. May, C. and
Finch, T [89]
2009 Normalisation Process Theory To explore the social organisation of the work (implementation),
of making practices routine elements of everyday life
(embedding), and of sustaining embedded practices in their
social contexts (integration)
35. May, C.
et al. [51]
2006 Normalisation Process Model To assist in explaining the processes by which complex
interventions become routinely embedded in health care
practice
36. Melnyk, B. and
Fineout-
Overholt,
E [90]
2011 The ARCC (Advancing Research and Clinical
practice through close Collaboration) model
To provide health care systems with a conceptual framework to
guide system-wide implementation and sustainability of EBP for
the purpose of improving quality of care and patient outcomes
37. Nelson, D.E.
et at [39]
2007 The five basic elements of program sustainability To suggest five basic elements of programme sustainability for
tobacco control programmes, to understand the factors
associated with success
38. Nystrom, M.E.
et al. [91]
2014 Strategies to facilitate implementation and
sustainability of large system transformations
To provide an approach to implement and sustain a large
national change programme
39. Okeibunor, J.
et al. [60]
2012 A model for evaluating the sustainability of
community-directed treatment
To provide critical indicators of project performance to evaluate
sustainability
40. Olsen, I. T [92] 1998 Sustainability of health care: A framework for
analysis
To study the sustainability of health services in developing
countries
41. Parand, A [38] 2012 Strategies to sustain Safer Patient Initiative (SPI) To recommend strategies to facilitate the sustainability of a
quality and safety improvement collaborative
42. Persaud, D [52] 2014 The ELIAS (Enhancing Learning, Innovation,
Adaptation, and Sustainability) Performance
Management Framework
To improve the sustainability of healthcare organisations
43. Rasschaert, F. et al.
[93]
2014 Conceptual framework on sustainability of
community-based programmes
To explore the data retrieved and to identify factors influencing
the sustainability
44. Racine, D.P [66] 2006 Model of sustaining innovations in their
effectiveness
To suggest a comprehensive conceptual framework of
programmatic, organisational and environmental factors that
may shape the circumstances for sustaining and replicating
effectiveness
45. Roy, M.
et al. [48]
2016 Framework for Sustained Retention To understand sustained retention, highlight barriers specific to
sustained retention and review interventions addressing
long-term, sustained retention
Lennox et al. Implementation Science (2018) 13:27 Page 6 of 17
e.g. ‘The ability of activities to continue appropriate to
the local context after withdrawal of external funding’
[27].
2. Continued health benefits (included in 44% (27/62))
e.g. ‘Sustainability is the ability to sustain population
health outcomes.’ [28]
3. Capacity built (included in 19% (12/62))
e.g. ‘our conceptualization of sustainability was on the
inter-organizational relationships that might serve as a
basis of the collaborative problem-solving capacity’ [29].
4. Further development (adaptation) (included in 16%
(10/62))
e.g. ‘Adapting successfully to change and providing a
range of valued service delivery opportunities and
practices in an effective and efficient manner’ [30].
Table 1 Papers included in review (Continued)
Author Year Name Purpose
46. Rudd, R. E.
et al. [94]
1999 A five-stage model for sustaining a community
campaign
The five-stage model offers a mechanism for expanding the life
of a campaign
47. Sarriot, E.G.
et al. [31]
2004 Child Survival Sustainability Assessment (CSSA)
framework and process
To provide a process for a participatory sustainability assessment
with communities and local partners
48. Sarriot, E.G.
et al. [28]
2008 The Sustainability Framework To organise thinking about sustainability as well as inform
planning, management, and evaluation of activities in order to
improve and maintain health outcomes at a population level
49. Saunders,
R.P [64]
2012 LEAP Sustainability Assessment To assess sustainability of the Lifestyle Education for Activity
Program (LEAP)
50. Savaya, R [49] 2009 Projected Likelihood of Project’s Continuation To examine projected sustainability and its predictors along a
continuum of forms
51. Schalock, R.
et al. [30]
2016 Sustainability model To consider what factors drive the organisation’s ability to both
adapt successfully to change
52. Scheirer, M.
and Dearing,
J.W [18]
2011 A Generic Conceptual Framework for Sustainability To guide the sustainability research agenda
53. Schell, S.F.
et al. [44]
2013 Capacity for sustainability framework To provide a framework on sustainability capacity, identifying
organisational and contextual characteristics necessary for
successfully sustaining programmes over time
54. Shediac-
Rizkallah, M.C.
& Bone, L.R [8]
1998 Conceptual framework for planning for
sustainability of community based health programs
To conceptualise and measure sustainability and provide
guidelines to facilitate sustainability in community programmes
55. Shigayeva, A.
and Coker, R
[15]
2015 Conceptual framework to support analyses of
sustainability
To support analyses of sustainability of communicable disease
programmes
56. Sivaram, S. and
Celentano, D.D
[27]
2003 Conceptual framework to develop a strategy that
will facilitate sustainability
To develop a strategy that will facilitate sustainability of outreach
worker efforts in AIDS prevention
57. Slaghuis, S.S.
et al. [32]
2011 A framework and a measurement instrument for
sustainability of work practices in long-term care
To analyse sustainability of actual changed work practices and
evaluate improvement projects
58. Song, B.
et al. [50]
2016 The framework for sustainability evaluation of
Community based LTC programmes
To evaluating community-based LTC programmes from the
sustainability perspective
59. Sridharan, S.
et al. [29]
2007 Analysis of strategic plans to assess planning for
sustainability of comprehensive community
initiatives
To assess planning for sustainability
60. Stefanini, A.
and Ruck, N
[41]
1992 Conceptual framework to monitor the performance
of externally-assisted health projects
To monitor a project’s efforts towards sustainability
61. Story et al. [67] 2017 Conceptual framework for institutionalization of
community-focused maternal, newborn & child
health strategies
To encourage collaboration and contribute to programme
planning and policy making for the institutionalisation of
community-focused health strategies
62. Tuyet Hanh, T.T.
et al. [40]
2009 Framework for Evaluating the Sustainability of
Community-based Dengue Control Projects
To provide a framework and tool for assessing sustainability
Lennox et al. Implementation Science (2018) 13:27 Page 7 of 17
http://www.sciencedirect.com/science/article/pii/S0149718906001157
http://www.sciencedirect.com/science/article/pii/S0149718906001157
5. Recovering costs (included in 3% (2/62))
e.g. ‘It is the ability of an organization to produce outputs
of sufficient value so that it acquires enough inputs to
continue production at a steady or growing rate’ [20].
Sustainability approach development
The sustainability approaches were developed through
several techniques often using a mixed-method ap-
proach (e.g. literature review and interviews) (Fig. 3).
Sixty-one percent (38/62) of the development processes
included a literature review or systematic review. This
was followed by 26% (16/62) using ‘professional expert-
ise’ such as an advisory panel and 24% (15/62) using
interviews.
Sustainability method characteristics
Type
The sustainability approaches come in a variety of forms:
frameworks/conceptual frameworks (32), models (16),
Fig. 2 Development of sustainability approaches by year. Graph displays sustainability approach development by year
Table 2 Theoretical perspectives
Diffusion of innovations [70, 95] Complexity theory/complex systems
theory [71]
Ecological theory [72, 96] General systems theory
or open systems theory
[73]
No. of
approaches
drawing on
theory
10 9 5 4
Sustainability
process
Sustainability is viewed as the
final stage of initiative life
cycle [18, 86]
Sustainability is a nonlinear
process where change,
adaptation and uncertainty are
expected [15, 31, 53, 68]
Views sustainability as an
ongoing and dynamic
process that occurs
throughout implementation
[11, 72]
Sustainability is a
process where things
can return to the norm
(‘homeostasis’) or adapt
to the environment to
survive. [85, 92]
Theory
application in
approaches
This perspective explores how
programme benefits and
burden will support or be a
barrier to sustainability [54, 66].
Within approaches using this
perspective, the role of adopters
of the initiatives were seen as
key to success, specifically to
achieve wider reach during
initiatives and maintain activities
after the initiatives come to an
end [27].
This perspective highlights how
the interactions that occur
between an initiative, the
setting, the broader
organisation and the sociocultural
context impact sustained change.
Initiatives were viewed as components
being introduced to complex adaptive
systems that change and adapt in
response to interactions with the
environments, individuals and wider
context [53].
This perspective focuses on
behaviour and how it is
influenced by and influences
individuals and
environments [72]
Approaches adopting this
perspective focused on the need
to find the right fit between
initiatives, contexts
and expectations to inform
the ongoing adaptation of
initiatives to achieve
sustainability [11]
This perspective views
an organisation as an
organism open to the
influence of its
environment with the
need to adapt to survive
in order to achieve
lasting change [73]
Approaches using this
perspective explored
perceived benefits and
burden of an initiatives,
availability of support for
initiatives and leadership
within organisations [54]
Lennox et al. Implementation Science (2018) 13:27 Page 8 of 17
tools (8), guidance strategies (4), checklists (1) and pro-
cesses (1). The highest proportion identified themselves
as frameworks. Our exploration indicates there is very
little consensus between approaches on what constitutes
a ‘framework’, ‘model’ or ‘tool’.
Aim
The highest proportion of approaches, 39% (24/62),
aimed to evaluate sustainability, followed by 23% (14/62)
of the approaches which aimed to support planning for
sustainability. The remaining approaches aimed to pro-
vide guidance and strategies to influence sustainability or
a combination of evaluation, planning and guidance.
When to assess
Two distinct perspectives on when approaches should
be used emerged from this review. The highest propor-
tion, 66% (41/62) of approaches, viewed the sustainabil-
ity as a prospective process to be explored throughout
implementation. Nine approaches viewed sustainability
as a linear process with sustainability being studied
retrospectively after implementation has been ‘com-
pleted’. The remaining 12 approaches specified they
could be used both prospectively and retrospectively,
during implementation or following implementation.
Level of use
The majority, 82% (51/62), of approaches have been de-
signed to examine or influence sustainability at a specific
intervention or programme level (e.g. a single improve-
ment project) [31]. Eleven approaches aimed to examine
sustainability at an organisational or systems level (e.g. a
long-term care organisation) [32].
Settings
Thirty-seven percent (23/62) of the approaches were de-
signed for use in general healthcare settings and did not
specify a specific healthcare setting for use. Public health
settings were specified in 31% (19/62) of the approaches,
followed by community healthcare in 26% of the ap-
proaches (16/62). A smaller number of approaches were
designed for use in acute, 3% (2/62), and e-health
settings, 3% (2/62).
Suggested users
Suggested users were specified in 55% (34/62) of the ap-
proaches (Fig. 4). The majority of these approaches have
been designed for use by multiple groups of professionals
or practitioners (e.g. researchers as well as nurses).
Sustainability constructs
Constructs across approaches were compared and con-
trasted, and 40 individual items for sustainability were
found. The number of constructs examined in each
method ranged from 8 to 31 with an average of 17 con-
structs per method. Additional file 2 provides a descrip-
tion of inclusion, a definition and an example for each of
the 40 constructs. To assess coding clarity and reliability,
an inter-rater reliability score (kappa coefficient) was cal-
culated between two coders using the NVivoPro coding
comparison function [33]. The test showed a high level of
agreement between scorers with an inter-rater reliability
score of 0.94 [34, 35].
A consolidated framework for sustainability constructs
in healthcare is presented in Table 3 and summarises the
frequency of sustainability constructs across the
approaches. The constructs have been organised under
the following six emergent themes: the initiative design
and delivery, negotiating initiative processes, the people
Fig. 3 Sustainability approach development techniques. Development techniques used to create sustainability approaches
Lennox et al. Implementation Science (2018) 13:27 Page 9 of 17
involved, resources, the external environment and the
organisational setting. Comparison across approaches
demonstrated that no two approaches contained the
same combination of the constructs nor did any single
approach capture all 40 constructs. Although variation
was seen, results show that there are consistent con-
structs across approaches regardless of proposed inter-
ventions, settings or application types. Six constructs
were included in over 75% of the approaches: ‘General
resources’ (90%), ‘Demonstrating effectiveness’ (89%),
‘Monitoring progress over time’ (84%), ‘Stakeholder par-
ticipation’ (79%), ‘Integration with existing programs and
policies’ (79%) and ‘Training and capacity building’
(76%).
Diversity in assessment
Although common constructs were found across ap-
proaches, each approach reported diverse means to in-
vestigating and defining individual constructs. As an
exemplar, the top 3 most common constructs are pre-
sented in more detail to highlight how similar constructs
are assessed across different approaches. Inclusion infor-
mation and definitions for all constructs are available in
Additional file 2.
Fig. 4 Suggested users for approaches
Table 3 Consolidated framework for sustainability constructs in healthcare
The initiative
design and
delivery
Negotiating initiative
processes
The people involved Resources The organisational setting The external
environment
• Demonstrating
effectiveness 89%
• Belief in the initiative
63%
• Stakeholder participation
79%
• General
resources 90%
• Integration with existing
programs and policies 79%
• Socioeconomic and
political considerations
63%
• Monitoring
progress over time
84%
• Accountability of roles
and responsibilities 56%
• Leadership and
champions 73%
•
68% • Intervention adaptation
and receptivity 73%
• Awareness and raising
the profile 45%
• Training and
capacity building
76%
• Defining aims and
shared vision 53%
• Relationships and
collaboration and
networks 65%
• Infrastructure
26%
• Organisational values and
culture 71%
• Urgency 5%
• Evidence base for
the initiative 52%
• Incentives 31% • Community participation
56%
• Resource_Staff
26%
• Organisational readiness
and capacity 56%
• Spread to other
organisations 5%
• Expertise 23% • Workload 27% • Staff involvement 42% • Resource_Time
6%
• Support available 40%
• The problem 15% • Complexity 24% • Ownership 26% • Opposition 5%
• Project duration
8%
• Job requirements 19% • Power 18%
• Improvement
methods 6%
• Patient involvement 16%
• Project type 2% • Satisfaction 11%
Lennox et al. Implementation Science (2018) 13:27 Page 10 of 17
� Resources. This construct included a complex
combination of potential resources to consider. Four
key resource types were found: funding,
infrastructure, staff and Time. The majority of the
approaches explicitly stated the need to assess
resources but not all indicated the type of resource.
Many approaches highlighted the importance of the
ability of an initiative to garner and maintain
resources [15, 27, 36–41] through stable sources [19,
36, 39, 42–45]. The ability of an initiative to share
resources with partners and other organisations [41,
46], seek out alternative and supplemental resources
[18, 47–49] and/or uncover multiple funding sources
[8, 36, 49, 50] were also highlighted across some
approaches as important to overall sustainability.
� Demonstrating effectiveness (assessing or measuring
project outcomes and impact). A number of
potential perspectives were taken to assess this
construct. While some approaches chose to look at
overall initiative evaluation or performance [32, 36,
44, 51, 52], others chose to specifically assess either
the ability of the initiative to function as intended
[15, 36, 39] or the ability of the initiative to produce
intended benefits [7, 11, 31, 47, 53–60]. A selection
of approaches took a wider perspective and looked
at whether the initiative benefits were perceived by
staff and other stakeholders as valuable [8, 30, 42,
51, 61-63].
� Monitoring progress over-time (the ability to monitor
the initiative using standardised systems or
mechanisms over-time) appeared in 84% of the
approaches. Approaches to monitoring included
diverse areas to assess including having appropriate
data to document progress [64, 65], having a
management or monitoring system in place [15, 53,
61, 66, 67], and having regular reporting and
feedback mechanisms [46, 47, 52, 54, 68, 69].
Top ten comparisons across approaches
Comparison across level of use
The top ten constructs for examining an organisation or
system’s sustainability versus an intervention or
programme’s sustainability are presented in Table 4.
Regardless of level of use, 5 of the top 10 constructs are
found across both types of approaches. Differences be-
tween these types of approaches demonstrate how the
‘level of use’ of an approach changes the potential con-
structs to be explored (shown in italics). In studying an
organisation or system’s sustainability, there is a greater
focus assessing the readiness and capacity for the initia-
tives and involving stakeholders and community members.
Approaches assessing organisational sustainability were
also much more likely to prioritise defining overall aims
for the programme and garnering belief in initiatives from
stakeholders. Approaches assessing an intervention’s sus-
tainability emphasised the need to consider how an initia-
tive becomes integrated into current programmes and
policies specifically looking what intervention adaption
may be needed. These approaches were also more likely to
assess how training and capacity building were conducted
to ensure staff were able to undertake the initiative tasks.
Comparison of prospective versus retrospective approaches
The top ten constructs for examining sustainability
throughout an initiative (prospective assessment) versus
after implementation (retrospective assessment) are
presented in Table 5. Several key differences are observed.
Prospective approaches are used for a combination of plan-
ning, guidance and evaluation. Prospective approaches
show a greater emphasis on building relationships and get-
ting stakeholder buy-in throughout an initiative. These ap-
proaches also highlighted the role of initiative adaptation
to ensure initiatives align with stakeholder and setting
needs. Retrospective approaches were more often designed
for evaluation purposes emphasising the need for a shared
vision and accountability to deliver the initiative. These ap-
proaches were more likely to specifically examine funding
for the initiative and highlight the need to have a defined
aim to show evidence for sustainability of an initiative once
it has been ‘completed’. These differences highlight how
retrospective approaches tend to focus on delivery and
evidence for continuation of initiatives while prospective ap-
proaches focus on building an initiative into an organisa-
tion, getting people on board and garnering networks that
may help along the way.
This review aimed to identify available approaches which
assess or influence sustainability in healthcare and ex-
plore what sustainability constructs were examined in
each to inform their future use in practice. This review
found that a substantial number of approaches exist with
62 approaches identified and included in this review.
Approach characteristics were wide-ranging with diverse
settings, interventions and designs. Each provided a
unique perspective on sustainability with no two being
exactly alike.
The reviewed sustainability approaches made connec-
tions to many different theoretical perspectives which
highlighted the complexity of measuring and planning
for sustainable initiatives. Four theoretical perspectives
(diffusion of innovations theory, complexity theory, eco-
logical theories and open systems theory) were most
common and revealed two distinct positions guiding the
use of sustainability approaches. The first views sustain-
ability as a linear process following implementation. In
this approach, sustainability is an end goal, a state to be
Lennox et al. Implementation Science (2018) 13:27 Page 11 of 17
reached or level of achievement [70]. The second views
sustainability concurrent process alongside implementa-
tion, where sustainability is a process to be influenced
and adapted to impact initiative longevity [71-73]. Value
is seen in both views, but depending on what theoretical
perspective is taken, planning, measurement and moni-
toring is significantly different [8, 13, 61, 74, 75]. Despite
previous work finding that ‘most frameworks proposed
tend to be deterministic in nature where sustainability is
viewed as an end goal’, we found that 66% of approaches
we reviewed saw sustainability as a process rather than
an end state [15]. The choice to evaluate, monitor or
plan for sustainability overtime rather than after imple-
mentation may indicate a shift in perspectives from sus-
tainability as an outcome to sustainability as an ongoing
process. As this perspective gains popularity, some have
cautioned that while it may be valuable to assess sustain-
ability throughout initiatives, data collection past the im-
plementation stage is still required to assess the
continuation of initiative activities or outcomes and de-
termine whether sustainability is actually achieved [18].
This highlights the need for the purpose of use to be
clear before an approach is applied. While some ap-
proaches explicitly aim to sustain outcomes, others are
meant to influence and promote action overtime. There-
fore, the aims and potential results from approaches
should be understood to ensure people are able to realis-
tically assess the outcomes they desire.
Results have demonstrated that sustainability is most
often defined and assessed as the maintenance of
programme activities. Although multiple definitions were
found (continuation of the health benefits from an initia-
tive, capacity built in the workforce or community, fur-
ther development or adaptation and the ability to
recover costs), there was a clear dependence on this one
measure which has been previously observed in the lit-
erature [76]. It is important to note that while measuring
continuation of programme activities is important to
assessing sustainability, relying solely on this measure
may risk other key sustainability variables being missed
[18, 76]. For example, it may result in the continuation
of ineffective or undesirable practices if health benefits
are not taken into account. This was observed in the
Drug Assistance Resistance Education programme in
America which continued to be implemented in schools
despite studies showing that it had little effect on pre-
vention or reduction of drug use by students [77]. Using
continuation of programme activities as the sole
Table 4 Comparison across level of use (difference shown in italics)
Organisational focus (11 approaches) Percent Intervention focus (51 approaches) Percent
1. Demonstrating effectiveness 100 1. Resources_General 90
2. Resources_General 91 2. Demonstrating effectiveness 86
3. Monitoring progress over time 91 3. Monitoring progress over time 82
4. Organisational readiness and capacity 82 4. Integration with existing programs and policies 82
5. Belief in the initiative 73 5. Training and capacity building 76
6. Organisational values and culture 73 6. Stakeholder participation 76
7. Community participation 73 7. Intervention adaptation and receptivity 75
8. Leadership and champions 73 8. Leadership and champions 73
9. Stakeholder participation 73 9. Organisational values and culture 71
10. Defining aims and shared vision 64 10. Funding 69
Table 5 Comparison of when to assess (differences shown in italics)
Retrospective assessment (9 approaches) Percent Prospective assessment (41 approaches) Percent
1. Demonstrating effectiveness 100 1. Resources_general 93
2. Resources_general 89 2. Demonstrating effectiveness 85
3. Leadership and champions 89 3. Monitoring progress over time 83
4. Accountability of roles and responsibilities 78 4. Stakeholder participation 83
5. Belief in the initiative 67 5. Integration with existing programs and policies 81
6. Defining aims and shared vision 67 6. Training and capacity building 78
7. Funding 67 7. Intervention adaptation and receptivity 73
8. Monitoring progress over time 67 8. Leadership and champions 73
9. Training and capacity building 67 9. Belief in the initiative 68
10. Integration with existing programs and policies 67 10. Relationships and collaboration and networks 68
Lennox et al. Implementation Science (2018) 13:27 Page 12 of 17
measure of sustainability also risks initiative being un-
fairly judged as failing to sustain if activities are adapted.
If the definition is broadened, adaptation could also sig-
nify sustained improvement, especially if the adaptations
contributed further to health benefits or cost recovery.
These examples highlight the need for careful consider-
ation of what will be sustained and what evidence there
is for sustainability to occur [66]. All definitions identi-
fied in the review represent interrelated facets of what
sustainability means in practice; therefore, those working
in this field should explore the breadth of available sus-
tainability domains in order to accurately represent the
sustainability process and account for its full complexity
and possible outcomes [7].
Our comparison across approaches demonstrated con-
sistent constructs were seen regardless of proposed inter-
ventions, setting or level of application. Within the six
constructs included in the majority of approaches, diverse
views and different assessment mechanisms were taken,
highlighting the complexity within each construct. This
demonstrates the need for careful planning and consider-
ation of how each construct is articulated and assessed
given the specific outcomes of interest desired. Interest-
ingly, no approaches contained the same combination of
the constructs nor did any single method capture all iden-
tified constructs. Given homogeneity of the individual
constructs found, we believe there is value in having an
overarching resource and summary, indicating the breadth
of possible sustainability constructs to consider for sus-
tainability in healthcare settings. The consolidated frame-
work for sustainability constructs in healthcare (Table 3)
provides a knowledge base for those who may wish to re-
view proposed sustainability constructs and draw on the
substantial work and research already conducted in this
area.
The framework can also help those considering creating
a sustainability method in their own setting. While there
are benefits of approaches created for specific settings,
there is also a risk in continually creating ‘new’ approaches
with similar constructs divided by semantics and personal
interpretations of the literature [18, 20]. Those consider-
ing creating a sustainability approach should consider the
information presented here and the available approaches
for use before ‘recreating the wheel’ as continuous produc-
tion may lead to further division and confusion in the lit-
erature and ultimately result in fewer robust studies on
the use of available sustainability approaches being pub-
lished [18]. The number of sustainability approaches may
grow with necessary alterations to design and further
development, but there is a need for future authors to
describe how new approaches fit within the findings
presented here. Authors should explicitly state how
approaches have been created (particularly drawing on
previous approaches which have informed the
development) and highlight if they are transferable to
other settings and if there are any specific benefits or bar-
riers to their use.
Strengths and limitations
This is the first review to consolidate available ap-
proaches for sustainability across diverse healthcare set-
tings. We believe this work represents a significant
contribution to the field in organising and describing
sustainability approaches which have until now remained
isolated across healthcare fields and disciplines [18]. This
review provides not only a resource for identifying avail-
able sustainability approaches but also outlines the aims,
applications and constructs in each approach so readers
can determine if one may be fit for their setting. This
work has demonstrated that although many approaches
were developed within specific interventions and set-
tings, similar constructs for sustainability were found in-
dicating general learning can be gathered from across
settings to inform sustainability processes and research.
Additionally, this paper provides a consolidated sum-
mary of all constructs deemed to be important across
approaches to serve as a sustainability knowledge base
that is useful beyond specific settings or interventions.
To aid readers in navigating the data extracted from
each approach, we propose a list of questions to guide
their decision-making process (Table 6). Readers can re-
spond to these questions and use their responses along
with full method details in Additional file 1 to establish
if an available method will suit their purposes.
The use of one author to conduct of the majority of
screening, data extraction and coding is also a limitation
of this work. Although double data extraction is recom-
mended in most systematic reviews, it is also recognised
that this is often not possible in many cases due to time
and resources constraints [78]. This may have resulted
in bias in inclusion or exclusion or resulted in missing
or erroneous information being collected. To address
this limitation, we involved multiple authors where pos-
sible in selection of the studies (20% screened by a sec-
ond author) and coding of constructs (25% of studies).
Table 6 Questions for consideration
Navigating available sustainability approaches—questions for
consideration
1. How do you wish to view sustainability? (a process or an end goal)
2. What is your aim? (evaluation, planning, guidance)
2. What does sustainability mean to you? (continuation of the health
benefits, continuation of activities, capacity built, further
development and/or cost recovery)
3. Where do you wish to use the sustainability approach? (specific
intervention or organisation)
4. Who will use the approach? (researcher, practitioner, managers etc.)
5. Does an existing approach meet your needs?
6. If not, what needs to change or be adapted and why?
Lennox et al. Implementation Science (2018) 13:27 Page 13 of 17
Data extraction was also checked against full-text arti-
cles for all included papers.
Another limitation of this work is the disproportionate
number of frameworks from the community health and
public health settings. These areas tend to dominate this
area of research so further work may be needed to ex-
plore sustainability in other acute and chronic care set-
tings [79].
Another key limitation of this work is that we did not use
an existing quality assessment tool and cannot attribute
value or accuracy of constructs from each approach. While
the quality criteria set out in our data extraction form
allowed us to ensure each paper had a minimum level of
data to adequately describe the approach, it did not assess
quality of the approaches themselves. We extracted infor-
mation on each of the approaches which others may wish
to use to attribute validity to findings. Details, particularly
those around sustainability approach development, may be
used by readers to assess whether they believe the approach
has enough merit to be used in their site. It is important to
note that many approaches (24%) were informed by profes-
sional expertise, a technique that may be difficult to assess
for quality but appears to be very significant in the creation
of sustainability approaches.
We reported which constructs were deemed to be im-
portant to assess, but this does not indicate that these are
the ‘right’ constructs or that they will lead to sustainability.
Although our assessment of frequency indicated some con-
sensus across approaches, with six constructs included in
over 75% of approaches, this does not tell us that assessing
these constructs will achieve sustainability in practice or
that they are correct or comprehensive. In order to under-
stand the validity of these findings, the approaches must be
applied and assessed in practice. Future work will explore if
and how these approaches have been applied to ascertain if
their constructs accurately represent sustainability in spe-
cific settings and if they fulfil their stated aims.
Future work
Many approaches presented in this review recommend
that they be used and evaluated further within other
healthcare initiatives and settings to explore applicability
and further development needed [11, 36, 56, 79]. Future
work in this field should now focus on applying the avail-
able approaches in practice to understand the application
processes and assess the overall impact of their use [18].
Conclusion
Sustainability of improvements has been recognised as a
challenge for some time, and while there is diversity in the
literature on how it is defined and how it can be influ-
enced, there is one clear and compelling message: sustain-
ability of initiatives requires thoughtful planning and
attention. If we do not address it appropriately, we
continue to risk wasting valuable resources and losing sig-
nificant progress and patient outcome improvements.
Choosing a sustainability approach to support this process
can pose a challenge to those looking to influence sustain-
ability because of the diverse approaches reported in the
literature. Understanding the purpose, perspectives and
constructs within each will aid potential users to make the
most of approach choice and application.
Additional file 1: Data extraction form. Full data extraction details for
each method. (XLSX 30 kb)
Additional file 2: Definition and description of sustainability constructs.
Table provides definitions, descriptions and examples for each of the 40
sustainability constructs found constructs. (PDF 446 kb)
Acknowledgements
The authors would like to thank Cathal Doyle for his contribution to an initial
literature review which supported the background work for this review.
Funding
This work was funded by the National Institute for Health Research in the
Collaboration for Leadership in Applied Health Research and Care for
Northwest London (CLAHRC) programme. The funding agency had no part
in the design, analysis or writing of the manuscript.
All data generated or analysed during this study are included in this
published article (and its supplementary information files).
This research was funded by the National Institute for Health Research (NIHR)
Collaborations for Leadership in Applied Health Research and Care
Northwest London (NIHR CLAHRC Northwest). The views expressed in this
article are those of the author(s) and not necessarily those of the NHS, the
NIHR, or the Department of Health.
LL conceived of the study and was responsible for the design and search
strategy. LL conducted the search. LL and LM conducted the data analysis
and produced the tables and graphs. JR provided input into the data
analysis and interpretation. The initial draft of the manuscript was prepared
by LL then circulated among all authors for comments and revision. All
authors read and approved the final manuscript.
Not applicable
Not applicable
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in
published maps and institutional affiliations.
1NIHR CLAHRC North West London, 369 Fulham Road, London SW10 9NH,
United Kingdom. 2Department of Primary Care and Public Health, Imperial
College London, 369 Fulham Road, London, United Kingdom. 3Ko Awatea I
Health System Innovation and Improvement, Middlemore Hospital, 100
Hospital Road, Otahuhu, New Zealand.
Lennox et al. Implementation Science (2018) 13:27 Page 14 of 17
dx.doi.org/10.1186/s13012-017-0707-4
dx.doi.org/10.1186/s13012-017-0707-4
Received: 5 October 2017 Accepted: 29 December 2017
1. Stirman SW, Kimberly J, Cook N, Calloway A, Castro F, Charns M. The
sustainability of new programs and innovations: a review of the empirical
literature and recommendations for future research. Implement Sci BioMed
Central Ltd. 2012;7:17.
2. Williams L, Daggett V, Slaven JE, Yu Z, Sager D, Myers J, et al. A cluster-
randomised quality improvement study to improve two inpatient stroke
quality indicators. BMJ Qual Saf. 2015; bmjqs-2015-004188
3. Bowman CC, Sobo EJ, Asch SM, Gifford AL. Measuring persistence of
implementation: QUERI series. Implement Sci. 2008;3
4. Virani T, Lemieux-charles L, Davis DA, Berta W. Sustaining change: once
evidence-based practices are transferred, what then? Healthc Q. 2009;
12:89–96.
5. Scheirer MA. Is sustainability possible? A review and commentary on
empirical studies of program sustainability. Am J Eval. 2005;26:320–47.
6. Ham C. Evaluation of the projects within the National Booking Program.
Birmingham; 2004.
7. Gruen RL, Elliott JH, Nolan ML, Lawton PD, Parkhill A, McLaren CJ, et al.
Sustainability science: an integrated approach for health-programme
planning. Lancet. 2008;372:1579–89. Elsevier Ltd
8. Shediac-Rizkallah MC, Bone LR. Planning for the sustainability of
community-based health programs: conceptual frameworks and future
directions for research, practice and policy. Health Educ Res. 1998;13:87–108.
9. Hovlid E, Bukve O, Haug K, Aslaksen AB, von Plessen C. Sustainability of
healthcare improvement: what can we learn from learning theory? BMC
Health Serv Res. 2012;12:235.
10. Martin GP, Weaver S, Currie G, Finn R, Mcdonald R. Innovation sustainability
in challenging health-care contexts: embedding clinically led change in
routine practice. Heal Serv Manage Res. 2012;25:190–9.
11. Chambers D a, Glasgow RE, Stange KC. The dynamic sustainability
framework: addressing the paradox of sustainment amid ongoing change.
Implementation Sci. 2013;8:117.
12. Greenhalgh T, Robert G, Bate P, Kyriakidou O, Macfarlane F, Peacock R. How
to spread good ideas: a systematic review of the literature on diffusion,
dissemination and sustainability of innovations in health service delivery
and organisation. 2004.
13. Pluye P, Potvin L, Denis J-L. Making public health programs last:
conceptualizing sustainability. Eval Program Plann. 2004;27:121–33.
14. Moore JE, Mascarenhas A, Bain J, Straus SE. Developing a comprehensive
definition of sustainability. Implement Sci. 2017;12:110.
15. Shigayeva A, Coker RJ. Communicable disease control programmes and
health systems: an analytical approach to sustainability. Health Policy Plan.
2015;30:368–85.
16. Folke C, Carpenter S, Elmqvist T, Gunderson L, Holling C, Walker B.
Resilience and sustainable development: building adaptive capacity in a
world of transformations. Ambio. 2002;31:437–40.
17. Fiksel J. Designing resilient, sustainable systems. Environ Sci Technol. 2003;
37:5330–9.
18. Scheirer MA, Dearing JW. An agenda for research on the sustainability of
public health programs. Am J Public Health. 2011;101:2059–67.
19. Johnson K, Hays C, Center H, Daley C. Building capacity and sustainable
prevention innovations: a sustainability planning model. Eval Program
Plann. 2004;27:135–49.
20. Brinkerhoff DW, Goldsmith AA. Promoting the sustainability of development
institutions: a framework for strategy. World Dev. 1992;20:369–83.
21. PRISMA. PRISMA Statement [Internet]. Transparent Report. Syst. Rev. meta-
analyses. 2015 [cited 2016 Jan 22]. Available from: http://www.prisma-
statement.org/PRISMAStatement/Default.aspx.
22. Barbour RS. Checklists for improving rigour in qualitative research: a case of
the tail wagging the dog? BMJ Br Med J. 2001;322:1115–7.
23. Li LC, Grimshaw JM, Nielsen C, Judd M, Coyte PC, Graham ID. Use of
communities of practice in business and health care sectors: a systematic
review. Implement Sci. 2009;4:27.
24. Walsh D, Downe S. Appraising the quality of qualitative research. Midwifery.
2006;22:108–19.
25. King N. Essential guide to qualitative methods in organisational research.
Cassel C, Symon G, editors. London, UK.: Sage Publications; 2004.
26. Lennox L, Maher L, Bell D, Reed J. The sustainability landscape: a systematic
review of sustainability strategies and their impact in healthcare settings.
PROSPERO Int Prospect Regist Syst Rev. 2016;CRD4201604:1–4.
27. Sivaram S, Celentano DD. Training outreach workers for AIDS prevention in
rural India: is it sustainable? Health Policy Plan. 2003;18:411–20.
28. Sarriot E, Yurkavitch J, Ryan L, The Sustained Health, Outcomes (SHOUT)
Group. Taking the long view: a practical guide to sustainability planning
and measurement in community-oriented health programming. Manual.
Calverton, MD.; 2008.
29. Sridharan S, Go S, Zinzow H, Gray A, Gutierrez BM. Analysis of strategic plans
to assess planning for sustainability of comprehensive community
initiatives. Eval Program Plann. 2007;30:105–13.
30. Schalock RL, Verdugo M, Lee T. A systematic approach to an organization’s
sustainability. Eval Program Plann. 2016;56:56–63.
31. Sarriot EG, Winch PJ, Ryan LJ, Bowie J, Kouletio M, Swedberg E, et al. A
methodological approach and framework for sustainability assessment in
NGO-implemented primary health care programs. Int J Health Plann Manag.
2004:23–41.
32. Slaghuis SS, Strating MMH, Bal RA, Nieboer AP. A framework and a
measurement instrument for sustainability of work practices in long-term
care. BMC Health Serv Res. 2011;11:314. BioMed Central Ltd
33. Nvivo. Run a coding comparison query [Internet]. Explor. your coding using
queries. 2017. p. 1. Available from: http://help-nv11.qsrinternational.com/
desktop/procedures/run_a_coding_comparison_query.htm.
34. Viera AJ, Garrett JM. Understanding interobserver agreement: Fam. Med.
2005;37:360–3.
35. Nvivo. Run a coding comparison query. Explor. your coding using queries.
2017. p. 1.
36. Luke D a, Calhoun A, Robichaux CB, Elliott MB, Moreland-Russell S. The
program sustainability assessment tool: a new instrument for public health
programs. Prev Chronic Dis. 2014;11:130184.
37. Mancini JA, Marek LI. Sustaining community-based programs for families:
conceptualization and measurement. Fam Relat. 2004;53:339–47.
38. Parand A, Benn J, Burnett S, Pinto A, Vincent C. Strategies for sustaining a
quality improvement collaborative and its patient safety gains. Int J Qual
Heal. Care. (C) International Society for Quality in Health Care and Oxford
University Press 2012. Published by Oxford University Press. All rights
reserved.: Department of Surgery and Cancer, Imperial College London, Rm
503, 5th Floor Wright Fleming Building, St Mary’s Campus, Norfolk Place,
London W2 1PG, UK; 2012. p. 380–90.
39. Nelson DE, Reynolds JH, Luke D a, Mueller NB, Eischen MH, Jordan J, et al.
Successfully maintaining program funding during trying times: lessons from
tobacco control programs in five states. J Public Health Manag Pract. 2007;
13:612–20.
40. Tuyet Hanh TT, Hill PS, Kay BH, Tran MQ. Development of a framework for
evaluating the sustainability of community-based dengue control projects.
Am J Trop Med Hyg. 2009;80:312–8.
41. Stefanini A, Ruck N. Managing externally-assisted health projects for
sustainability in developing countries. Int J Health Plan Manag. 1992;7:
199–210.
42. Edwards JC, Feldman PH, Sangl J, Polakoff D, Stern G, Casey D. Sustainability
of partnership projects: a conceptual framework and checklist. Jt Comm J
Qual Patient Saf. 2007;33:37–47.
43. Hanson D, Hanson J, Vardon P, McFarlane K, Lloyd J, Muller R, et al. The
injury iceberg: an ecological approach to planning sustainable community
safety interventions. Health Promot J Austr. 2005;16:5–10.
44. Schell SF, Luke D a, Schooley MW, Elliott MB, Herbers SH, Mueller NB, et al.
Public health program capacity for sustainability: a new framework.
Implementation Sci. 2013;8:15.
45. Azeredo TB, Oliveira MA, Santos-Pinto CDB, Miranda ES, Osorio-de-
Castro CGS. Sustainability of ARV provision in developing countries:
challenging a framework based on program history. Cien Saude Colet.
2017;22:2581–94.
46. Dorsey SG, Schiffman R, Redeker NS, Heitkemper M, McCloskey DJ, Weglicki
LS, et al. National Institute of Nursing Research Centers of Excellence: a logic
model for sustainability, leveraging resources, and collaboration to
accelerate cross-disciplinary science. Nurs Outlook U S. 2014;62:384–93.
47. Ford JH, Alagoz E, Dinauer S, Johnson KA, Pe-Romashko K, Gustafson DH.
Successful organizational strategies to sustain use of A-CHESS: a mobile
intervention for individuals with alcohol use disorders. J Med Internet Res.
2015;17:e201.
Lennox et al. Implementation Science (2018) 13:27 Page 15 of 17
48. Roy M, Czaicki N, Holmes C, Chavan S, Tsitsi A, Odeny T, et al.
Understanding sustained retention in HIV/AIDS care and treatment: a
synthetic review. Curr HIV/AIDS Rep. 2016;13:177–85.
49. Savaya R, Elsworth G, Rogers P. Projected sustainability of innovative social
programs. Eval Rev. 2009;33:189–205.
50. Song B, Sun Q, Li Y, Que C. Evaluating the sustainability of community-
based long-term care programmes: a hybrid multi-criteria decision making
approach. Sustainability. 2016;8:1–19.
51. May C. A rational model for assessing and evaluating complex interventions
in health care. BMC Health Serv Res. 2006;6:11.
52. Persaud D. Enhancing learning, innovation, adaptation, and sustainability in
health care organizations: The ELIAS Performance Management Framework.
Heal. Care Manag. (C) 2014 Wolters Kluwer Health | Lippincott Williams &
Wilkins: Author Affiliation: School of Health Administration, Dalhousie
University, Halifax, Canada.; 2014. p. 183–204.
53. Atun R, de Jongh T, Secci F, Ohiri K, Adeyi O. Integration of targeted health
interventions into health systems: a conceptual framework for analysis. Heal.
Policy Plan. (C) Copyright Oxford University Press 2010.: (1)Professor of
International Health Management, Imperial College, London, UK,
(2)Researcher, Centre for Health Management, Imperial College London, UK,
(3)Doctoral Researcher, Centre for Health Management, Imperial College
London, UK, (4)Health Specialis; 2010. p. 104–11.
54. Hodge L, Turner KMT. Sustained implementation of evidence-based
programs in disadvantaged communities: a conceptual framework of
supporting factors. Am J Community Psychol. 2016;58:192–210.
55. Achieving a high-reliability organization through implementation of the
ARCC model for systemwide sustainability of evidence-based practice. Nurs.
Adm. Q. B.M. Melnyk, College of Nursing, The Ohio State University, 1585
Neil Ave, Columbus, OH 43210, USA.; 2012. p. 127–35.
56. Lennox L, Doyle C, Reed J, Bell D. What makes a sustainability tool valuable,
practical, and useful in real world healthcare practice? A qualitative study on
the development of the long term success tool in Northwest London. BMJ
Open. 2017;7:1–13.
57. Blanchet K, Girois S. Selection of sustainability indicators for health services
in challenging environments: balancing scientific approach with political
engagement. Eval Program Plann Elsevier Ltd. 2013;38:28–32.
58. Fleiszer AR, Semenic SE, Ritchie JA, Richer MC, Denis JL. An organizational
perspective on the long- term sustainability of a nursing best practice
guidelines program: a case study. BMC Health Serv Res. 2015;15:204–18.
59. Knight T, Cropper S, Smith J. Developing sustainable collaboration: learning
from theory and practice. Prim Health Care Res Dev. 2001:139–48.
60. Okeibunor J, Bump J, Zouré HGM, Sékétéli A, Godin C, Amazigo UV. A
model for evaluating the sustainability of community-directed treatment
with ivermectin in the African Program for Onchocerciasis Control. Int J
Health Plann Manage. 2012;27:257–71.
61. Maher L, Gustafson D, Evans A. Sustainability model and guide. Coventry:
NHS Institute for Innovation and Improvement; 2010.
62. Alexander JA, Weiner BJ, Metzger ME, Shortell SM, Bazzoli GJ, Hasnain-Wynia
R, et al. Sustainability of collaborative capacity in community health
partnerships. Med Care Res Rev. 2003;60:130S–60S.
63. Dauphinee WD, Dauphinee WD, Reznick R, Reznick R. A framework for
designing, implementing, and sustaining a national simulation network.
Simul Healthc. 2011;6:94–100.
64. Saunders RP, Pate RR, Dowda M, Ward DS, Epping JN, Dishman RK.
Assessing sustainability of Lifestyle Education for Activity Program ( LEAP ).
Health Educ Res. 2012;27:319–30.
65. Amaya AB, Caceres CF, Spicer N, Balabanova D, Amaya AB, Caceres CF, et al.
After the Global Fund: who can sustain the HIV/AIDS response in Peru and
how? Glob Public Health Taylor & Francis. 2014;9:176–97.
66. Racine D. Reliable effectiveness: a theory on sustaining and replicating
worthwhile innovations. Adm Policy Ment Heal. (C)2006 Kluwer Academic
Publishers: (1)Independent Scholar, 1101 Parkside Avenue, Ewing, NJ 08618,
USA; 2006. p. 356–87.
67. Story WT, LeBan K, Altobelli LC, Gebrian B, Hossain J, Lewis J, et al.
Institutionalizing community-focused maternal, newborn, and child health
strategies to strengthen health systems: a new framework for the sustainable
development goal era. Glob Health Globalization Health. 2017;13:37.
68. Iwelunmor J, Blackstone S, Veira D, Nwaozuru U, Airhihenbuwa C,
Munodawafa D, et al. Toward the sustainability of health interventions
implemented in sub-Saharan Africa: a systematic review and conceptual
framework. Implementation Sci. 2016;11:43.
69. Blackford J, Street A. Tracking the route to sustainability: A service
evaluation tool for an advance care planning model developed for
community palliative care services. J. Clin. Nurs. Copyright (C) 2012
Blackwell Publishing Ltd.: Blackwell Publishing Ltd (9600 Garsington Road,
Oxford OX4 2XG, United Kingdom); 2012. p. 2136–48.
70. Rogers EM. Diffusion of innovations. Macmillian Publ. Co. 2003.
71. Schneider M, Somers M. Organizations as complex adaptive systems:
implications of complexity theory for leadership research. Leadersh Q. 2006;
17:351–65.
72. Bronfenbrenner U. The ecology of human development. Experiments by
nature and design. London: Harvard Univerity Press; 1979.
73. Katz D, Kahn RL. The social psychology of organizations. John Wiley & Sons
Inc: Hoboken; 1978.
74. Rycroft-Malone J. The PARIHS framework—a framework for guiding the
implementation of evidence-based practice. J Nurs Care Qual. 2004;19:
297–304.
75. Aarons GA, Hurlburt M, Horwitz SM. Advancing a conceptual model of
evidence-based practice implementation in public service sectors. Adm
Policy Ment Health Serv Res. 2011;38:4–23.
76. Francis L, Dunt D, Cadilhac DA. How is the sustainability of chronic disease
health programmes empirically measured in hospital and related healthcare
services?-a scoping review. BMJ Open. 2016;6:e010944.
77. Frumkin P, Reingold D. Evaluation research and institutional pressures:
challenges in public-nonprofit contracting. 2004. Report No.: 23.
78. Buscemi N, Hartling L, Vandermeer B, Tjosvold L, Klassen TP. Single data
extraction generated more errors than double data extraction in systematic
reviews. J Clin Epidemiol. 2006;59:697–703.
79. Fox A, Gardner G, Osborne S. A theoretical framework to support research
of health service innovation. Aust Health Rev. 2014;39:70–5.
80. Ament SMC, Gillissen F, Moser A, Maessen JMC, Dirksen CD, von Meyenfeldt
MF, et al. Identification of promising strategies to sustain improvements in
hospital practice: a qualitative case study. BMC Health Serv Res. 2014;14:641.
81. Bray P, Cummings DM, Pharm D, Wolf M, Massing MW, Reaves J. After the
collaborative is over: what sustains quality improvement initiatives in
primary care practices? Jt Comm J Qual Patient Saf. 2009;35:502–8.
82. Dominick GM, Tudose A, Pohlig RT, Saunders RP. Sustainability of physical
activity promoting environments and influences on sustainability following
a structural intervention in residential children’s homes. 2016;31:207–219.
83. Feldstein AC, Glasgow RERE. A practical, robust implementation for
integrating research findings into practice. Jt Comm J Qual patient Saf.
2008;34:228–43.
84. Finch TL, Mair FS, O’Donnell C, Murray E, May CR. From theory to
“measurement” in complex interventions: methodological lessons from the
development of an e-health normalisation instrument. BMC Med Res
Methodol Engl. 2012;12:2–16.
85. Goodman RM, McLeroy KR, Steckler AB, Hoyle RH. Development of level of
institutionalization scales for health promotion programs. Health Educ Q.
1993;20:161–78.
86. Goodman RM, Steckler A. A model for the institutionalisation of a health
promotion program . Fam Community Health. 1989:63–78.
87. Isabalija RS, Kituyi GM, Mbarika V. A framework for sustainable
implementation of E-medicine in transitioning countries. Int J Telemed
Appl. 2013;201:12.
88. Leffers J, Mitchell E. Conceptual model for partnership and sustainability in
global health. Public Heal. Nurs. Copyright (C) 2011 Blackwell Publishing
Ltd.: Blackwell Publishing Inc.; 2011. p. 91–102.
89. May C, Finch T. Implementing, embedding, and integrating practices: an
outline of normalization process theory. Sociology. 2009;43:535–54.
90. Melnyk BM, Fineout-Overholt E, Gallagher-Ford L, Stillwell S. Evidence-based
practice, step by step: sustaining evidence-based practice through
organizational policies and an innovative model. Am J Nurs. 2011;111:57–60.
91. Nystrom ME, Strehlenert H, Hansson J, Hasson H. Strategies to facilitate
implementation and sustainability of large system transformations: a case
study of a national program for improving quality of care for elderly people.
BMC Health Serv Res. 2014;14:401.
92. Olsen IT. Sustainability of health care: a framework analysis. Health Policy
Plan. 1998;13:287–95.
93. Rasschaert F, Decroo T, Remartinez D, Telfer B, Lessitala F, Biot M, et al.
Sustainability of a community-based anti-retroviral care delivery model—a
qualitative research study in Tete, Mozambique. J Int AIDS Soc Switz. 2014;
17:1–10.
Lennox et al. Implementation Science (2018) 13:27 Page 16 of 17
94. Rudd RE, Goldberg J, Dietz W. A five stage model for sustaining a
community campaign. J Health Commun. 1999;4:37–48.
95. Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O. Diffusion of
innovations in service organizations: systematic review and
recommendations. Milbank Q. 2004;82:581–629.
96. McLeroy KR, Bibeau D, Steckler A, Glanz K. An ecological perspective on
health promotion programs. Health Educ Q. 1988:351–77.
• We accept pre-submission inquiries
• Our selector tool helps you to find the most relevant journal
• We provide round the clock customer support
• Convenient online submission
• Thorough peer review
• Inclusion in PubMed and all major indexing services
• Maximum visibility for your research
Submit your manuscript at
www.biomedcentral.com/submit
Submit your next manuscript to BioMed Central
and we will help you at every step:
Lennox et al. Implementation Science (2018) 13:27 Page 17 of 17
-
Abstract
Background
Methods
Results
Conclusions
Trial registration
Background
Methods
Search and information sources
Data collection process and study selection
Quality assessment and data extraction strategy
Data synthesis and presentation
Risk of bias in individual studies and across studies
Registration
Results
Sustainability approaches
Theoretical perspectives, definitions and development details
Theoretical perspectives
Definitions of sustainability
Sustainability approach development
Sustainability method characteristics
Type
Aim
When to assess
Level of use
Settings
Suggested users
Sustainability constructs
Diversity in assessment
Top ten comparisons across approaches
Comparison across level of use
Comparison of prospective versus retrospective approaches
Discussion
Strengths and limitations
Future work
Conclusion
Additional files
Funding
Availability of data and materials
Disclaimer
Authors’ contributions
Ethics approval and consent to participate
Consent for publication
Competing interests
Publisher’s Note
Author details
References
Original Article
A Test of the ARCC C© Model Improves
Implementation of Evidence-Based Practice,
Healthcare Culture, and Patient Outcomes
Bernadette Mazurek Melnyk, RN, PhD, CPNP/PMHNP, FAANP, FNAP, FAAN •
Ellen Fineout-Overholt, RN, PhD, FNAP, FAAN • Martha Giggleman, RN, DNP,
NEA-BC • Katie Choy, RN, DNP, CNS, NEA-BC
Keywords
ARCC,
evidence-based
practice,
organizational
culture,
patient outcomes
ABSTRACT
Background: Although several models of evidence-based practice (EBP) exist, there is a paucity
of studies that have been conducted to evaluate their implementation in healthcare settings.
Aim: The purpose of this study was to examine the impact of the Advancing Research and
Clinical practice through close Collaboration (ARCC) Model on organizational culture, clinicians’
EBP beliefs and EBP implementation, and patient outcomes at one healthcare system in the
western United States.
Design: A pre-test, post-test longitudinal pre-experimental study was conducted with follow-up
immediately following full implementation of the ARCC Model.
Setting and Sample: The study was conducted at a 341-bed acute care hospital in the western
region of the United States. The sample consisted of 58 interprofessional healthcare professionals.
Methods: The ARCC Model was implemented in a sequential format over 12 months with the
key strategy of preparing a critical mass of EBP mentors for the healthcare system. Healthcare
professionals’ EBP beliefs, EBP implementation, and organizational culture were measured with
valid and reliable instruments. Patient outcomes were collected in aggregate from the hospital’s
medical records.
Results: Findings indicated significant increases in clinicians’ EBP beliefs and EBP implementation
along with positive movement toward an organizational EBP culture. Study findings also indicated
substantial improvements in several patient outcomes.
Linking Evidence to Action: Implementation of the ARCC Model in healthcare systems can en-
hance clinicians’ beliefs and implementation of evidence-based care, improve patient outcomes,
and move organizational culture toward EBP.
INTRODUCTION AND BACKGROUND
It is well known that evidence-based practice (EBP) improves
healthcare quality, safety, and patient outcomes as well as fos-
ters clinicians’ active engagement in their practices. Nurses
who use an evidence-based approach to care and practice in
cultures that support EBP are more empowered as they are
able to make a difference in the care of their patients. Although
the positive impact of EBP has been demonstrated through
multiple studies, major barriers exist that prevent EBP from
becoming the standard of care throughout the world. These
barriers include (a) inadequate EBP knowledge and skills of
clinicians, (b) misperceptions that EBP takes too much time,
(c) organizational culture and politics, (d) lack of support from
nurse leaders and managers, and (e) inadequate resources and
investment in EBP (Jun, Kovner, & Stimpfel, 2016; Melnyk
et al., 2016; Melnyk, Fineout-Overholt, Gallagher-Ford, & Ka-
plan, 2012). Aside from equipping clinicians with the knowl-
edge and skills needed to attain the EBP competencies and con-
sistently implement evidence-based care, findings from studies
have indicated that clinician access to EBP mentors can play a
key role in their implementation of EBP and the development
of organizational cultures that support the delivery of evidence-
based care (Fineout-Overholt & Melnyk, 2015; Melnyk, 2007).
Although several EBP models exist, most are process mod-
els that outline the steps of EBP or the sequence of conducting
an EBP project. EBP process models include the Johns Hopkins
Nursing Evidence-Based Practice Model (Dearholt & Dang,
2012), the Iowa Model of Evidence-Based Practice to Promote
Quality Care (Titler et al., 2001), the Model for Evidence-Based
Practice Change (Rosswurm & Larabee, 1999), and the ACE
Star Model of Knowledge Transformation (Stevens, 2012).
Unlike EBP process models, the Advancing Research and
Worldviews on Evidence-Based Nursing, 2017; 14:1, 5–9. 5
C© 2016 Sigma Theta Tau International
17416787, 2017, 1, D
ow
nloaded from
https://sigm
apubs.onlinelibrary.w
iley.com
/doi/10.1111/w
vn.12188 by Southern C
ross U
niversity, W
iley O
nline L
ibrary on [06/03/2023]. See the T
erm
s and C
onditions (https://onlinelibrary.w
iley.com
/term
s-and-conditions) on W
iley O
nline L
ibrary for rules of use; O
A
articles are governed by the applicable C
reative C
om
m
ons L
icense
A Test of the ARCC C© Model Improves Implementation of Evidence-Based Practice
Figure 1. The Advancing Research and Clinical Practice Through Close Collaboration (ARCC) Model.
Clinical practice through close Collaboration (ARCC) Model is
a system-wide model to advance and sustain EBP in healthcare
systems (see Figure 1). The first step in implementing the
ARCC Model is an organizational assessment of the current
EBP culture in order to identify strengths and major barriers
to EBP in the healthcare system so that strategies can be
implemented to remove those barriers. At the core of the
ARCC Model is a critical mass of EBP mentors who, through
intentional strategic initiatives, assist point of care clinicians
in enhancing their beliefs about the value of EBP and their
confidence in implementing it. As a result, ARCC contends
that heightened EBP beliefs in clinicians result in greater
implementation of evidence-based care, which ultimately
leads to higher job satisfaction, less staff turnover, and
improved patient outcomes. Several studies now support the
relationships among key constructs in the ARCC Model (Levin,
Fineout-Overholt, Melnyk, Barnes, & Vetter, 2011; Melnyk,
2012; Melnyk & Fineout-Overholt, 2002; Melnyk et al., 2004;
Melnyk, Fineout-Overholt, & Mays, 2008; Melnyk, Fineout-
Overholt, Giggleman, & Cruz, 2010; Wallen et al., 2010).
AIM
The purpose of this study was to examine the impact of the
ARCC Model on organizational culture, clinicians’ EBP beliefs
and EBP implementation, and patient outcomes at one health-
care system in the western region of the United States.
DESIGN
A pre-test, post-test longitudinal pre-experimental study was
conducted with follow-up immediately following full imple-
mentation of the ARCC Model. Institutional Review Board ap-
proval was obtained from the authors’ institution as well as the
organization’s research subject review board.
SETTING AND SAMPLE
This study was conducted at Washington Hospital Healthcare
System, a 341-bed acute care hospital in the San Francisco
bay area. The sample consisted of 58 interprofessional health-
care professionals, with complete follow-up data for 45 partic-
ipants. Participants were point of care nurses, administrators,
nurse managers, clinical nurse specialists, respiratory thera-
pists, occupational therapists, physical therapists, dieticians,
social workers, and pharmacists. Although physician cham-
pions participated in the projects, they were not part of the
data collection. Only the project teams participated in data
collection.
METHODS
The ARCC Model was implemented in a sequential format
over 12 months with the key strategy of preparing a critical
mass of EBP mentors for the healthcare system. Intensive EBP
workshops were first provided to the 58 participants in order
to enhance their knowledge and skills in the seven steps of
6 Worldviews on Evidence-Based Nursing, 2017; 14:1, 5–9.
C© 2016 Sigma Theta Tau International
17416787, 2017, 1, D
ow
nloaded from
https://sigm
apubs.onlinelibrary.w
iley.com
/doi/10.1111/w
vn.12188 by Southern C
ross U
niversity, W
iley O
nline L
ibrary on [06/03/2023]. See the T
erm
s and C
onditions (https://onlinelibrary.w
iley.com
/term
s-and-conditions) on W
iley O
nline L
ibrary for rules of use; O
A
articles are governed by the applicable C
reative C
om
m
ons L
icense
Original Article
Table 1. Examples of PICOT Questions Formulated
by the EBP Teams
� In ventilated intensive care unit patients (P), how does early
ambulation (I) compared to routinely scheduled ambulation
(C) affect length of stay and episodes of ventilator
associated pneumonia while in the intensive care unit (T)
� In congestive heart failure patients (P), how does
comprehensive pre-discharge education (I) compared to
standard pre-discharge education (C), affect readmission
rates to the hospital (O)?
EBP. In addition, content and skills building in the workshops
focused on how to facilitate individual behavior change of clin-
icians to implement EBP and how to facilitate an EBP organi-
zational culture. The 58 participants were divided into working
teams of six to eight members who were to collaborate on
an EBP change project to improve patient outcomes within
the hospital. Each team was then charged with formulating
a PICOT (Patient population, Intervention or Issue of inter-
est, Comparison intervention or issue, Outcome, and Time for
the intervention to achieve the outcome if relevant) question
about an important clinical issue, systematically searching for
the best evidence, and critically appraising and synthesizing
the evidence culminating in a recommendation for practice.
See Table 1 for examples of PICOT questions developed by
the teams. Strategic plans were then developed by the inter-
professional EBP mentor teams to implement and evaluate the
impact of the EBP changes on clinical outcomes within their
organization. After implementation and evaluation of the prac-
tice changes were completed, the final step for the teams was
to submit their projects for presentation at local, regional, or
national conferences to disseminate their successes to others
within the healthcare community.
OUTCOMES
Study variables were measured with the following valid and reli-
able instruments. The Evidence-Based Practice Beliefs (EBPB)
Scale Melnyk & Fineout-Overholt, 2003a) measured clinicians’
beliefs about EBP and their ability to implement it. The 16-item
Likert scale has established face, content, and construct valid-
ity with internal consistency reliabilities greater than .85 across
multiple studies (Melnyk et al., 2008). Responses on the scale
range from 1 (strongly disagree) to 5 (strongly agree). Examples
of items on the scale include (a) I am clear about the steps in
EBP, (b) I am sure that I can implement EBP, and (c) I am sure
that evidence-based guidelines can improve care.
The Evidence-Based Practice Implementation (EBPI) Scale
measured delivery of evidence-based care (Melnyk & Fineout-
Overholt, 2003b). Participants respond to each of the 18 Likert
scale items on the EBPI by answering how often in the last
eight weeks they have performed certain EBP activities, such as
(a) generated a PICOT question about my practice, (b) used evi-
dence to change my clinical practice, (c) evaluated the outcomes
of a practice change, and (d) shared the outcome data collected
with colleagues. The EBPI has established face, content, and
construct validity as well as internal consistency reliabilities
greater than .85 across multiple studies (Melnyk et al., 2008).
The Organizational Culture and Readiness Scale for
System-Wide Integration of Evidence-Based Practice (OCR-
SIEP) measured the organization’s culture and its readiness
for system-wide EBP (Fineout-Overholt & Melnyk, 2006). This
instrument contains 26 Likert scale items that identify a de-
scription of the existing support in the current culture for EBP,
which offers insight into the strengths and opportunities for
fostering evidence-based care within a healthcare system. The
OCRSIEP scale has established face and content validity along
with excellent internal consistency reliability of greater than .85
across multiple samples (Melnyk & Fineout-Overholt, 2015).
Examples of items on the OCRSIEP include the following:
(a) To what extent is EBP clearly described as central to the
mission and philosophy of your institution? (b) To what extent
do you believe that EBP is practiced in your organization? And
(c) To what extent is the nursing staff with whom you work
committed to EBP?
Patient Outcomes
Aggregate data were gathered by the teams, including data
from the hospital’s medical records (e.g., number of cases of
ventilator associated pneumonia, hospital readmission rates)
before and after implementation of the ARCC Model to evaluate
relevant patient outcomes as results of the EBP projects.
Analyses
T tests and effect sizes were calculated for study variables to
evaluate pre-to-post differences. A p value of .05 was set for
statistical significance.
RESULTS
Findings indicated that the clinicians’ EBP beliefs, EBP im-
plementation, and movement of organizational culture toward
EBP significantly increased over the 12-month project. Specif-
ically, clinicians’ EBP beliefs (n = 45) increased significantly
from baseline (M = 60.7, SD = 7.6) to follow-up (M = 64.9,
SD = 6.7; t = 4.2; p = .00; effect size = .62, which is a medium
to large positive effect for ARCC). EBP implementation also
significantly increased from baseline (M = 17.8, SD = 10.3) to
follow-up (M = 51.9, SD = 16.8; t = 12.9; p = .00; effect size =
2.3, indicating a large positive effect for ARCC). In addition,
organizational culture and readiness for EBP increased signifi-
cantly from baseline (M = 80.9; SD = 90.8) to follow-up (M =
90.8; SD = 14.7; t = 3.9; p = .00; effect size = .70, which
is a medium to large positive effect for ARCC). In addition,
as a result of implementing the ARCC Model, evidence-based
interventions improved key patient outcomes (see Table 2).
Worldviews on Evidence-Based Nursing, 2017; 14:1, 5–9. 7
C© 2016 Sigma Theta Tau International
17416787, 2017, 1, D
ow
nloaded from
https://sigm
apubs.onlinelibrary.w
iley.com
/doi/10.1111/w
vn.12188 by Southern C
ross U
niversity, W
iley O
nline L
ibrary on [06/03/2023]. See the T
erm
s and C
onditions (https://onlinelibrary.w
iley.com
/term
s-and-conditions) on W
iley O
nline L
ibrary for rules of use; O
A
articles are governed by the applicable C
reative C
om
m
ons L
icense
A Test of the ARCC C© Model Improves Implementation of Evidence-Based Practice
Table 2. Project Outcomes From Implementation
of the EBP Changes
� A practice change to early ambulation in the ICU led to a 2.7
reduction in ventilator days (11.6–8.9) and no ventilator
associated pneumonia.
� With the implementation of a pressure ulcer prevention
nursing standardized procedure on a medical-surgical unit,
the acquired pressure ulcer rate was significantly decreased
from 6.07% to 0.62% 1 year later.
� Comprehensive education of congestive heart failure
patients led to a 14.7% reduction in hospital readmissions.
� After implementation of family centered care on the
pediatric unit, 75% of parents perceived the overall quality
of care as excellent compared to 22% pre-implementation.
� The percentage of mothers not supplementing their breast
milk with formula increased from 61.7% to 71.1% after the
evidence-based baby friendly hospital initiative was
implemented.
� After implementation of a nurse-initiated pain protocol in
the emergency room (ER), wait time for pain medication
decreased from 46 minutes to 13 minutes and length of stay
in the ER also decreased from 120 minutes to 91 minutes.
DISCUSSION
Findings support the positive impact of implementing the
ARCC Model on clinicians’ EBP beliefs and a dramatic in-
crease in EBP implementation in those who participated in the
project. Organizational culture at the hospital shifted greatly
toward system-wide EBP. Most important, as a result of imple-
menting ARCC, there were multiple improvements in patient
outcomes.
The establishment of a cadre of EBP mentors is cen-
tral to building an organizational culture of EBP and im-
plementing evidence-based care. The EBP mentors in this
study garnered the knowledge and skills needed to successfully
implement and evaluate EBP changes within the hospital as
well as to work with their colleagues in creating an EBP culture
in which to deliver high-quality evidence-based care. These
findings affirm that culture eats strategy and assists clini-
cians in making EBP the social norm within a system (Mel-
nyk, 2016b). Without a culture and environment that supports
EBP, high-quality evidence-based care will not sustain (Melnyk,
2016a).
Numerous healthcare systems and hospitals throughout the
United States and globe have implemented the ARCC Model in
their efforts to build and sustain an EBP culture and environ-
ment in their organizations. As a part of building this culture,
position descriptions have been created or changed to include
responsibilities as an EBP mentor. For example, at The Ohio
State University Wexner Medical Center, the primary responsi-
bility of the clinical nurse specialists throughout the healthcare
system is to serve as EBP mentors for point of care staff in
improving patient outcomes. Part of this role is ensuring
compliance with the EBP competencies for advanced practice
nurses (Melnyk, Gallagher-Ford, & Fineout-Overholt, 2016;
Melnyk, Gallagher-Ford, Long & Fineout-Overholt, 2015).
Research is needed to further confirm the advantages of
using particular EBP models in real-world practice settings,
including how implementation of these models impact both
clinician, leader and patient outcomes (Dang et al., 2015). Com-
parative effectiveness studies that evaluate the benefits of in-
dividual models as well as combining models also are needed.
Those hospitals and systems who use an EBP model to guide
implementation of evidence-based care should document their
experiences and outcomes in order to better understand the
model’s usefulness in facilitating EBP and share this impor-
tant information with others who might use the model (Gra-
ham, Tetroe, & KT Theories Research Group, 2007). Return
on investment by including cost outcomes also should be eval-
uated. WVN
LINKING EVIDENCE TO ACTION
� The ARCC Model is an evidence-based system-
wide model for advancing the implementation and
sustainability of EBP.
� A key strategy in the ARCC model is the develop-
ment of a critical mass of EBP mentors who assist
point of care clinicians in the consistent imple-
mentation of evidence-based care.
� Use of ARCC EBP mentors enhances the EBP be-
liefs and EBP implementation of clinicians and
strengthens the EBP culture of an organization.
� An organizational culture of EBP is central to sup-
porting sustainable high quality evidence-based
care.
� Implementation of the ARCC Model can substan-
tially improve patient outcomes.
Author information
Bernadette Mazurek Melnyk, Associate Vice President for
Health Promotion, University Chief Wellness Officer, Dean
and Professor, College of Nursing, Professor of Pediatrics &
Psychiatry, and College of Medicine, The Ohio State Univer-
sity, Columbus, Ohio; Ellen Fineout-Overholt, Mary Coulter
Dowdy Distinguished Professor of Nursing, College of Nurs-
ing & Health Sciences University of Texas at Tyler, Tyler, Texas;
Martha Giggleman, Healthcare Consultant & Advocate Liver-
more, California; Katie Choy, Senior Director, Nursing Practice
and Education, Washington Hospital Healthcare System, Fre-
mont, California
8 Worldviews on Evidence-Based Nursing, 2017; 14:1, 5–9.
C© 2016 Sigma Theta Tau International
17416787, 2017, 1, D
ow
nloaded from
https://sigm
apubs.onlinelibrary.w
iley.com
/doi/10.1111/w
vn.12188 by Southern C
ross U
niversity, W
iley O
nline L
ibrary on [06/03/2023]. See the T
erm
s and C
onditions (https://onlinelibrary.w
iley.com
/term
s-and-conditions) on W
iley O
nline L
ibrary for rules of use; O
A
articles are governed by the applicable C
reative C
om
m
ons L
icense
Original Article
Address correspondence to Dr. Bernadette Mazurek Melnyk,
The Ohio State University, 145 Newton Hall, 1585 Neil Avenue,
Columbus, OH 43210; Melnyk.15@osu.edu
Accepted 16 September 2016
Copyright C© 2017, Sigma Theta Tau International
References
Dang, D., Melnyk, B. M., Fineout-Overholt, E., Ciliska, D., Di-
Censo, A., Cullen, L., . . . & Stevens, R. K. (2015). Models to
guide implementation and sustainability of evidence-based prac-
tice. In B. M. Melnyk & E. Fineout-Overholt (Eds.) Evidence-based
practice in nursing & healthcare. A guide to best practice (3rd ed.,
pp. 274–315). Philadelphia, PA: Wolters Kluwer.
Dearholt, S. L., & Dang, D. (2012). Johns Hopkins nursing evidence-
based practice model and guidelines (2nd ed.). Indianapolis, IN:
Sigma Theta Tau International.
Fineout-Overholt, E., & Melnyk, B. M. (2015). ARCC evidence-
based practice mentors: The key to sustaining evidence-based
practice. In B. M. Melnyk & E. Fineout-Overholt (Eds.) Evidence-
based practice in nursing & healthcare. A guide to best practice (3rd
ed., pp. 376–385). Philadelphia, PA: Wolters Kluwer.
Fineout-Overholt, E., & Melnyk, B. M. (2006). Organizational cul-
ture and readiness scale for system-wide integration of evidence-based
practice. Gilbert, AZ: ARCC, llc.
Graham, I. D., & Tetroe, J. & the KT Theories Research Group.
(2007). Some theoretical underpinnings of knowledge transla-
tion. Academic Emergency Medicine, 14(11), 936–941.
Jun, J., Kovner, C. T., & Stimpfel, A. W. (2016). Barriers and
facilitators of nurses’ use of clinical practice guidelines: An
integrative review. International Journal of Nursing Studies, 60,
54–68.
Levin, R. F., Fineout-Overholt, E., Melnyk, B. M., Barnes, M., &
Vetter, M. J. (2011). Fostering evidence-based practice to improve
nurse and cost outcomes in a community health setting: A pilot
test of the advancing research and clinical practice through close
collaboration model. Nursing Administration Quarterly, 35(1), 21–
33.
Melnyk, B. M. (2007). The evidence-based practice mentor: A
promising strategy for implementing and sustaining EBP in
healthcare systems. Worldviews on Evidence-Based Nursing, 4(3),
123–125.
Melnyk, B. M. (2012). Achieving a high-reliability organization
through implementation of the ARCC model for system wide
sustainability of evidence-based practice. Nursing Administration
Quarterly, 36(2), 127–135.
Melnyk, B. M. (2016a). An urgent call to action for nurse lead-
ers to establish sustainable evidence-based practice cultures and
implement evidence-based interventions to improve healthcare
quality. Worldviews on Evidence-Based Nursing, 13(1), 3–5.
Melnyk, B. M. (2016b). Culture eats strategy every time: What
works in building and sustaining an evidence-based practice cul-
ture in healthcare systems. Worldviews on Evidence-Based Nurs-
ing, 13(2), 99–101.
Melnyk, B. M., & Fineout-Overholt, E. (2002). Putting research
into practice. Reflections on Nursing Leadership, 28(2), 22–25.
Melnyk, B. M., & Fineout-Overholt, E. (2003a). Evidence-based prac-
tice beliefs scale. Gilbert, AZ: ARCC Publishing.
Melnyk, B. M., & Fineout-Overholt, E. (2003b). Evidence-based prac-
tice implementation scale (3rd ed.). Gilbert, AZ: ARCC Publishing.
Melnyk, B. M., & Fineout-Overholt, E. (2015). Evidence-based prac-
tice in nursing and healthcare: A guide to best practice. Philadelphia,
PA: Lippincott, Williams & Wilkins.
Melnyk, B. M., Fineout-Overholt, E., Fischbeck Feinstein, N., Li,
H., Small, L., Wilcox, L., & Kraus, R. (2004). Nurses’ perceived
knowledge, beliefs, skills, and needs regarding evidence-based
practice: Implications for accelerating the paradigm shift. World-
views on Evidence-Based Nursing, 1(3), 185–193.
Melnyk, B. M., Fineout-Overholt, E., Gallagher-Ford, L., & Kaplan,
L. (2012). The state of evidence-based practice in U.S. nurses:
Critical implications for nurse leaders and educators. Journal of
Nursing Administration, 42(9), 410–417.
Melnyk, B. M., Fineout-Overholt, E., Giggleman, M., & Cruz, R.
(2010). Correlates among cognitive beliefs, EBP implementa-
tion, organizational culture, cohesion and job satisfaction in
evidence-based practice mentors from a community hospital
system. Nursing Outlook, 58(6), 301–308.
Melnyk, B. M., Gallagher-Ford, L., & Fineout-Overholt, E. (2016).
Implementing the evidence-based practice competencies in healthcare.
A practical guide for improving quality, safety and patient outcomes.
Indianapolis, IN: Sigma Theta Tau International.
Melnyk, B. M., Fineout-Overholt, E., & Mays, M. (2008). The
evidence-based practice beliefs and implementation scales: Psy-
chometric properties of two new instruments. Worldviews on
Evidence-Based Nursing, 5(4), 208–216.
Melnyk, B. M., Gallagher-Ford, L., Thomas, B. K., Troseth, M.,
Wyngarden, K., & Szalacha, L. (2016). A study of chief nurse
executives indicates low prioritization of evidence-based practice
and shortcomings in hospital performance metrics across the
United States. Worldviews on Evidence-based Nursing, 13(1), 6–14.
Rosswurm, M. A., & Larrabee, J. H. (1999). A model for change
to evidence-based practice. Image: Journal of Nursing Scholarship,
31(4), 317–322.
Stevens, K. R. (2012). Star model of EBP: Knowledge transformation.
Academic Center for Evidence-based Practice, TX: The Univer-
sity of Texas Health Science Center at San Antonio.
Titler, M. G., Kleiber, C., Steelman, V. J., Rakel, B. A., Budreau,
G., Everett, L. Q., & . . . Goode, C. J. (2001). The Iowa Model
of evidence-based practice to promote quality care. Critical Care
Nursing Clinics of North America, 13(4), 497–509.
Wallen, G. R., Mitchell, S. A., Melnyk, B. M., Fineout-Overholt, E.,
Miller-Davis, C., Yates, J., & Hastings, C. (2010). Implement-
ing evidence-based practice: Effectiveness of a structured mul-
tifaceted mentorship programme. Journal of Advanced Nursing,
66(12), 2761–2771.
doi 10.1111/wvn.12188
WVN 2017;14:5–9
Worldviews on Evidence-Based Nursing, 2017; 14:1, 5–9. 9
C© 2016 Sigma Theta Tau International
17416787, 2017, 1, D
ow
nloaded from
https://sigm
apubs.onlinelibrary.w
iley.com
/doi/10.1111/w
vn.12188 by Southern C
ross U
niversity, W
iley O
nline L
ibrary on [06/03/2023]. See the T
erm
s and C
onditions (https://onlinelibrary.w
iley.com
/term
s-and-conditions) on W
iley O
nline L
ibrary for rules of use; O
A
articles are governed by the applicable C
reative C
om
m
ons L
icense
D I S C U R S I V E P A P E R
The
nexus of nursing leadership and a culture of safer patient
care
Melanie Murray RN, Doctoral Candidate1 | Deborah Sundin PhD, Senior Lecturer1 |
Vicki Cope PhD, Associate Professor Nursing, Academic Chair2
1School of Nursing and Midwifery, Edith
Cowan University, Joondalup, WA, Australia
2School of Health Professions, Murdoch
University, Murdoch, WA, Australia
Correspondence
Melanie Murray, School of Nursing and
Midwifery, Edith Cowan University,
Joondalup, WA, Australia.
Email: mmurray7@our.ecu.edu.au
Aims and objectives: To explore the connection between +6 nursing leadership and
enhanced patient safety.
Background: Critical reports from the Institute of Medicine in 1999 and Francis QC
report of 2013 indicate that healthcare organisations, inclusive of nursing leadership,
were remiss or inconsistent in fostering a culture of safety. The factors required to
foster organisational safety culture include supportive leadership, effective commu-
nication, an orientation programme and ongoing training, appropriate staffing, open
communication regarding errors, compliance to policy and procedure, and environ-
mental safety and security. As nurses have the highest patient interaction, and
leadership is discernible at all levels of nursing, nurse leaders are the nexus to influ-
encing organisational culture towards safer practices.
Design: The position of this article was to explore the need to form a nexus
between safety culture and leadership for the provision of safe care.
Conclusions: Safety is crucial in health care for patient safety and patient out
comes.
A culture of safety has been exposed as a major influence on patient safety prac-
tices, heavily influenced by leadership behaviours. The relationship between leader-
ship and safety plays a pivotal role in creating positive safety outcomes for patient
care. A safe culture is one nurtured by effective leadership.
Relevance to practice: Patient safety is the responsibility of all healthcare workers,
from the highest executive to the bedside nurse, thus effective leadership through-
out all levels is essential in engaging staff to provide high quality care for the best
possible patient outcomes.
K E YWORD S
health care, leadership, patient safety, safety culture
1 | INTRODUCTION
Patient safety culture is reflected in the beliefs, attitudes, percep-
tions, values and patterns of behaviour of an organisation and its
employees towards safety (Bowie, 2010; Muls et al., 2015). As
recently evidenced in such investigations as the Mid-Staffordshire
Trust enquiry (Francis, 2013), healthcare environments have been
acknowledged as high-risk and a lack of a safety culture have major
effects on patient outcomes. Alternatively, a
positive safety culture
in health care is evidenced by visibility of leaders and credible sup-
port for patient safety initiatives (Agnew, Flin, & Reid, 2012; Castel,
Ginsburg, Zaheer, & Tamim, 2015). Avoidance of adverse events
requires a system-wide approach as it is recognised that errors are
the result of failures related to “. . .management decisions and organ-
isational processes” (Auer, Schwendimann, Koch, De Geest, &
Ausserhofer, 2014, p. 23; Kaufman & McCaughan, 2013). To
Accepted: 25 July 2017
DOI:
10.1111/jocn.13980
J Clin Nurs. 2018;27:1287–1293. wileyonlinelibrary.com/journal/jocn © 2017 John Wiley & Sons Ltd | 1287
13652702, 2018, 5-6, D
ow
nloaded from
https://onlinelibrary.w
iley.com
/doi/10.1111/jocn.13980 by Southern C
ross U
niversity, W
iley O
nline L
ibrary on [06/03/2023]. See the T
erm
s and C
onditions (https://onlinelibrary.w
iley.com
/term
s-and-conditions) on W
iley O
nline L
ibrary for rules of use; O
A
articles are governed by the applicable C
reative C
om
m
ons L
icense
http://orcid.org/0000-0002-6335-1356
http://orcid.org/0000-0002-6335-1356
http://orcid.org/0000-0002-6335-1356
http://wileyonlinelibrary.com/journal/JOCN
overcome system faults, setting patient safety as an organisational
priority assists in fostering a culture of safety (Auer et al., 2014). A
strong safety culture will use failings to adapt work practices aiming
to improve and enhance patient care, thus increasing positive out-
comes (Bowie, 2010). Development of a safety culture relies upon
engagement at all levels, communication between executive and unit
levels, and trust in organisational leaders and management (Ammouri,
Tailakh, Muliira, Geethakrishnan, & Al Kindi, 2015; Auer et al.,
2014).
Safety culture has only recently been emphasised within health
care, after having been prominent in safety critical industries such as
aviation and the military for many years. A positive safety culture
does not just happen, and it requires the input and alignment of
quality and organisational properties. These properties have been
identified as: teamwork, evidence-based practice, communication,
ongoing education, a just culture, leadership and patient-centred care
(Reid & Dennison, 2011; Sammer, Lykens, Singh, Mains, & Lackan,
2010). A culture of patient safety within an organisation requires
support from all parties, especially organisational leaders. The Mid-
Staffordshire public enquiry exposed a lack of basic patient care, and
a negative organisational culture fostered by a focus on systems
statistics and reports rather than patient experiences and outcomes
(Francis, 2013). A positive safety culture, on the other hand, provides
a platform on which to base patient-centred care with safe care
delivery, shared values, zero tolerance for substandard care, empow-
erment of front-line staff, recognition of staff for their contributions
and professional responsibility described as foundations for patient-
centred care (Francis, 2013; Muls
et al., 2015).
Patient safety has been on the global healthcare agenda since
the Institute of Medicine released its report “To Err is Human”
where medical errors were highlighted as taking more lives than
motor vehicle accidents, breast cancer and AIDS (Kohn, Corrigan, &
Donaldson, 2000). This sparked several initiatives from the World
Health Organization (WHO) such as the perioperative Surgical Safety
Checklist and the introduction of Hand Hygiene programmes to
globally improve patient safety (WHO, 2009). A major systemic influ-
ence on patient care, and thus patient safety, is nursing leadership
(Agnew et al., 2012; Auer et al., 2014; Cummings et al., 2010; Dig-
nam et al., 2011; O’Connor & Carlson, 2016; Vaismoradi, Bondas,
Salsali, Jasper, & Turunen, 2012). As nurses have the highest patient
interaction, nurse leaders are in the best position to influence organi-
sational culture towards safer practices (Hendricks, Cope, & Baum,
2015; Vaismoradi et al.,
2012).
2 | METHODS
An integrative review of research literature was undertaken to
develop an understanding of leadership in health care and its influ-
ence on patient safety and safety culture. Inclusion and exclusion cri-
teria were determined to provide consistency and rigour to the
literature review. Articles were included if they were written in Eng-
lish; full-text; peer-reviewed research published between 2010 and
2016. The articles include content on both leadership and patient
safety. Papers were excluded if they were not published in English
and did not meet the inclusion criteria.
The initial search of the databases identified 905 articles, of
which 298 articles met inclusion criteria based on their titles. A
review of the keywords in these 298 articles led to elimination of a
further 194 articles, leaving 104 articles for initial review. Fifty-four
were discarded following review of their abstracts. The initial selec-
tion of articles yielded 50 articles for inclusion and following review,
a further 10 articles were excluded as there was no discussion or
findings correlating leadership to patient safety or safety culture,
leaving 30 for inclusion.
Themes developed during the review of the literature based on
the influences on safety culture in health care. These themes have
been labelled “leadership and employee engagement and empower-
ment,” “barriers to a safety culture,” and “leadership styles and
patient outcomes.” Before discussion of these themes, a description
of leadership styles prominent in nursing is warranted.
2.1 | Leadership styles
Leadership theories have been studied and developed throughout
history starting with the Great Man Theory of the 1800s to the
Army Leadership model of 2007 (Ledlow & Coppola, 2014). These
theories have seen many styles of leadership emerge, although not
all leadership styles suit all situations. Styles of leadership explain
how leaders engage with others. Leadership may be categorised
under two main style types: relational and task-oriented (Cummings,
2012; Cummings et al., 2010). The leadership styles prominent in
healthcare literature are transformational and transactional leader-
ship. In-depth discussion on leadership theories and styles is beyond
the scope of this paper; however, a short description is warranted.
Transformational leadership theory was developed in the 1970s
by Burns (Burns, 1978) and has recently been explored throughout
nursing literature. Transformational leadership is a relational leader-
ship style adopted by Magnet hospitals (Brewer et al., 2016) to lead
charge in developing and maintaining standards of excellence in
patient safety and patient outcomes. This style of leadership is asso-
ciated with positive patient outcomes resulting from a blameless
safety culture (Lievens & Vlerick, 2013; McFadden, Stock, & Gowen,
2015; Merrill, 2015). A blameless safety culture, or a just culture, is
What does this paper contribute to the wider
global clinical community?
• Leadership engagement enhances patient safety through
positive safety culture
• Leadership education is important for the support of
emerging nurse leaders
• Adopting an organisation wide blame-free philosophy
breaks down barriers to a safety culture
1288 | MURRAY ET AL.
13652702, 2018, 5-6, D
ow
nloaded from
https://onlinelibrary.w
iley.com
/doi/10.1111/jocn.13980 by Southern C
ross U
niversity, W
iley O
nline L
ibrary on [06/03/2023]. See the T
erm
s and C
onditions (https://onlinelibrary.w
iley.com
/term
s-and-conditions) on W
iley O
nline L
ibrary for rules of use; O
A
articles are governed by the applicable C
reative C
om
m
ons L
icense
a nonpunitive environment where inadvertent actions are used as a
stepping stone to improve practice, but where reckless behaviour
will not be tolerated (Jarrett, 2017).
The characteristics of a transformational leader include the ability
to engage, motivate, inspire and empower followers to aim above and
beyond their own boundaries to achieve a shared vision or organisa-
tional goal. Leaders are visible; they set clear expectations and pro-
mote open multidisciplinary communication; and they see errors as an
opportunity for improvement (McFadden et al., 2015; Merrill, 2015;
O’Connor & Carlson, 2016). Transformational leaders invoke change
and demonstrate emotional intelligence, consult with their followers
before making decisions and share the load (Cope & Murray, 2017;
Doody & Doody, 2012; Giltinane, 2013). This style of leadership fos-
ters a safety culture within an organisation through the development
of trust and a just, blame-free environment (Merrill, 2015; Vogelsme-
ier, Scott-Cawiezell, Miller, & Griffith, 2010). Transformational leaders
value their followers opinions, respect their experience affirm their
nurse colleagues ideas and involve them in decision-making (Sherman,
2012). In large organisations, there may be pockets of excellence in
units or areas where the leaders have an exceptional following; how-
ever, this may not be represented across an organisation. Similarities
exist in transactional leaders described to follow.
Transactional leadership, a task-oriented leadership style, uses
rewards to motivate followers to achieve goals (Ledlow & Coppola,
2014). This in turn can have a positive influence on follower’s satis-
faction levels. This leadership style is very effective when decisions
need to be made with haste, such as during medical crises, however,
may have negative effects on patient outcomes as it may reinforce
task-based behaviours in nurses which lies in contrast to holistic
nursing care (Cope & Murray, 2017; Giltinane, 2013). Both transfor-
mational and transactional styles of leadership use forms of motiva-
tion to engage staff or followers.
2.2 | Leadership and employee engagement and
empowerment
Engagement has been defined by Schaufeli, Matinez, Pinto, Salanova
and Bakker (cited in Bargagliotti, 2012, p. 1416) as a “. . .positive, ful-
filling work-related state of mind”. Nurses who are engaged have
better patient and organisational outcomes and leadership engage-
ment is influential in bedside nurse performance (Brady Germain &
Cummings, 2010; Day, 2014). Nurse leaders who set clear guideli-
nes, share their vision and lead by example have greater employee
engagement associated with increased performance from bedside
nurses, which is important for safe and innovative practice (Brady
Germain & Cummings, 2010). Senior nurse leaders may use recogni-
tion of good practice to motivate and empower bedside nurses to
improve quality of care across the board (Haycock-Stuart & Kean,
2012). Leadership engagement at the unit level has significant posi-
tive effects on the reporting of errors and adverse events as leaders
who engage their staff create an open communication environment
where there is no fear of repercussions for reporting errors (Castel
et al., 2015).
Employee engagement by nurse leaders assists in developing
trust in leaders’. Trust boosts safety culture, and visibility of leaders
fosters trust. With the establishment of trust, organisational staff
believe concerns will be heard and that the necessary patient safety
changes will occur. Open communication channels developed
through trust leads to a nonblame culture (Vogelsmeier
et al., 2010).
Studies suggest that organisations that have created a nonblame
safety culture have better patient outcomes (O’Connor & Carlson,
2016). These outcomes occur when leaders create an environment
where staff are encouraged to report errors, adverse events, near
misses and unsafe practices so system changes can be made (O’Con-
nor & Carlson, 2016; Sammer et al., 2010). In a culture of safety,
staff are also enabled to seek help, without the threat of derision,
but by knowing that they can voice their need for assistance to
avoid possible harm (Squires, Tourangeau, Spence Laschinger, &
Doran, 2010).
Reports such as Francis (2013) revealed poor work environments
that had negative impacts on patient outcomes. Such environments
develop in the presence of dissatisfied nurses who may be suffering
burnout or emotional exhaustion from ineffective leadership either
at a unit level or throughout an organisation (Daly, Jackson, Mannix,
Davidson, & Hutchinson, 2014). With burnout, emotional exhaustion,
and dissatisfaction, comes high attrition. This may be alleviated in
part through leadership engagement of bedside nurses who realise
the importance of their own clinical work and that of the quality
agenda of the organisation, thus creating positive work environments
(Daly et al., 2014).
Empowerment of staff through leadership engagement is a key
variable in job satisfaction, organisational commitment and intention
to stay (Cowden & Cummings, 2015). Intention to stay is an impor-
tant consideration for healthcare organisations the world over with
Australia predicting a nursing shortfall of approximately 109 000
within the next 10 years (Roche, Duffield, Dimitrelis, & Frew, 2015).
Through the creation of positive work environments, support for
bedside nurses and active promotion of organisational goals and
visions to encourage organisational commitment, transformational
leaders play a direct role in nursing job satisfaction and intention to
stay (Brewer et al., 2016; Roche et al., 2015).
Clinical nurse leaders are essential for ongoing quality of safe
patient care (Hendricks et al., 2015). Organisations need to invest in
leadership development as part of their succession planning. Glob-
ally, there is a push for nurses to receive leadership education at
undergraduate level, at entry level to the profession and through
ongoing leadership programmes within their work environments so
as to grow and nurture leaders at all levels for succession planning
(Sherman & Pross, 2010; Squires et al., 2010). Shared governance
frameworks have placed patient safety as the responsibility of all
healthcare workers and serve to empower staff to participate in
organisational decision-making to enhance patient outcomes (Kut-
ney-Lee et al., 2016). These frameworks also promote leadership
education and development programmes for all nursing levels,
including new graduate nurses (Hendricks et al., 2015). Such pro-
grammes have been evaluated as being beneficial as nurses learn
MURRAY ET AL. | 1289
13652702, 2018, 5-6, D
ow
nloaded from
https://onlinelibrary.w
iley.com
/doi/10.1111/jocn.13980 by Southern C
ross U
niversity, W
iley O
nline L
ibrary on [06/03/2023]. See the T
erm
s and C
onditions (https://onlinelibrary.w
iley.com
/term
s-and-conditions) on W
iley O
nline L
ibrary for rules of use; O
A
articles are governed by the applicable C
reative C
om
m
ons L
icense
necessary leadership skills, gain awareness of the political and organ-
isational needs that promote leader and organisational engagement,
build self-awareness and become empowered in their practice which
has the flow on effect of empowering others. These enhanced skills
empower nurses, especially the new graduate nurse, to autono-
mously make decisions at the bedside to maintain high levels of safe
patient care (Hendricks et al., 2015). Fresh perspectives on quality
care and safety challenges require engaged leadership and engaged
employees.
The employee engagement initiative of Leadership WalkRounds
has been introduced in many hospitals worldwide. Leadership Walk-
Rounds (WR) involve senior leaders and organisational executives
engaging with bedside nurses to discuss patient safety concerns
(Rotteau, Shojania, & Webster, 2014; Sexton et al., 2014). The Walk-
Round provides visibility of organisational or senior clinical leaders
creating opportunities for bedside staff to raise patient safety con-
cerns to the executive level (Rotteau et al., 2014; Sexton et al.,
2014). WRs have been documented as having a positive effect on
patient safety outcomes with personal feedback to those bedside
nurses raising concerns (Sexton et al., 2014). Alternatively, the WR
can potentially provide a barrier to safety if competing goals are not
recognised and mixed messages are sent. WalkRounds may expose a
disparity in perceptions between bedside nurses and hospital or
organisational leaders’ opinions of the most critical issues to be con-
sidered regarding safety practices for positive patient outcomes and
those perceptions of hospital or organisational leaders (Haycock-
Stuart & Kean, 2012; Rotteau et al., 2014; Sexton et al., 2014).
Hospital leaders may well steer conversations to issues of concern in
their remit while missing or ignoring vital issues at the bedside.
A further opportunity for organisational leaders to engage with
nursing staff is through the Chief Nursing Officer (CNO). CNOs are
leaders within healthcare organisations and in the best position to
be a nexus between bedside nurses and the organisation executive.
Having a CNO within the organisational leadership team assists in
the engagement of bedside nurses through leadership visibility and
promotion of the quality and safety agenda, however, nurses are
rarely represented on governing boards where decisions are made
on policy and strategic priorities (Disch, Dreher, Davidson, Sinioris, &
Wainio, 2011). As hospital boards are typically physician heavy,
there is an imbalance in strategic priorities in that perceptions of
safety and what constitutes an error may be different between
nurses and physicians which may potentiate breakdowns in commu-
nication and mistrust in the leaders (Castel et al., 2015; Vogelsmeier
et al., 2010). Healthcare organisations globally are promoting safety
culture through accreditation schemes, in some cases redesigning
work environments to support safety culture changes (Ammouri
et al., 2015).
Many hospitals, primarily in the United States of America (USA),
but also in Australia, Canada, Lebanon and Saudi Arabia have gained
Magnet accreditation through the American Nurses Credentialing
Center (ANCC) (American Nurses Credentialing Center, 2017). Mag-
net accreditation status is awarded to facilities who implement and
maintain exemplary health care through the five forces of
magnetism: transformational leadership; structural empowerment;
exemplary professional practice; new knowledge, innovation and
improvements; and empirical quality results (American Nurses Cre-
dentialing Center, 2017). This Magnet recognition framework pro-
motes exemplary nursing care through a positive workplace culture.
The framework provides a basis on which to build an enhanced set-
ting that recruits and retains highly qualified staff and through strong
leadership maintains high staff satisfaction levels that have been pro-
ven to flow on to decreased patient mortality rates (Aiken et al.,
2011; Moss, Mitchell, & Casey, 2017). High-quality work environ-
ments are the building blocks on which a culture of safety can be
built; however, they cannot occur nor be sustained unless nurse
leaders acknowledge their importance, and wholeheartedly endeav-
our to continue the work required to support them.
2.3 | Barriers to a safety culture
Safety culture is influenced by several factors both positive and
negative. Blame has been recognised as a negative influence on
patient safety and linked to under reporting of errors (Ammouri
et al., 2015; Castel et al., 2015; Kaufman & McCaughan, 2013;
O’Connor & Carlson, 2016 Vogelsmeier et al., 2010; Zaheer, Gins-
burg, Chuang, & Grace, 2015). While reporting systems are present
in many organisations, a culture that does not foster safety initia-
tives, or not seen to be acting on reports, leads to distrust in the
system by bedside nurses (Zaheer et al., 2015). Fears of recrimina-
tion through reporting voiced by bedside nurses include the follow-
ing: disciplinary action, limited career advancement, and retaliation
affecting livelihood (Castel et al., 2015; Kaufman & McCaughan,
2013). A culture of blame may also stem from a major disparity in
perceptions between organisational leaders who “. . .declare patient
safety as an organisational priority” (Vogelsmeier et al., 2010, p.
288) and bedside nurses who “. . .continue to report concerns about
actual safety practices and priorities” yet nothing gets done
(Vogelsmeier et al., 2010, p. 288).
Disempowerment of health professionals has been evident in
health care in recent years due to relentless organisational change
where the focus has strayed from the patient (Dignam et al.,
2011).
This was evident in the Francis report (2013) where it was testified
that bedside nurses in the Mid-Staffordshire NHS Foundation Trust
were not engaged and not empowered to provide safe and appropri-
ate care to patients as it was not seen as a priority from organisa-
tional leaders. Nurse leaders have also expressed frustration in being
able to keep up-to-date with research and evidence-based practice
to best support a safety environment and have requested more
ongoing education concerning clinical care, conflict management but
also specifically on leadership (Sherman, Schwarzkopf, & Kiger,
2011).
Organisational changes have led to an increase in administrative
duties for nurse leaders giving them less time to provide clinical
leadership to bedside nurses (Brady Germain & Cummings, 2010;
Dignam et al., 2011). These increased administrative tasks decrease
leader visibility to bedside nurses and hamper channels for reporting
1290 | MURRAY ET AL.
13652702, 2018, 5-6, D
ow
nloaded from
https://onlinelibrary.w
iley.com
/doi/10.1111/jocn.13980 by Southern C
ross U
niversity, W
iley O
nline L
ibrary on [06/03/2023]. See the T
erm
s and C
onditions (https://onlinelibrary.w
iley.com
/term
s-and-conditions) on W
iley O
nline L
ibrary for rules of use; O
A
articles are governed by the applicable C
reative C
om
m
ons L
icense
safety concerns or errors. This may also lead to decreased patient
safety initiatives from the bedside (Dignam et al., 2011).
Inadequate or inappropriate leadership education has also been
recognised as a barrier to nurse’s acceptance of leadership roles
(Enterkin, Robb, & McLaren, 2013; Grindel, 2016). Recruitment to
nurse leader positions is, and has been difficult, due to inadequate
succession planning or preparation and development of bedside
nurses through leadership education (Enterkin et al., 2013; Grindel,
2016). In some instances, this is due to the leader role taking on
increased management and administrative responsibilities and the
bedside nurses being aware of same (Enterkin et al., 2013). Effective
organisational leadership communications organisational goals and
visions (Enterkin et al., 2013). When this is not happening, engage-
ment and recruitment of bedside nurses to leadership roles is further
hampered. Engaging bedside nurses, creating awareness of leader-
ship roles, offering leadership education or supporting staff who
enrol in leadership courses, recognising clinical expertise and
supporting new nursing graduates will improve recruitment into lead-
ership education programmes and to consider leadership roles (Grin-
del, 2016).
Bedside nurses have been reported as believing that the quality
of patient care is dependent on the individual nurse delivering the
care, whereas nurse leaders believe leadership impacts the quality of
care by driving the quality agenda through policy development and
leadership from the executive perspective (Haycock-Stuart & Kean,
2012).
2.4 | Leadership styles and patient outcomes
Nurse leadership may have both positive and negative impacts on
the work environment, depending upon the leadership style and atti-
tudes of the leader. A leader need not be a manager, a person of
power or someone in the organisational hierarchy, although leaders
in these areas will be highly influential on organisational culture
(Daly et al., 2014).
Care settings having strong leadership, with satisfactory staff-
ing levels, multidisciplinary collaboration and empowerment to con-
tribute to policy development, have decreased incidence of
adverse events such as medication errors, healthcare acquired
infections, complaints related to care and falls (Wong & Gial-
lonardo, 2013). Patient safety is also concomitant to the nursing
work environment and the influence of leadership on the workings
of this care setting (Squires et al., 2010). Poor leadership beha-
viours, such as those of the laissez-faire leader: no leadership, or
ineffective leadership impact absenteeism, stress, emotional
exhaustion and intention to leave (Cope & Murray, 2017; Merrill,
2015). These factors have a significant effect on the quality of
care provided by bedside nurses and on patient safety (Squires
et al., 2010).
Creating safe care environments requires nurse leaders to “listen
and learn” (Squires et al., 2010, p. 916) and involve employees in
decision-making, to develop trust among bedside nurses and to look
to errors as an opening for learning and an opportunity to improve
practice (Merrill, 2015; O’Connor & Carlson, 2016). Leaders and
nurses need to take responsibility to seek leadership education and
participation for their own professional growth and to develop their
own confidence and competence in leadership. Further, followers of
leaders should reflect on their influence on the support of their
leader. Encouragement and support, rather than criticism and pas-
sive-aggressive commentary, can diminish a leader’s effectiveness.
Hospital-acquired harm decreases as ward safety culture increases,
with leadership behaviour having a direct influence on patient out-
comes (O’Connor & Carlson, 2016).
3 | CONCLUSION
Patient safety is the responsibility of all healthcare workers, from
the highest executive to the bedside nurse; thus, effective leader-
ship is the nexus to engagement of staff to provide high-quality
care. Creating a just, blame-free workplace safety culture through
effective leadership and the recognition and fostering of up and
coming leader’s only serves to strengthen the team for the best pos-
sible patient outcomes. The positive well-being of our patients relies
on a culture of safety as the patient safety practices at the bedside
are heavily influential on patient outcomes. Whether directly or indi-
rectly, those with the most influence on a patient’s outcome is the
nurse at the bedside. With effective leadership, these bedside
nurses can be empowered to go above and beyond their self-
imposed boundaries to meet a vision shared by their leader without
fear of recrimination. A leader is looked upon for clear guidance
towards a common goal, this and more is provided by the effectual
leader.
Leadership styles that have claimed credence in today’s health-
care literature are those of transformational and transactional lead-
ers. These leaders can engage their staff to bring about the
necessary changes that make their nursing units stand out from the
crowd with increased levels of excellence in patient care. Unfortu-
nately, several barriers to the creation of a safety culture exist that
has resulted in poor patient care worldwide. The safety of our
patients relies on a culture of safety. A safe culture is one nurtured
by effective leadership and leadership styles used by organisational
and nurse leaders are the nexus to enhanced patient safety out-
comes.
4 | RELEVANCE TO PRACTICE
Leaders may not necessarily be in formal senior positions but may
be anyone who is influential in patient care. Patient safety is the
responsibility of all healthcare workers, from the highest executive
to the bedside nurse. Patient experiences are influenced not only by
the nurse at the bedside but the overall workings of the organisation
thus effective leadership throughout all levels, especially from clinical
nurses at the bedside, is essential in engaging staff to provide high
quality care for the best possible patient outcomes.
MURRAY ET AL. | 1291
13652702, 2018, 5-6, D
ow
nloaded from
https://onlinelibrary.w
iley.com
/doi/10.1111/jocn.13980 by Southern C
ross U
niversity, W
iley O
nline L
ibrary on [06/03/2023]. See the T
erm
s and C
onditions (https://onlinelibrary.w
iley.com
/term
s-and-conditions) on W
iley O
nline L
ibrary for rules of use; O
A
articles are governed by the applicable C
reative C
om
m
ons L
icense
CONTRIBUTIONS
Study design: MM, data collection and analysis: MM; and manuscript
preparation: MM, DS, VC.
ORCID
Melanie Murray http://orcid.org/0000-0002-6335-1356
REFERENCES
Agnew, C., Flin, R., & Reid, J. (2012). Nurse leadership and patient safety.
BMJ, https://doi.org/10.1136/bmj.e4589
Aiken, L. H., Sloane, D. M., Clarke, S., Poghosyan, L., Cho, E., You, L., . . .
Aungsuroch, Y. (2011). Importance of work environments on hospital
outcomes in nine countries. International Journal for Quality in Health
Care, 23(11), 357–364.
Ammouri, A. A., Tailakh, A. K., Muliira, J. K., Geethakrishnan, R., & Al
Kindi, S. N. (2015). Patient safety culture among nurses. International
Nursing Review, 62(1), 102–110. https://doi.org/10.1111/inr.12159
American Nurses Credentialing Center. (2017). Announcing a New Model
for ANCC’s Magnet Recognition Program. Retrieved from: http://
www.nursecredentialing.org/MagnetModel
Auer, C., Schwendimann, R., Koch, R., De Geest, S., & Ausserhofer, D.
(2014). How hospital leaders contribute to patient safety through the
development of trust. The Journal of Nursing Administration, 44(1),
23–29. https://doi.org/10.1097/nna.0000000000000017
Bargagliotti, L. A. (2012). Work engagement in nursing: a concept analy-
sis. Journal of Advanced Nursing, 68(6), 1414–1428. https://doi.org/
10.1111/j.1365-2648.2011.05859.x
Bowie, P. (2010). Leadership and implementing a safety culture. Practice
Nurse, 40(10), 32–35. Retrieved from http://www.practicenurse.co.uk/
Brady Germain, P., & Cummings, G. G. (2010). The influence of nursing
leadership on nurse performance: A systematic literature review.
Journal of Nursing Management, 18(4), 425–439. https://doi.org/10.
1111/j.1365-2834.2010.01100.x
Brewer, C. S., Kovner, C. T., Djukic, M., Fatehi, F., Greene, W., Chacko, T.
P., & Yang, Y. (2016). Impact of transformational leadership on nurse
work outcomes. Journal of Advanced Nursing, 72(11), 2879–2893.
https://doi.org/10.1111/jan.13055
Burns, J. M. (1978). Leadership. New York, NY: Harper & Row.
Castel, E. S., Ginsburg, L. R., Zaheer, S., & Tamim, H. (2015). Understand-
ing nurses’ and physicians’ fear of repercussions for reporting errors:
Clinician characteristics, organization demographics, or leadership fac-
tors? BMC Health Services Research, 15(1), https://doi.org/10.1186/
s12913-015-0987-9
Cope, V. C., & Murray, M. (2017). Leadership styles in nursing. Nursing
Standard, 31(43), 61–70. https://doi.org/10.7748/ns.2017.e10836
Cowden, T.L., & Cummings, G.G. (2015) Testing a theoretical model of
clinical nurses’ intent to stay. Health Care Management Review. 40(2),
169–181. https://doi.org/10.1097/HMR.0000000000000008
Cummings, G. (2012). Editorial: Your leadership style—How are you
working to achieve a preferred future? Journal of Clinical Nursing, 21
(23–24), 3325–3327. https://doi.org/10.1111/j.1365-2702.2012.
04290.x
Cummings, G. G., MacGregor, T., Davey, M., Lee, H., Wong, C. A., Lo, E.,
. . . Stafford, E. (2010). Leadership styles and outcome patterns for
the nursing workforce and work environment: A systematic review.
International Journal of Nursing Studies, 47(3), 363–385. https://doi.
org/10.1016/j.ijnurstu.2009.08.006
Daly, J., Jackson, D., Mannix, J., Davidson, P., & Hutchinson, M. (2014).
The importance of clinical leadership in the hospital setting. Journal
of Healthcare Leadership, 75, https://doi.org/10.2147/jhl.s46161
Day, H. (2014). Engaging staff to deliver compassionate care and reduce
harm. British Journal of Nursing, 23(18), 974–980. https://doi.org/10.
12968/bjon.2014.23.18.974
Dignam, D., Duffield, C., Stasa, H., Gray, J., Jackson, D., & Daly, J. (2011).
Management and leadership in nursing: An Australian educational
perspective. Journal of Nursing Management, 20(1), 65–71. https://doi.
org/10.1111/j.1365-2834.2011.01340.x
Disch, J., Dreher, M., Davidson, P., Sinioris, M., & Wainio, J. A. (2011).
The role of the chief nurse officer in ensuring patient safety and
quality. The Journal of Nursing Administration, 41(4), 179–185.
https://doi.org/10.1097/nna.0b013e318211874b
Doody, O., & Doody, C. M. (2012). Transformational leadership in nursing
practice. British Journal of Nursing, 21(20), 1212–1218. https://doi.
org/10.12968/bjon.2012.21.20.1212
Enterkin, J., Robb, E., & McLaren, S. (2013). Clinical leadership for high-
quality care: Developing future ward leaders. Journal of Nursing Man-
agement, 21(2), 206–216. https://doi.org/10.1111/j.1365-2834.2012.
01408.x
Francis, R. (2013). Report of the Mid Staffordshire NHS Foundation
Trust Public Inquiry. Retrieved from http://www.midstaffpublicin
quiry.com
Giltinane, C. L. (2013). Leadership styles and theories. Nursing Standard,
27(41), 35–39. https://doi.org/10.7748/ns2013.06.27.41.35.e7565
Grindel, C. G. (2016). Clinical leadership: A call to action. Medsurg Nurs-
ing, 25(1).
Haycock-Stuart, E., & Kean, S. (2012). Does nursing leadership affect the
quality of care in the community setting? Journal of Nursing Manage-
ment, 20(3), 372–381. https://doi.org/10.1111/j.1365-2834.2011.
01309.x
Hendricks, J., Cope, V., & Baum, G. (2015). Postgraduate nurses’ insights
into the nursing leadership role. Do they intuitively link the role to
patient safety? Journal of Nursing Education and Practice, 5(9), 72–77.
https://doi.org/10.5430/jnep.v5n9p72
Jarrett, M. (2017). Patient safety and leadership: Do you walk the walk?
Journal of Healthcare Management, 62(2), https://doi.org/10.1097/
JHM-D-17-00005
Kaufman, G., & McCaughan, D. (2013). The effect of organisational cul-
ture on patient safety. Nursing Standard, 27(43), 50–56. https://doi.
org/10.7748/ns2013.06.27.43.50.e7280
Kohn, L. T., Corrigan, J., & Donaldson, M. S. (2000). To err is human:
Building a safer health system. Washington, DC: National Academy
Press.
Kutney-Lee, A., Germack, H., Hatfield, L., Kelly, S., Maguire, P., Dierkes,
A., . . . Aiken, L.H. (2016). Nurse engagement in shared governance
and patient and nurse outcomes. The Journal of Nursing Administra-
tion, 46(11), 605–612. https://doi.org/10.1097/NNA.
0000000000000412
Ledlow, G., & Coppola, N. (2014). Leadership for health professionals:
Theory, skills and applications, 2nd ed.. Burlington, MA: Jones & Bar-
tlett Learning.
Lievens, I., & Vlerick, P. (2013). Transformational leadership and safety
performance among nurses: The mediating role of knowledge-related
job characteristics. Journal of Advanced Nursing, 70(3), 651–661.
https://doi.org/10.1111/jan.12229
McFadden, K. L., Stock, G. N., & Gowen, C. R. (2015). Leadership, safety
climate, and continuous quality improvement. Health Care Manage-
ment Review, 40(1), 24–34. https://doi.org/10.1097/hmr.
0000000000000006
Merrill, K. C. (2015). Leadership style and patient safety. The Journal of
Nursing Administration, 45(6), 319–324. https://doi.org/10.1097/nna.
0000000000000207
Moss, S., Mitchell, M., & Casey, V. (2017). Creating a culture of success:
Using the Magnet Recognition Program as a framework to engage
nurses in and Australian healthcare facility. Journal of Nursing Admin-
istration, 47(2), 116–122.
1292 | MURRAY ET AL.
13652702, 2018, 5-6, D
ow
nloaded from
https://onlinelibrary.w
iley.com
/doi/10.1111/jocn.13980 by Southern C
ross U
niversity, W
iley O
nline L
ibrary on [06/03/2023]. See the T
erm
s and C
onditions (https://onlinelibrary.w
iley.com
/term
s-and-conditions) on W
iley O
nline L
ibrary for rules of use; O
A
articles are governed by the applicable C
reative C
om
m
ons L
icense
http://orcid.org/0000-0002-6335-1356
http://orcid.org/0000-0002-6335-1356
http://orcid.org/0000-0002-6335-1356
https://doi.org/10.1136/bmj.e4589
https://doi.org/10.1111/inr.12159
http://www.nursecredentialing.org/MagnetModel
http://www.nursecredentialing.org/MagnetModel
https://doi.org/10.1097/nna.0000000000000017
https://doi.org/10.1111/j.1365-2648.2011.05859.x
https://doi.org/10.1111/j.1365-2648.2011.05859.x
http://www.practicenurse.co.uk/
https://doi.org/10.1111/j.1365-2834.2010.01100.x
https://doi.org/10.1111/j.1365-2834.2010.01100.x
https://doi.org/10.1111/jan.13055
https://doi.org/10.1186/s12913-015-0987-9
https://doi.org/10.1186/s12913-015-0987-9
https://doi.org/10.7748/ns.2017.e10836
https://doi.org/10.1097/HMR.0000000000000008
https://doi.org/10.1111/j.1365-2702.2012.04290.x
https://doi.org/10.1111/j.1365-2702.2012.04290.x
https://doi.org/10.1016/j.ijnurstu.2009.08.006
https://doi.org/10.1016/j.ijnurstu.2009.08.006
https://doi.org/10.2147/jhl.s46161
https://doi.org/10.12968/bjon.2014.23.18.974
https://doi.org/10.12968/bjon.2014.23.18.974
https://doi.org/10.1111/j.1365-2834.2011.01340.x
https://doi.org/10.1111/j.1365-2834.2011.01340.x
https://doi.org/10.1097/nna.0b013e318211874b
https://doi.org/10.12968/bjon.2012.21.20.1212
https://doi.org/10.12968/bjon.2012.21.20.1212
https://doi.org/10.1111/j.1365-2834.2012.01408.x
https://doi.org/10.1111/j.1365-2834.2012.01408.x
http://www.midstaffpublicinquiry.com
http://www.midstaffpublicinquiry.com
https://doi.org/10.7748/ns2013.06.27.41.35.e7565
https://doi.org/10.1111/j.1365-2834.2011.01309.x
https://doi.org/10.1111/j.1365-2834.2011.01309.x
https://doi.org/10.5430/jnep.v5n9p72
https://doi.org/10.1097/JHM-D-17-00005
https://doi.org/10.1097/JHM-D-17-00005
https://doi.org/10.7748/ns2013.06.27.43.50.e7280
https://doi.org/10.7748/ns2013.06.27.43.50.e7280
https://doi.org/10.1097/NNA.0000000000000412
https://doi.org/10.1097/NNA.0000000000000412
https://doi.org/10.1111/jan.12229
https://doi.org/10.1097/hmr.0000000000000006
https://doi.org/10.1097/hmr.0000000000000006
https://doi.org/10.1097/nna.0000000000000207
https://doi.org/10.1097/nna.0000000000000207
Muls, A., Dougherty, L., Doyle, N., Shaw, C., Soanes, L., & Stevens, A.
(2015). Influencing organisational culture: A leadership challenge. Bri-
tish Journal of Nursing, 24(12), 633–638. https://doi.org/10.12968/
bjon.2015.24.12.633
O’Connor, S., & Carlson, E. (2016). Safety culture and senior leadership
behavior. The Journal of Nursing Administration, 46(4), 215–220.
https://doi.org/10.1097/NNA.0000000000000330
Reid, K. B., & Dennison, P. (2011). The clinical nurse leader (CNL): Point-
of-care safety clinician. Online Journal of Issues in Nursing, 16(1).
Roche, M. A., Duffield, C., Dimitrelis, S., & Frew, B. (2015). Leadership
skills for nurse unit managers to decrease intention to leave. Nursing:
Research and Reviews, 5, 57–64.
Rotteau, L., Shojania, K. G., & Webster, F. (2014). ‘I think we should just
listen and get out’: A qualitative exploration of views and experiences
of Patient Safety Walkrounds: Table 1. BMJ Quality & Safety, 23(10),
823–829. https://doi.org/10.1136/bmjqs-2012-001706
Sammer, C. E., Lykens, K., Singh, K. P., Mains, D. A., & Lackan, N. A.
(2010). What is patient safety culture? A review of the literature.
Journal of Nursing Scholarship, 42(2), 156–165. https://doi.org/10.
1111/j.1547-5069.2009.01330.x
Sexton, J. B., Sharek, P. J., Thomas, E. J., Gould, J. B., Nisbet, C. C.,
Amspoker, A. B., . . . Profit, J. (2014). Exposure to leadership Walk-
Rounds in neonatal intensive care units is associated with a better
patient safety culture and less caregiver burnout. BMJ Quality and
Safety, 23(10), 814–822. https://doi.org/10.1136/bmjqs-2013-
002042
Sherman, R. (2012). What followers want in their nurse leaders. American
Nurse Today, 7(9).
Sherman, R., & Pross, E. (2010). Growing future nurse leaders to build
and sustain healthy work environments at the Unit Level. OJIN, 15
(1), https://doi.org/10.3912/OJIN.Vol15No01Man01
Sherman, R., Schwarzkopf, R., & Kiger, A. J. (2011). Charge nurse per-
spectives on frontline leadership in acute care environments.
International Scholarly Research Network, https://doi.org/10.5402/
2011/164052
Squires, M., Tourangeau, A., Spence Laschinger, H. K., & Doran, D.
(2010). The link between leadership and safety outcomes in hospitals.
Journal of Nursing Management, 18(8), 914–925. https://doi.org/10.
1111/j.1365-2834.2010.01181.x
Vaismoradi, M., Bondas, T., Salsali, M., Jasper, M., & Turunen, H. (2012).
Facilitating safe care: A qualitative study of Iranian nurse leaders.
Journal of Nursing Management, 22(1), 106–116. https://doi.org/10.
1111/j.1365-2834.2012.01439.x
Vogelsmeier, A., Scott-Cawiezell, J., Miller, B., & Griffith, S. (2010). Influ-
encing leadership perceptions of patient safety through just culture
training. Journal of Nursing Care Quality, 25(4), 288–294. https://doi.
org/10.1097/ncq.0b013e3181d8e0f2
Wong, C. A., & Giallonardo, L. M. (2013). Authentic leadership and
nurse-assessed adverse patient outcomes. Journal of Nursing Manage-
ment, 21(5), 740–752. https://doi.org/10.1111/jonm.12075
World Health Organisation. (2009, August). Hand hygiene: Why, how &
when? Retrieved from http://www.who.int/gpsc/5may/Hand_Hygie
ne_Why_How_and_When_Brochure ?ua=1
Zaheer, S., Ginsburg, L., Chuang, Y., & Grace, S. L. (2015). Patient safety
climate (PSC) perceptions of frontline staff in acute care hospitals.
Health Care Management Review, 40(1), 13–23. https://doi.org/10.
1097/hmr.0000000000000005
How to cite this article: Murray M, Sundin D, Cope V. The
nexus of nursing leadership and a culture of safer patient
care. J Clin Nurs. 2018;27:1287–1293. https://doi.org/
10.1111/jocn.13980
MURRAY ET AL. | 1293
13652702, 2018, 5-6, D
ow
nloaded from
https://onlinelibrary.w
iley.com
/doi/10.1111/jocn.13980 by Southern C
ross U
niversity, W
iley O
nline L
ibrary on [06/03/2023]. See the T
erm
s and C
onditions (https://onlinelibrary.w
iley.com
/term
s-and-conditions) on W
iley O
nline L
ibrary for rules of use; O
A
articles are governed by the applicable C
reative C
om
m
ons L
icense
https://doi.org/10.12968/bjon.2015.24.12.633
https://doi.org/10.12968/bjon.2015.24.12.633
https://doi.org/10.1097/NNA.0000000000000330
https://doi.org/10.1136/bmjqs-2012-001706
https://doi.org/10.1111/j.1547-5069.2009.01330.x
https://doi.org/10.1111/j.1547-5069.2009.01330.x
https://doi.org/10.1136/bmjqs-2013-002042
https://doi.org/10.1136/bmjqs-2013-002042
https://doi.org/10.3912/OJIN.Vol15No01Man01
https://doi.org/10.5402/2011/164052
https://doi.org/10.5402/2011/164052
https://doi.org/10.1111/j.1365-2834.2010.01181.x
https://doi.org/10.1111/j.1365-2834.2010.01181.x
https://doi.org/10.1111/j.1365-2834.2012.01439.x
https://doi.org/10.1111/j.1365-2834.2012.01439.x
https://doi.org/10.1097/ncq.0b013e3181d8e0f2
https://doi.org/10.1097/ncq.0b013e3181d8e0f2
https://doi.org/10.1111/jonm.12075
http://www.who.int/gpsc/5may/Hand_Hygiene_Why_How_and_When_Brochure ?ua=1
http://www.who.int/gpsc/5may/Hand_Hygiene_Why_How_and_When_Brochure ?ua=1
https://doi.org/10.1097/hmr.0000000000000005
https://doi.org/10.1097/hmr.0000000000000005
https://doi.org/10.1111/jocn.13980
https://doi.org/10.1111/jocn.13980
Seers et al. Implementation Science (2018) 13:137
https://doi.org/10.1186/s13012-018-0831-9
RESEARCH Open Access
Facilitating Implementation of Research
Evidence (FIRE): an international cluster
randomised controlled trial to evaluate
two models of facilitation informed
by the Promoting Action on Research
Implementation in Health Services
(PARIHS) framework
Kate Seers1* , Jo Rycroft-Malone2, Karen Cox3, Nicola Crichton4, Rhiannon Tudor Edwards5, Ann Catrine Eldh6,7,
Carole A. Estabrooks8, Gill Harvey9, Claire Hawkes10, Carys Jones5, Alison Kitson11, Brendan McCormack12,
Christel McMullan13, Carole Mockford14, Theo Niessen3, Paul Slater15, Angie Titchen15, Teatske van der Zijpp3
and Lars Wallin7,16,17
: Health care practice needs to be underpinned by high quality research evidence, so that the best possible
care can be delivered. However, evidence from research is not always utilised in practice. This study used the Promoting
Action on Research Implementation in Health Services (PARIHS) framework as its theoretical underpinning to test
whether two different approaches to facilitating implementation could affect the use of research evidence in practice.
: A pragmatic clustered randomised controlled trial with embedded process and economic evaluation was
used. The study took place in four European countries across 24 long-term nursing care sites, for people aged 60 years
or more with documented urinary incontinence. In each country, sites were randomly allocated to standard dissemination,
or one of two different types of facilitation. The primary outcome was the documented percentage compliance with the
continence recommendations, assessed at baseline, then at 6, 12, 18, and 24 months after the intervention.
Data were analysed using STATA15, multi-level mixed-effects linear regression models were fitted to scores for compliance
with the continence recommendations, adjusting for clustering.
Results: Quantitative data were obtained from reviews of 2313 records. There were no significant differences in the
primary outcome (documented compliance with continence recommendations) between study arms and all study arms
improved over time.
(Continued on next page)
* Correspondence: Kate.seers@warwick.ac.uk
The articles related to this article are available online at https://doi.org/
10.1186/s13012-018-0811-0 and https://doi.org/10.1186/s13012-018-0812-z.
1Warwick Research in Nursing, Warwick Medical School, University of
Warwick, Coventry, UK
Full list of author information is available at the end of the article
© The Author(s). 2018 Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0
International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and
reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to
the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver
(http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
http://crossmark.crossref.org/dialog/?doi=10.1186/s13012-018-0831-9&domain=pdf
http://orcid.org/0000-0001-7921-552X
mailto:Kate.seers@warwick.ac.uk
https://doi.org/10.1186/s13012-018-0811-0
https://doi.org/10.1186/s13012-018-0811-0
https://doi.org/10.1186/s13012-018-0812-z
http://creativecommons.org/licenses/by/4.0/
http://creativecommons.org/publicdomain/zero/1.0/
Seers et al. Implementation Science (2018) 13:137 Page 2 of 11
(Continued from previous page)
: This was the first cross European randomised controlled trial with embedded process evaluation that
sought to test different methods of facilitation. There were no statistically significant differences in compliance with
continence recommendations between the groups. It was not possible to identify whether different types and “doses”
of facilitation were influential within very diverse contextual conditions. The process evaluation (Rycroft-Malone et al.,
Implementation Science. doi: 10.1186/s13012-018-0811-0) revealed the models of facilitation used were limited in their
ability to overcome the influence of contextual factors.
Trial registration: Current Controlled Trials ISRCTN11598502. Date 4/2/10.
The research leading to these results has received funding from the European Union’s Seventh Framework Programme
(FP7/2007–2013) under grant agreement no. 223646.
Keywords: Facilitation, Implementation, PARIHS, Urinary incontinence, Context, Older people, RCT
Background
It is important that health care practice is underpinned by
high quality research evidence, so that the best possible
care can be delivered. However, evidence from research is
not always utilised in practice [1–3]. This study used the
Promoting Action on Research Implementation in Health
Services (PARIHS) framework [4] as its theoretical under-
pinning to test whether two different approaches to facili-
tation could affect the use of research evidence in
practice. The PARIHS framework was built upon an argu-
ment that three factors influence the uptake of research
evidence in practice: the nature (strength) of the evidence,
the context in which it is used, and the extent of facilita-
tion (or help) that people have to use the evidence. The
published protocol for this study [5] and an online sum-
mary report for the funder [6] contain further details.
Consistent with recent calls for an increase in theory-
based implementation research [7], we used the PARIHS
framework and identified two alternative types of facilita-
tion to evaluate within the FIRE study. We chose to evalu-
ate facilitation because whilst it is a promising approach
to implementation, it has received relatively little attention
and the limited results available of its effectiveness were
mixed. [8–10] Facilitation has been described as a process
and a role [11]. More recently it has been argued [12]
“conceptual ambiguities” challenge our understanding of
facilitation’s effectiveness and we do not know how to “ap-
propriately set the degree of facilitation”. It is clear from
the literature that the role and effectiveness of facilitation
in implementing evidence into practice needs to be ex-
plored and tested. This study was novel in scale with a
cross-country setting, and in that it sought to compare fa-
cilitation approaches that varied in terms of focus, dur-
ation and intensity.
Urinary incontinence in long-term care settings is a
major issue and was thus selected as an exemplar for
evaluating different approaches to implementing evidence
into practice. Incontinence is a “discrediting and stigma-
tising” condition that affects quality of life [13]. It has a
high prevalence in long-term care settings, between 40
and 70% [14], and it is a key priority within international
health policy [15]. The relevance and fit of the PARIHS
framework in long-term care settings for older people [16]
highlighted that the factors discussed as important for
change in their setting showed a good fit with those iden-
tified in the PARIHS framework, and recommended its
use in these settings. We designed the FIRE trial to test
two different approaches to facilitation and compare these
against standard dissemination of recommendations for
continence promotion [5].
Aims
We aimed to extend knowledge of facilitation as a
process for getting research evidence into practice by
testing the effectiveness of and evaluating the contribu-
tion two different models of facilitation can make to
implementing evidence-based urinary continence recom-
mendations into practice.
The objectives of the study were to
1) Extend existing knowledge of facilitation as a
process for translating research evidence into practice.
2) Evaluate the feasibility and effectiveness of two dif-
ferent models of facilitation in promoting the uptake of
research-based recommendations on continence promo-
tion, compared with standard dissemination.
3) Advance existing knowledge of guideline implemen-
tation in healthcare, with a particular focus on under-
standing the impact of contextual factors on the
processes and outcomes of implementation.
4) Implement a pro-active dissemination strategy that
complements the design of the study and facilitates the
diffusion of the study findings to a wide policy and prac-
tice community throughout Europe and beyond. This
objective is not considered further in this paper.
Methods
Design
A pragmatic cluster randomised controlled trial with em-
bedded process and economic evaluation was undertaken.
https://doi.org/10.1186/ISRCTN11598502
Seers et al. Implementation Science (2018) 13:137 Page 3 of 11
The process evaluation is reported in a linked paper
(Rycroft-Malone et al. [17]).
Participants
Staff: an internal facilitator (a member of staff from the
long-term care setting) nominated in each intervention
site to work with external facilitators (EFs) to implement
the urinary incontinence (UI) recommendations.
Residents: aged 60 years or more with documented
urinary incontinence.
Setting
The study took place in four European countries (Eng-
land, Sweden, Netherlands, Republic of Ireland), and
each country planned to recruit six long-term nursing
care sites (nursing homes and other residential settings
with long term nursing care) (total 24 sites) for people
aged 60 years or more with documented urinary incon-
tinence. All settings had publicly funded places.
The intervention
In arm one, the eight settings randomised to the standard
dissemination control group had the urinary continence
recommendations and a PowerPoint presentation on im-
plementation (based on one utilised by Rycroft-Malone
et al. [18]) sent to the head of each site. Both the interven-
tion groups also received the same as the standard dissem-
ination sites.
In addition, EFs prepared two different facilitator de-
velopment programmes, each of which involved an ini-
tial residential programme, followed by virtual support
(monthly telephone group supervision and email com-
munication) for the internal facilitators (IFs) in imple-
menting the UI recommendations. Arm two received a
type of facilitation that we termed ‘type A’, which is a
goal-focused approach to facilitation based on principles
of quality improvement, management studies and organ-
isational learning. This involved a 12 month programme
for IFs nominated by each of the eight sites in this arm.
This started with the IFs taking part in a 3-day residen-
tial programme run by two EFs (GH and AK), followed
by 10 days over 12 months to work locally on the imple-
mentation and evaluation of recommendations, sup-
ported by 12 half-days for monthly teleconferences and
self-directed study (16 days in total).
Arm three received a type of facilitation that we
termed ‘type B’, which is underpinned by principles of
stakeholder empowerment and overcoming external and
internal obstacles to using research evidence in practice.
This is achieved through the creation of workplace cul-
tures of effectiveness in which work-based learning as
inquiry is valued and supported at all levels of the organ-
isation. This approach is informed by critical social the-
ory and holistic facilitation. IFs nominated by each of
the eight settings participated in a 24-month develop-
ment programme. This started with a 5-day residential
programme run by two EFs (BMcC and AT) followed by
20 days to work on the local implementation and evalu-
ation of the recommendations, supported by 24 half-day
learning groups via teleconferencing, and 12 half-days
for self-directed study (38 days in total). The EFs each
have over 20 years’ experience of facilitation. Add-
itional file 1: File S1 contains more details on the under-
pinning theories and activities in each intervention.
A model of co-facilitation was used in both facilitation
arms where a second staff member in the organisation, a
“buddy”, worked with the IF, using this as a development
opportunity, including taking the lead if the initial facili-
tator was unable to continue.
Outcomes measures
The primary outcome was the documented percentage
compliance with continence recommendations produced
by the fourth International Consultation on Incontin-
ence [19]. Percentage compliance is calculated for each
resident, so it is measured at the resident level.
These recommendations included (1) the resident
should be actively screened for incontinence (five com-
ponents), (2) a detailed assessment should be carried out
(15 components), (3) an individualised treatment plan
should be in place (13 components) and (4) a specialist
referral should be made if needed (one component).
These outcomes were assessed at baseline, then at 6, 12,
18 and 24 months.
2: File S2 lists all com-
ponents of the continence recommendations.
Secondary outcomes
Included the documented incidence of level of cognitive
impairment (as this influences the type of continence
care the guidance recommends), depression, incontinence-
related dermatitis, urinary tract infections (UTIs),
health-related quality of life (EQ-5D [20] and IQoL
[21]) and the proportion of residents in the setting with
incontinence and use of pelvic floor exercises. Organisa-
tional context was assessed using the Alberta Context
Tool (ACT) [22, 23]. The ACT data was collected from
Nurses, Licenced Practical Nurses (LPN) and Health Care
Assistants (HCA) at baseline in 23 of the 24 sites.
Sample size and power calculations
There was no information on existing compliance with
the continence recommendations. We took a 50% com-
pliance as an initial assumption. It was assumed that
each setting would have 50 residents available for asses-
sing compliance. For 90% power to detect compliance of
15% better in the intervention compared to control arm
and allowing for an intra-cluster correlation of 0.01 (typ-
ically found in Primary Care Studies [24]) and statistical
Seers et al. Implementation Science (2018) 13:137 Page 4 of 11
tests carried out at the 5% level, for a cluster size of 50,
seven clusters (long term care settings) were required
per intervention arm. Thus 7 × 3 arms = 21 clusters were
needed. Allowing for potential attrition, this was in-
creased to 8 clusters per arm, so 24 clusters in total.
This equates to 6 long-term nursing care settings in each
of the four countries with 50 or more residents per set-
ting. Consent was sought at cluster and at individual
level, the former before randomisation and the latter
after randomisation.
Randomisation sequence generation, allocation concealment,
implementation and blinding
In each country, sites were randomly allocated to one of
three arms (standard dissemination, and two different
intensities and kinds of a facilitation intervention), using
a random sequence generated by the statistician. A cen-
tralised randomisation point was set up by the study
statistician to ensure allocation concealment. Long-term
care settings were enrolled by country leads for the
study. The statistician was blinded to the intervention
group. It was not possible to blind site staff to interven-
tion. Research fellows who collected data from records
and where necessary obtained consent from residents
were blinded to the intervention group, but as discussed
in the protocol [5], previous experience suggested this
blinding may be inadvertently broken by the sites.
Quantitative analysis—statistical methods
Data were analysed using STATA15. The primary out-
come measures, percentage compliance, were analysed by
fitting multi-level mixed-effects linear regression models
with standard errors adjusted for the clustering at the level
of the nursing care setting (site level) [25]. Data was col-
lected every 6 months, but because the resident popula-
tion was constantly changing it is necessary to consider
the data as repeated cross-sectional assessments of resi-
dents in the care settings rather than longitudinal assess-
ment of individuals within the care settings. The
regression models include three independent variables:
study arm (three levels), country (four levels), time period
(five levels baseline, 6 months, 12 months, 18 months,
24 months), interaction terms would only be fitted if study
arm main effects were significant. Intra-cluster correlation
coefficients (ICC) for the baseline measurements of the
compliance scores were calculated through ANOVA with
adjustment for clustering and unequal cluster size.
Post-estimation ICCs are calculated after fitting the re-
gression models. Descriptive statistics, ANOVA and
chi-square tests were used, where appropriate, to examine
differences between groups with regard to secondary out-
comes. Data were examined by an independent data mon-
itoring committee.
Qualitative analysis
The process evaluation data were analysed from a realist
perspective [26] and are reported in the linked paper
(Rycroft-Malone et al. [17]).
Findings
In each country, we planned to recruit six sites (two sites
per arm). This happened for Sweden and the Netherlands.
However, one site in England withdrew before the study
started. When no additional site was forthcoming in Eng-
land within the timeframe, an additional control site was
recruited in Republic of Ireland and ethical clearance ob-
tained. This final site had data collected up to month 18
only as there was not time to collect data at 24 months.
There were thus five sites in England (with one site in the
control arm) and seven in the Republic of Ireland (three
sites in the control arm). Each cluster (site) received the
allocated intervention and were analysed for primary and
secondary outcomes.
Quantitative data were available from 2313 resident
records across all time points (n = 430 at baseline,
n = 462 at 6 months, n = 497 at 12 months, n = 479
at 18 months and n = 445 at 24 months after the
intervention). The sample is described and then the
primary outcome, compliance with the four contin-
ence recommendations is presented. The study took
place between 2010 and 2013.
Description of resident sample
Most residents were included at one time point only. In
all four countries, at baseline the mean age of residents
varied from 82 to 87 years. This was almost unchanged
at 24 months later (range 82–86 years). In all four coun-
tries, there were more female than male residents at
baseline (the percentage female in each site ranged from
60 to 71%), and this was similar at 24 months (range
54–80% females). At baseline, the mean age of residents
allocated to the three intervention groups was very simi-
lar (control 85.34 years (s.d. 7.39); type A 86.35 years
(s.d. 7.19); type B 83.20 years (s.d. 8.48)). The gender mix
was also similar for the three intervention groups (control
68.8% female; type A 62.2% female; type B 73.8%).
To understand the health status of the residents, data
from the EQ-5D visual analogue scale (VAS) measure of
health state that we administered at 24 months provides
summary information for each intervention group
(Table 1). Data at 24 months are chosen because
EQ-5D-VAS was available for a higher proportion of resi-
dents than any other time point. Higher scores on a scale
of 0–100 represent better health states. Table 1 shows
there was no significant difference in the mean EQ-5D
scale for the intervention groups; so on average resident
health status was similar in all the intervention groups.
Table 1 Summary statistics for EQ-5D-VAS scale for each
intervention group
Intervention Number of residents
with completed scale
Mean
(SE robust)a
95% CI
for mean
Range
Standard
dissemination
(control)
109 54.2 (4.737) 44.35, 64.00 0, 100
Type A 113 59.2 (4.325) 50.19, 68.13 0, 100
Type B 124 55.6 (2.918) 49.57, 61.67 0, 90
aSE robust allows for the clustering, and ANOVA allowing for clustering to
compare the three means, gave p = 0.34
Table 2 Multi-level mixed-effect linear regression
model—percentage compliance with recommendation 1 (the
resident should be actively screened for urinary incontinence),
with adjustment of standard errors to allow for clustering
Coefficient Std. Err. z p value 95% confidence
interval
Type A 2.9293 3.1298 0.94 0.349 − 3.2049, 9.0635
Type B − 4.1688 3.6966 − 1.13 0.259 − 11.4141, 3.0765
Sweden − 31.0840 4.0940 − 7.59 0.000 − 39.1082, − 23.0599
Ireland 13.8449 4.5226 3.06 0.002 4.9808, 22.7091
England 10.3152 5.0742 2.03 0.042 0.3699, 20.2604
+ 6 months 0.1104 2.8436 0.04 0.969 − 5.4630, 5.6837
+ 12 months 12.9885 4.4264 2.93 0.003 4.3130, 21.6641
+ 18 months 4.9052 3.3204 1.48 0.140 − 1.6025, 11.4130
Seers et al. Implementation Science (2018) 13:137 Page 5 of 11
Not all residents were able to complete or have a proxy
complete an EQ-5D so numbers completed are lower than
total number of residents.
+ 24 months 9.3776 4.3632 2.15 0.032 0.8259, 17.9292
Constant 33.7259 4.2278 7.98 0.000 25.4396, 42.0122
N = 2313; model fit: Wald χ2 (9)=1970.23, p < 0.001; post-estimation ICC 0.0910 (se 0.0219)
Primary outcomes—compliance with the four continence
recommendations
(Full details of all the components of each of the four
continence recommendations are available in the
Additional file 1: File S1). The ICC for percentage compli-
ance with recommendations has been calculated from the
baseline data, making allowance for both the clustering
and the unequal numbers from the 24 long-term care set-
tings. The ICC for percentage compliance with recom-
mendation 1 is 0.545 (95% CI 0.361, 0.730); for percentage
compliance with recommendation 2, the ICC is 0.404
(95% CI 0.220, 0.587), and for percentage compliance with
recommendation 3, ICC was 0.455 (95% CI 0.270, 0.641).
These ICCs are much higher than expected and those
usually found in Primary Health Care studies of 0.01 [24],
they are more similar to those found in some educational
cluster trials [27].
The results reported in Tables 2, 4 and 6 are from fit-
ting multi-level mixed-effect linear regressions models
to the compliance scores for each of the recommenda-
tions 1, 2 and 3 respectively. These models account for
the cluster design by treating site as a random effect and
adjusting the standard error for the 24 site clusters. The
model includes three independent variables: study arm
(three levels—control, type A and type B), country (four
levels—Netherlands, Sweden, Republic of Ireland and
England), time period (five levels—baseline, 6 months,
12 months, 18 months and 24 months). The first level
for each variable (control arm for intervention,
Netherlands for country and baseline for time) are taken
as the base level and other levels are compared to this.
In this model, we are considering the effect of interven-
tion allowing for country and time. The assumptions of
linear regression were examined, and there was no evi-
dence that the data failed to meet these assumptions. As
a sensitivity analysis, the linear regression models were
also fitted omitting the country covariate, and this did
not change any of the findings with regard to the signifi-
cance of the intervention effect.
Compliance with recommendation 1: The resident should
be actively screened for urinary incontinence
Compliance with recommendation 1 can range from 0 to
5 depending on which of five potential components of this
recommendation are documented. For each component
documented, one point is scored, percentage compliance
is a score out of 5 as a percentage. Table 2 reports the
model for compliance with recommendation 1 and shows
outcome scores in the intervention arms did not reach
statistical significance. Country is significant with Sweden
having poorer compliance (a negative coefficient)
compared to the Netherlands. Ireland and England had
significantly better compliance than the Netherlands
(positive coefficients). The 12- and 24-month data collec-
tion parameters were significant, but the other points were
not significantly different to baseline. The post-estimation
ICC following the fitting of this model for compliance
with recommendation 1 is 0.091. Table 3 shows the mean
percentage for each intervention group at each time point,
showing the small increase in percentage compliance
score for type A and type B intervention up to 12 months,
though as the regression model indicates there is no sig-
nificant difference in the study arms over the duration of
the study.
Compliance with recommendation 2: A detailed
assessment should be carried out
There are 15 items in the detailed assessment, so scores
can range from 0 to 15 for recommendation 2. Percent-
age compliance is a score out of 15 as a percentage.
Table 3 Mean percentage compliance with recommendation 1
by intervention group for each time point
Intervention
group
Mean score
Baseline 6 months 12 months 18 months 24 months
Control 28.4 22.3 29.2 27.0 23.2
Type A 19.2 21.5 38.8 30.6 35.5
Type B 14.1 17.0 44.4 23.4 28.7
N = 2313 residents are included in this analysis
Table 5 Mean percentage compliance with recommendation 2
by intervention group for each time point
Intervention
group
Mean score
Baseline 6 months 12 months 18 months 24 months
Control 37.5 34.6 36.5 34.1 34.4
Type A 34.6 35.1 45.1 39.7 44.6
Type B 35.3 34.8 43.2 38.2 45.9
N = 2313 residents are included in this analysis
Seers et al. Implementation Science (2018) 13:137 Page 6 of 11
Table 4 reports the fitted model for compliance with
recommendation 2. The intervention is not effective; nei-
ther the type A facilitation or type B facilitation interven-
tions had significant coefficients. Ireland was significantly
different having higher compliance with recommendation
2, but the coefficients for the other countries were not sig-
nificant, so England and Sweden are not significantly dif-
ferent to the Netherlands after allowing for time point and
intervention group. The 24-month data collection param-
eter is significant, with increased compliance by
24 months, but the other points are not significantly dif-
ferent to baseline. The post-estimation ICC following the
fitting of this model for compliance with recommendation
2 is 0.351. Table 5 shows mean percentage compliance
score for recommendation 2 by intervention group. Mean
percentage compliance was low at baseline, in all groups,
but improved by 24 months in the type A and type B
intervention groups.
Compliance with recommendation 3: An individualised
treatment plan should be in place
A score from 0 to 13 is possible for compliance with rec-
ommendation 3. Percentage compliance is a score out of
13 as a percentage.
Table 4 Multi-level mixed-effect linear regression
model—percentage compliance with recommendation 2
(a detailed assessment should be carried out), with adjustment
of standard errors to allow for clustering
Coefficient Std. Err. z p value 95% confidence
interval
Type A 5.6514 4.0014 1.41 0.158 − 2.1912, 13.4941
Type B 3.7903 4.4807 0.85 0.398 − 4.9917, 12.5724
Sweden − 1.9108 2.7374 − 0.70 0.485 − 7.2760, 3.4545
Ireland 14.9312 3.6627 4.08 0.000 7.7524, 22.1099
England 11.7997 7.0278 1.68 0.093 − 1.9745, 25.5738
+ 6 months − 0.2220 1.2763 − 0.17 0.862 − 2.7235, 2.2794
+ 12 months 3.3623 2.1118 1.59 0.111 − 0.7767, 7.5014
+ 18 months − 0.0031 1.6463 − 0.00 0.998 − 3.2298, 3.2235
+ 24 months 4.4827 2.1665 2.07 0.039 0.2364, 8.7290
Constant 30.1617 3.3204 9.08 0.000 23.6538, 36.6696
N = 2313; model fit: Wald χ2 (9) = 64.76, p < 0.001; post-estimation ICC 0.3517 (se 0.0758)
Table 6 reports the fitted model for compliance with
recommendation 3. The intervention was not effective,
neither the type A facilitation or type B facilitation
interventions had significant coefficients. All country pa-
rameters were significant with Sweden, Ireland and
England all having significantly higher compliance with
recommendation 3 than the Netherlands. All time points
were significant, and the parameter value increased for
each successive time period, thus suggesting improvement
over time in compliance with recommendation 3. This
suggests learning over time in all countries, but no signifi-
cant difference in the effectiveness of the three study in-
terventions. The post-estimation ICC following the fitting
of this model for compliance with recommendation 3 is
0.126. Table 7 shows mean percentage compliance for rec-
ommendation 3 by intervention group. It can be seen that
all three groups appear to improve over time, with little
difference between the interventions as indicated by the
regression model.
Recommendation 4: Specialist referral should be made if
necessary
There were very few specialist referrals made and in the
data collection it was not always clear whether a lack of
documentation meant no referral was made or whether
a referral was not necessary. It is therefore difficult to
fully assess compliance with this guideline. However, the
level of referral was so low that it is very unlikely that
study arm has a significant impact on compliance with
this recommendation. In only 4% of residents was a re-
ferral recommended. Although these referrals were re-
corded as specialist referrals, 17 were to a general
practitioner (family doctor) and 6 to an unknown spe-
cialist. There were only 11 referrals to a continence spe-
cialist nurse and 6 referrals to urology.
In summary, for the primary outcome (documented
compliance score or percentage compliance with contin-
ence recommendations), there was no significant differ-
ence between study arms; all study arms improved over
time in all countries.
Secondary (clinical) outcomes
These data are being considered as two cross-sectional
reviews of the resident populations in the long-term care
Table 6 Multi-level mixed-effect linear regression
model—percentage compliance with recommendation 3
(an individualised treatment plan should be in place), with
adjustment of standard errors to allow for clustering
Coefficient Std. Err. z p value 95% confidence
interval
Type A 0.3391 4.0168 0.08 0.933 − 7.5336, 8.2118
Type B 1.0372 3.0579 0.34 0.734 − 4.9562, 7.0305
Sweden 23.7959 1.9736 12.06 0.000 19.9278, 27.6640
Ireland 24.5448 3.9162 6.27 0.000 16.8692, 32.2204
England 15.3118 3.8489 3.98 0.000 7.7681, 22.8555
+ 6 months 9.8431 4.1862 2.35 0.019 1.6382, 18.0479
+ 12 months 14.2761 3.7488 3.81 0.000 6.9285, 21.6237
+ 18 months 15.9399 3.7804 4.22 0.000 8.5305, 23.3494
+ 24 months 19.9791 3.3984 5.88 0.000 13.3183, 26.6399
Constant 6.5831 3.0927 2.13 0.033 0.5216, 12.6446
N = 2313; model fit: Wald χ2 (9)=387.72, p < 0.001; post-estimation ICC 0.1265 (se 0.0502)
Seers et al. Implementation Science (2018) 13:137 Page 7 of 11
settings as there are very few individual residents in-
cluded at both baseline and 24 months data collection.
At 24 months, there was no significant difference
between the three intervention groups with regard to
the proportion of residents who had no documented
record of the assessment of cognition (p = 0.076 from
chi-square test). At 24 months, there was a significant
difference between the three intervention groups with
regard to the proportion of residents who had no
documented record of the level of cognitive impairment
(p < 0.001 from chi-squared test), the proportion being
higher in the control group than in the type A and type
B groups. At 24 months, there was a significant differ-
ence between the three intervention groups with regard
to the proportion of residents who had no documented
record of the assessment of depression (p = 0.017 from
chi-squared test), the proportion being higher in the
control group than in the type A and type B groups. At
24 months, there was no significant difference between
the three intervention groups with regard to the propor-
tion of residents who had no documented record of the
assessment of incontinence-related dermatitis (p = 0.479
from chi-square test).
Table 7 Mean percentage compliance with recommendation
3 by intervention group for each time point
Intervention
group
Mean score
Baseline 6 months 12 months 18 months 24 months
Control 20.9 30.8 40.9 45.0 48.9
Type A 23.8 32.2 41.9 42.7 45.2
Type B 26.7 38.6 40.7 41.1 45.9
N = 2313 residents are included in this analysis
Between baseline and 24 months, there was a statisti-
cally significant decrease in the proportion of residents
who had no documented record of an assessment of
cognition in type B facilitation (p < 0.001) but no signifi-
cant change for type A; there was a significant decrease
in the proportion of residents who have no documented
record of the level of cognitive impairment in interven-
tion type A (p < 0.001) and type B (p < 0.001); there was
a significant reduction in the percentage of residents
who had no documentation of assessment of depression
in the type A (p < 0.001) and type B (p < 0.001) groups.
There was a significant decrease in the percentage of
residents who had no documentation of incontinence-
associated dermatitis between baseline and 24 months in
the type A (p < 0.001) and type B (p < 001) groups.
There was no significant improvement in the control
group for any of the secondary outcomes.
Whether the impact of urinary incontinence on
quality of life been assessed was not documented for
the majority of residents. It was not assessed more
than seven times in any group, so this was not ex-
plored further. Very few UTIs were documented. In
the month prior to the baseline data collection, only
15 UTIs were recorded in all countries, decreasing to
only seven at the 24 month data collection point. No
further analysis was done.
It was not possible to reliably calculate the propor-
tion of residents in each long-term care setting with
incontinence, thus no further analysis was done. At
baseline, pelvic floor exercises were not used with
any residents, and at 24 month follow up pelvic,
floor exercises were only used with three residents.
With such low numbers, no further exploration of
this is sensible.
In summary, for secondary outcomes, both the facilita-
tion intervention groups (type A and type B) showed
significantly better documentation of three outcomes:
the level of cognitive impairment, depression and
incontinence-associated dermatitis between baseline and
24 months, and this improvement did not occur in the
standard dissemination (control) group. Clinically, this
change was not large, and a substantial proportion of
residents still had no documented assessment of level of
cognitive impairment (68% in type A and 65% in type B)
depression (61% in type A and 65% in type B) and
incontinence-associated dermatitis (66% in type A and
73% in type B).
There was a large amount of missing data on the
Urinary Incontinence Quality of Life (I-QoL) outcome
measure [21] as residents found it too much to
complete, so this is not reported further. It had been
planned to report length of stay data, but it was not
possible to collect this data consistently across all
sites, so it is not reported further.
Seers et al. Implementation Science (2018) 13:137 Page 8 of 11
Health economics
Health economic analysis was undertaken, but since
there was no significant difference in the primary
outcome between the intervention groups, these data
are not presented here in detail because the cost ana-
lysis showed that, as expected, standard dissemination
would be the least costly intervention to implement.
(see Additional file 1: File S3 for intervention cost tables).
Alberta Context Tool
For all concepts, higher scores represent a better work
context. All responses for a site (Nurse, LPN, HCA)
were considered together to provide an overall picture of
the site. The questionnaire completed by Nurses, LPN’s
and HCAs were identical except with regard to informal
interactions in which the HCA group had one less ques-
tion (9) than the other groups of staff who had 10 ques-
tions in this section.
Table 8 shows for each concept the mean score given by
all staff rating a site within the intervention arm. Formal
interactions are notably lower than other scores. The lar-
gest differences are for structural and electronic resources
and for organisational slack-space. On the basis of the
similarity of these mean scores, we conclude the study
groups were similar with regard to ACT concepts.
The 12 months type A and the 24 months type B facilita-
tion interventions did not have different levels of impact
on documented compliance with recommendations. It
was thus not possible to identify the type and “dose” of fa-
cilitation that worked best within the highly varied con-
textual conditions identified in this study. In addition, the
process evaluation revealed important issues about the
models of facilitation used and the characteristics of the
facilitators [17, 28].
Table 8 Mean scores on ACT concepts by intervention group at bas
ACT concepta Number of items Range for score C
Leadershipb 6 1–5 3
Cultureb 6 1–5 3
Feedbackb 6 1–5 3
Formal interactionsc 4 0–4 1
Informal interactionsc 9 or 10 0–10 3
Connections (social capital)b 6 1–5 4
Structural and electronic resourcesc 11 0–11 3
Organisational slack-staffingb 3 1–5 2
Organisational slack-spaceb 3 1–5 3
Organisational slack-timeb 4 1–5 2
aDefinitions of ACT concepts and scaling are provided [21, 22], and relevant papers
bScaled
cCount based
So why was it that the facilitation intervention did not
make a statistically significant difference to the docu-
mented implementation of continence recommendations?
Was an element of the PARIHS framework, facilitation,
purported to be necessary for getting research evidence
used in practice, actually not so important? Other research
has found some type of help with getting research imple-
mented does make a difference [29, 30]. Baskerville et al.’s
[31] systematic review of practice facilitation in primary
care suggests facilitation improves uptake of clinical prac-
tice guidelines by nearly three times. A facilitation inter-
vention was found to reduce neonatal mortality by 50%
[32]. Although the facilitation not working in this study is
a possible explanation and the high ICCs meant the study
was underpowered, the process evaluation qualitative re-
search evidence [17] suggested this was not the most likely
explanation. It may be facilitation works differently along
the continuum of context. It could be that using only doc-
umented evidence of compliance with the recommenda-
tions under-estimated what might have happened in
practice but was not documented. A lack of intervention
fidelity is another possible explanation [17].
Although the intervention groups improved, it was not
possible to say the improvement was due to the inter-
vention as the control group also improved. We do not
know why this was, but it could be that for control sites,
being in the study, including six monthly follow-ups for
2 years, was enough of an incentive to improve. How-
ever, the qualitative data suggests for most control sites
they did not use the written recommendations or the
implementation guide. One site mentioned to the re-
searcher that they checked their documentation and
practice knowing the researcher would be visiting, and
thus even collecting follow-up data in the control group
can be seen as having an effect.
Etheridge et al. [33] concluded that four active ingredi-
ents were required to effect change in long-term care
eline (N = 725 staff are included in this analysis)
ontrol sites Mean (SD) Type A sites Mean (SD) Type B sites Mean (SD)
.6 (0.81) 3.7 (0.82) 3.7 (0.76)
.9 (0.65) 3.9 (0.57) 3.9 (0.61)
.5 (0.79) 3.4 (0.82) 3.4 (0.85)
.3 (1.14) 1.1 (1.08) 1.2 (1.13)
.5 (2.11) 3.2 (2.08) 3.3 (2.04)
.0 (0.67) 3.8 (0.59) 3.9 (0.59)
.1 (2.34) 3.4 (2.14) 2.8 (1.89)
.7 (1.13) 2.8 (1.09) 2.6 (1.00)
.6 (1.01) 3.1 (1.14) 3.3 (1.10)
.8 (0.69) 2.8 (0.70) 2.8 (0.74)
are listed at https://trecresearch.ca/alberta_context_tool
https://trecresearch.ca/alberta_context_tool
Seers et al. Implementation Science (2018) 13:137 Page 9 of 11
settings: urgency, solidarity, intensity and accumulation.
The continence programme they reviewed failed, and
one of the reasons they identified may also apply in our
study: there was no buy-in from participants. Although
all sites agreed to take part in the study, the topic area
and the intervention were already decided. In addition,
participants changed during the study, so, for example,
as managers changed, new managers did not necessarily
see this study as a priority, thus reducing even further
the extent of organisational buy-in and support [17].
The proposition that underpins the PARIHS frame-
work is that successful implementation is a function
of the nature of the evidence being implemented, the
context into which it is being implemented and ap-
propriate facilitation to help people implement the
evidence. There was no weighting given to these three
aspects of evidence, context and facilitation. This re-
search suggested that facilitation with one or two
people in a team may not easily overcome contextual
factors. The level of experience and expertise of the
IF, and relationship of the IF to managers in the set-
ting, may be more important [34] as may unravelling
how facilitation and context interact.
It was not possible to identify a “good enough” model
of facilitation that affected the primary outcome (docu-
mented compliance with continence recommendations)
and could address the different contexts. Facilitation did
however result in some identifiable practice changes (e.g.
new assessment processes, new forms and awareness of
the impact of incontinence on residents).
It may be that in practice, tailoring the type of facilita-
tion to both the setting and the internal facilitator is im-
portant. Just how one could map the contextual
characteristics to a type of facilitation and to type of in-
ternal facilitator would need further evaluation. Van der
Zijpp et al. [34], part of this study, argued the interac-
tions between managerial leaders and IFs were import-
ant, summarised by three themes: realising commitment,
negotiating conditions and encouraging to keep the mo-
mentum going. The reciprocal relationships between
managers and IFs influenced the process of implementa-
tion, and future interventions should target managers in
a focused way. In studies that evaluate implementation
of complex interventions such as facilitation, it may be
appropriate to adopt a theoretical perspective on fidelity,
focusing on the intended mechanisms of the interven-
tion. For example, in this study, the theory of type A fa-
cilitation required IFs to develop skills and confidence in
audit and feedback. Achieving this mechanism, even if it
meant IFs needed varying levels of external facilitation,
would demonstrate theoretical fidelity. This type of ap-
proach has been proposed in public health [35] and is
discussed in the linked papers (Rycroft-Malone et al.
[17], Harvey et al. [28]).
ACT considers organisational concepts as a unit-based
score. In this study, these were considered as site level
variables. Mean baseline and follow-up mean scores
were compared with either an ANOVA where multiple
time points were available or with a t-test when only one
follow-up time point was available. There were very few
changes that were significant. We are thus not confident
to make any claims about the effects of the intervention
on organisational culture as assessed with ACT. Possible
explanations for this include the organisations were
stable and at site level the concepts were unaffected by
the interventions.
Limitations
In reality, the planned interventions did not always work
as originally envisaged, as revealed by the process evalu-
ation [17] and our analysis of the facilitation interven-
tion [28]. This was for several reasons, relating to initial
selection and preparation of the IFs; engagement in the
facilitation intervention; ability to progress according to
plan. The linked papers illustrate the issues that compro-
mised the fidelity of the intervention [17, 28]. It was also
challenging to recruit resident participants in some
homes, so we had fewer than planned. In addition, al-
though each of the long-term settings had agreed to take
part, for individual staff within the home it was not ne-
cessarily a priority. The unexpectedly high ICC meant
the study was underpowered. Although we felt the ICC
we used in the sample size calculation was reasonable, in
planning future cluster RCTs with a more educational
focus, it is important to be aware that not all ICCs will
be as low as those reported for recent primary care trials
[24]. In the design of the study, it was assumed that
there would not be large country differences regarding
compliance with the recommendations; hence, it would
be viable to have a small number of sites from each
country in each study arm. In practice, it appears the
countries are behaving differently, but the study was not
powered to investigate within country effect of the dif-
ferent interventions on the primary outcome.
Conclusions
Pressman and Wildavsky [36] a long time ago reported that
“the study of implementation requires understanding that
apparently simple sequences of events depend on complex
chains of reciprocal interaction” (pxvii) and referred to the
complexities of implementation as “the lumpy stuff of life”
[37] (p165). This study supports those assertions.
This was the first cross European randomised controlled
trial with embedded process and economic evaluations
that sought to test different methods of facilitation. There
was no significant difference in the primary outcome be-
tween any of the three study arms. It found both models
Seers et al. Implementation Science (2018) 13:137 Page 10 of 11
of facilitation were broadly viable but were not signifi-
cantly better than a control in improving documented
compliance with recommendations to promote contin-
ence. Contextual issues were not always overcome by the
approaches to facilitation adopted in this study as our
linked papers demonstrate (Rycroft-Malone et al., Harvey
et al. [17, 28]).
Additional file
Additional file 1: File S1 Underpinning theories and activities of type A
and type B facilitation. File S2 List of all components of the continence
recommendations. File S3 FIRE programme costs. File S4 Study Flow
Diagram, Consort Checklist and Tidier checklist. (DOCX 71 kb)
EF: External facilitator; FIRE: Facilitating Implementation of Research Evidence;
IF: Internal facilitator; PARIHS: Promoting Action on Research Implementation
in Health Services; UI: Urinary incontinence; UTI: Urinary tract infection
Thank you to all those who were involved in this study as participants or
who advised us on the study.
The research leading to these results has received funding from the European
Union’s Seventh Framework Programme (FP7/2007–2013) under grant
agreement no. 223646.
The datasets generated or analysed during the current study are not publicly
available because consent to make data publicly available was not part of
the consent by participants.
All authors read and approved the final manuscript. KS (principal investigator)
led the application for funding. She contributed to the overall design of the
study, designed the cluster RCT aspect of the study, and contributed to the
analysis. She led the writing of this paper. JRM (collaborator) participated in
designing the study. She led the design of the evaluation package and was
country co-ordinator for England. She wrote the process evaluation aspects
of this paper and reviewed the manuscript critically for important intellectual
content. KC (collaborator) participated in designing the study and reviewed
the manuscript critically for important intellectual content. She was country
co-ordinator for the Netherlands. NC (statistician) advised on study design,
contributed to the analysis plan and undertook the analysis within the cluster
RCT, and reviewed the manuscript critically for important intellectual content.
RTE and CJ (health economists) were responsible for the economic evaluation
study design and reviewed the manuscript critically for important intellectual
content. ACE (research fellow) participated in the design and analysis of the
evaluation package, collected data in Sweden, contributed to the analysis and
reviewed the manuscript critically for important intellectual content. CAE
(collaborator) participated in study design and coordinated the use of the
Alberta Context Tool including its translation into Swedish and Dutch.
She reviewed the manuscript critically for important intellectual content. CH
(research fellow) participated in the design of the process evaluation and
associated data collection tools, the development of the economic evaluation,
and was responsible for the day to day running of the process evaluation,
contributed to the data collection in England and the analysis and reviewed
the manuscript critically for important intellectual content. GH (collaborator)
participated in the design of the overall study and in the design of the facilitator
intervention in particular. She co-led type A facilitation work package and
reviewed the manuscript critically for important intellectual content. AK
(collaborator) participated in the design of the overall study and in the design
of the facilitator intervention in particular. She co-led type A facilitation work
package and reviewed the manuscript critically for important intellectual
content. BMcC (collaborator) participated in the design of the overall study
and in the design of the facilitation intervention in particular. He co-led type B
facilitation work package and reviewed the manuscript critically for important
intellectual content. He was also country coordinator for Ireland. CM (research
fellow) participated in the design of the RCT and associated data collection
tools, participated in the collection of data in England, contributed to the
analysis and reviewed the manuscript critically for important intellectual
content. AT (collaborator) participated in the design of the overall study,
especially the Type B facilitation intervention. She co-led type B facilitation
work package and reviewed the manuscript critically for important intellectual
content. PS and CMcC (research fellows) participated in the collection of data in
Republic of Ireland, contributed to the analysis and reviewed the manuscript
critically for important intellectual content. TN and TvdZ (research fellows)
participated in the collection of data in Netherlands, contributed to the analysis
and reviewed the manuscript critically for important intellectual content. LW
(collaborator) participated in the design of the overall study and in the design
of the intervention evaluation in particular. He reviewed the manuscript critically
for important intellectual content and was Country Coordinator for Sweden.
Ethical Committee approval was obtained in England (10/WSE04/20),
Sweden (2009/1806-31/2) and Republic of Ireland (ECM4(u)02/03/10). In the
Netherlands, the researchers followed advice to get permission from either
an ethical committee at site level, or where this did not exist, from a scientific
or residents committee at the site (HAZ-11087777-JGS). Research Governance
approval was also obtained in England and permission to collect data at the
sites obtained in Sweden and Republic of Ireland.
Consent form allowed the use of anonymised quotations in publications.
We acknowledge that CE is involved in the development of the Alberta
Context Tool and AK GH JRM BMcC KS and AT have all been involved in the
development of the PARIHS framework.
Springer Nature remains neutral with regard to jurisdictional claims in
published maps and institutional affiliations.
1Warwick Research in Nursing, Warwick Medical School, University of
Warwick, Coventry, UK. 2School of Health Care Sciences, Bangor University,
Bangor, UK. 3Fontys School of People and Health Studies, Fontys University
of Applied Sciences, Eindhoven, The Netherlands. 4School of Health and
Social Care, London South Bank University, 103 Borough Road, London SE1
0AA, UK. 5Centre for Health Economics and Medicines Evaluation (CHEME),
Bangor University, Bangor, UK. 6Faculty of Medicine and Health Science,
Department of Nursing, Linkoping University, Linkoping, Sweden.
7Department of Neurobiology, Care Sciences and Society, Division of
Nursing, Karolinska Institutet, Stockholm, Sweden. 8Faculty of Nursing,
University of Alberta, Edmonton, Alberta, Canada. 9Adelaide Nursing School,
University of Adelaide, Adelaide, Australia. 10Clinical Trials Unit, Warwick
Medical School, University of Warwick, Coventry, UK. 11College of Nursing
and Health Sciences, Flinders University, Adelaide, South Australia, Australia.
12Division of Nursing, Queen Margaret University, Edinburgh, UK. 13Institute
of Applied Health Research, University of Birmingham, Birmingham, UK.
14Warwick Medical School, University of Warwick, Coventry, UK. 15Institute of
Nursing and Health Research, Ulster University, Shore Rd, Belfast, Northern
Ireland. 16School of Education, Health and Social Studies, Dalarna University,
Falun, Sweden. 17Department of Health and Care Sciences, The Sahlgrenska
Academy, University of Gothenburg, Gothenburg, Sweden.
Received: 13 April 2018 Accepted: 22 October 2018
1. Grol R. Success and failures in the implementation of evidence-based
guidelines for clinical practice. Med Care. 2001;39(8 Suppl 2):1146–54.
2. McGlynn EA, Asch SM, Adams J, Keesey J, Hicks J, DeCristofaro A, Kerr EA.
The quality of care delivered to adults in the United States. N Engl J Med.
2003;348(26):2635–45.
https://doi.org/10.1186/s13012-018-0831-9
Seers et al. Implementation Science (2018) 13:137 Page 11 of 11
3. Katikreddi SV, Higgins M, Bond L, Bonnell C, Macintyre S. How evidence
based is English public health policy? BMJ. 2011;343:d7310.
4. Kitson A, Rycroft-Malone J, Harvey G, McCormack B, Seers K, Titchen A.
Evaluating the successful implementation of evidence into practice using
the PARIHS framework: theoretical and practical challenges. Implement Sci.
2008;3:1. https://doi.org/10.1186/1748-5908-3-1.
5. Seers K, Cox K, Crichton NJ, Tudor-Edwards R, Eldh A, Estabrooks CA, Harvey
G, Hawkes C, Kitson A, Linck P, McCarthy G, McCormack B, Mockford C,
Rycroft-Malone J, Titchen A, Wallin L. FIRE (Facilitating Implementation of
Research Evidence): a study protocol. Implement Sci. 2012;7:25. https://doi.
org/10.1186/1748-5908-7-25.
6. Seers K, Cox K, Crichton NJ, Tudor-Edwards R, Eldh A, Estabrooks CA, Harvey
G, Hawkes C, Kitson A, Linck P, McCarthy G, McCormack B, Mockford C,
Rycroft-Malone J, Titchen A, & Wallin L. (2013) Final report summary – FIRE
(Facilitating Implementation of Research Evidence). http://cordis.europa.eu/
result/rcn/149765_en.html. Accessed 3 Apr 2018.
7. Eccles MP, Armstrong D, Baker R, Cleary K, Davies H, Davies S, Glasziou P,
Ilott I, Kinmonth AL, Leng G, Logan S, Michie S, Rogers H, Rycroft-Malone J,
Sibbald B. An implementation research agenda. Implement Sci. 2009;4:18.
https://doi.org/10.1186/1748-5908-4-18.
8. Grimshaw JM, Thomas RE, MAcLennan G, Fraser C, Ramsay CR, Vale L,
Whitty P, Eccles M, Matowe L, Shirran L, Wensing M, Dijkstra R, Donaldson
C. Effectiveness and efficiency of guideline dissemination and
implementation strategies. Health Technol Assess. 2004;8:6.
9. Thompson DS, Estabrooks CA, Scott-Findlay S, Moore K, Wallin L.
Interventions aimed at increasing research use in nursing: a systematic
review. Implement Sci. 2007;2:15 http://www.implementationscience.com/
content/pdf/1748-5908-2-15 .
10. Medves J, Godfrey C, Turner C, Paterson M, Harrison M, MacKenzie L, Durando
P. Systematic review of practice guidelines dissemination and implementation
strategies for healthcare teams and team-based practice. Int J Evidence Based
Healthcare. 2010;8:78–89. https://doi.org/10.1136/bmj.a2390.
11. Dogherty EJ, Harrison MB, Graham ID. Facilitation as a role and process in
achieving evidence-based practice in nursing: a focused review of concept
and meaning. Worldviews Evidence Based Nursing. 2010;7(2):76–89.
12. Berta W, Cranley L, Dearing JW, Dogherty EJ, Squires JE, Estabrooks CA. Why
(we think) facilitation works: insights from organizational learning theory.
Implement Sci. 2015;10:141. https://doi.org/10.1186/s13012-015-0323-0.
13. Brittain KR, Shaw C. The social consequences of living with and dealing
with incontinence – a carers perspective. Soc Sci Med. 2007;65:1274–83.
14. Ouslander JG, Schnelle JF. Incontinence in the nursing home. Ann Intern
Med. 1995;122(6):438–49.
15. National Guideline Clearinghouse. Guideline for Urinary Incontinence in the
long term setting. American Medical Directors Association. Agency For
Healthcare Research and Quality. 2012; http://www.guideline.gov/content.
aspx?id=45526. Archived. https://www.ahrq.gov/professionals/clinicians-
providers/guidelines-recommendations/archive.htm. Accessed 3 Apr 2018.
16. Perry L, Bellchambers H, Howie A, Moxey A, Parkinson L, Capra S, Byles J.
Examination of the utility of the Promoting Action on Research
Implementation in health Services framework for implementation of
evidence based practice in residential aged care settings. Journal of
Advanced Nursing. 2011;67(10):2139–50.
17. Jo Rycroft-Malone; Kate Seers; Ann Catrine Eldh; Karen Cox; Nicola Crichton;
Gill Harvey; Claire Hawkes; Alison Kitson; Brendan George McCormack;
Christel McMullan; Carole Mockford; Theo Niessen; Paul Slater; Angie
Titchen; Teatske van der Zijpp; Lars Wallin. A realist process evaluation
within the Facilitating Implementation of Research Evidence (FIRE) cluster
randomised controlled international trial: an exemplar. Implement Sci.
https://doi.org/10.1186/s13012-018-0811-0.
18. Rycroft-Malone J, Seers K, Crichton N, Chandler J, Hawkes C, Allen C, Bullock
I, Strunin L. A pragmatic randomised controlled trial evaluating three
implementation interventions. Implement Sci. 2012;7:80.
19. DuBeau CE, Kuchel GA, Johnson T, Palmer MH, Wagg A. Committee 11.
Incontinence in the frail elderly. In: Abrams P, Cardozo L, Khoury S, Wein A,
editors. Incontinence. 4th International Consultation on Incontinence:
Health Publications Limited; 2009. pp961–1024 and pp1796–1789. http://
www.icsoffice.org/Publications/ICI_4/book . Accessed 3 Apr 2018.
20. Devlin N, Parkin D, Browne J. Using the EQ-5D as a performance
measurement tool in the NHS. Health Econ. 2010;19(8):886–905.
21. Patrick DL, Martin ML, Bushnell DM, Yalcin I, Wagner TH, Buesching DP.
Quality of life with women with urinary incontinence: further development
of the incontinence quality of life instrument (I-QOL). Urology. 1999;53:71–6.
22. Estabrooks CA, Squires JE, Cummings GG, Birdsell JM, Norton PG.
Development and assessment of the Alberta Context Tool. BMC Health Serv
Res. 2009;9:234. https://doi.org/10.1186/1472-6963-9-234.
23. Squires JE, Hayduk L, Hutchinson AM, Mallick R, Norton PG, Cummings GG,
Estabrooks CA. Reliability and validity of the Alberta Context Tool (ACT) with
professional nurses: findings from a multi-study analysis. PLoS One. 2015;
10(6):e0127405.
24. Campbell MK, Mollison J, Steen N. Grimshaw JM & Eccles MAnalysis of
cluster randomized trials in primary care: a practical approach. Fam Pract.
2000;17:192–6.
25. Donner A, Klar N. Design and analysis of cluster randomized trials in health
research. London: Arnold; 2000.
26. Pawson R. The science of evaluation: a realist manifesto. London: Sage; 2013.
27. Brooks G, Burton M, Cole P, Miles J, Torgerson C, Torgerson D. Randomised
controlled trial of incentives to improve attendance at adult literacy classes.
Oxf Rev Educ. 2008;34:493–504.
28. Harvey G McCormack B Kitson A Lynch E Titchen A. Designing and
implementing two facilitation interventions within the ‘Facilitating
Implementation of Research Evidence (FIRE)’ study: a qualitative analysis
from an external facilitators’ perspective. Implement Sci. https://doi.org/10.
1186/s13012-018-0812-z.
29. Eriksson L, Huy TQ, Duc DM, Ekholm Selling K, Hoa DP, Thuy NT, Nga NT,
Wallin L. Process evaluation of a knowledge translation intervention using
facilitation of local stakeholder groups to improve neonatal survival in the
Quang Ninh province, Vietnam. Trials. 2016;17:23.
30. McCormack B, Rycroft-Malone J, DeCorby K, Hutchinson AM, Bucknall T,
Kent B, Schultz A, Snelgrove-Clarke E, Stetler C, Titler M, Wallin L, Wilson L. A
realist review of interventions and strategies to promote evidence-informed
healthcare: a focus on change agency. Implement Sci. 2013;8:107. https://
doi.org/10.1186/1748-5908-8-107.
31. Baskerville B, Liddy C, Hogg W. Systematic review and meta-analysis of practice
facilitation within primary care settings. Ann Fam Med. 2012;37(8):63–74.
32. Persson LÅ, Nguyen TN, Målqvist M, Dinh Thi Phuong H, Eriksson L,
Wallin L, Tran QH, Duong MD, Tran VT, Vu Thi Thu T, Ewald U. Effect
of facilitation of local maternal-and-child stakeholder groups on
neonatal mortality. The NeoKIP cluster-randomised trial in Quang
Ninh province, Vietnam. PLoS Med. 2013:10(5) https://doi.org/10.1371/
journal.pmed.1001445.
33. Etheridge F, Couturier Y, Denis JL, Tremblay L, Tannenbaum C. Explaining the
success or failure of quality improvement initiatives in long term care
organisations from a dynamic perspective. J Appl Gerontol. 2014;33(6):672–89.
34. Van der Zijpp TJ, Niessen T, Eldh A, Hawkes C, McMullan C, Mockford C,
Wallin L, McCormack B, Rycroft-Malone J, Seers K. A bridge over turbulent
waters – illustrating the interaction between managerial leaders and
facilitators when implementing research evidence. Worldviews Evid-Based
Nurs. 2016;13(1):25–31. https://doi.org/10.1111/wvn.12138.
35. Hawe P. Lessons from complex interventions to improve health. Annu Rev
Public Health. 2015;36:307–23.
36. Pressman JL, Wildavsky A. Implementation, How great expectations in
Washington are dashed in Oakland or why it’s amazing that federal programs
work at all this being a saga of the economic development administration as
told by two sympathetic observers who seek to build morals on a Foundation
of Ruined Hopes. Berkeley: University of California Press; 1973.
37. Pressman JL, Wildavsky A. Implementation, How great expectations in
Washington are dashed in Oakland; or why it’s amazing that federal programs
work at all this being a saga of the economic development administration as
told by two sympathetic observers who seek to build morals on a Foundation
of Ruined Hopes. 3rd ed. Berkeley: University of California Press; 1984.
https://doi.org/10.1186/1748-5908-3-1
https://doi.org/10.1186/1748-5908-7-25
https://doi.org/10.1186/1748-5908-7-25
http://cordis.europa.eu/result/rcn/149765_en.html
http://cordis.europa.eu/result/rcn/149765_en.html
https://doi.org/10.1186/1748-5908-4-18
http://www.implementationscience.com/content/pdf/1748-5908-2-15
http://www.implementationscience.com/content/pdf/1748-5908-2-15
https://doi.org/10.1136/bmj.a2390
https://doi.org/10.1186/s13012-015-0323-0
http://www.guideline.gov/content.aspx?id=45526
http://www.guideline.gov/content.aspx?id=45526
https://www.ahrq.gov/professionals/clinicians-providers/guidelines-recommendations/archive.htm
https://www.ahrq.gov/professionals/clinicians-providers/guidelines-recommendations/archive.htm
https://doi.org/10.1186/s13012-018-0811-0
http://www.icsoffice.org/Publications/ICI_4/book
http://www.icsoffice.org/Publications/ICI_4/book
https://doi.org/10.1186/1472-6963-9-234
https://doi.org/10.1186/s13012-018-0812-z
https://doi.org/10.1186/s13012-018-0812-z
https://doi.org/10.1186/1748-5908-8-107
https://doi.org/10.1186/1748-5908-8-107
https://doi.org/10.1371/journal.pmed.1001445
https://doi.org/10.1371/journal.pmed.1001445
https://doi.org/10.1111/wvn.12138
-
Abstract
Background
Methods
Results
Conclusions
Trial registration
Background
Aims
Methods
Design
Participants
Setting
The intervention
Outcomes measures
Secondary outcomes
Sample size and power calculations
Randomisation sequence generation, allocation concealment, implementation and blinding
Quantitative analysis—statistical methods
Qualitative analysis
Findings
Description of resident sample
Primary outcomes—compliance with the four continence recommendations
Compliance with recommendation 1: The resident should be actively screened for urinary incontinence
Compliance with recommendation 2: A detailed assessment should be carried out
Compliance with recommendation 3: An individualised treatment plan should be in place
Recommendation 4: Specialist referral should be made if necessary
Secondary (clinical) outcomes
Health economics
Alberta Context Tool
Discussion
Limitations
Conclusions
Additional file
Abbreviations
Acknowledgements
Funding
Availability of data and materials
Authors’ contributions
Ethics approval and consent to participate
Consent for publication
Competing interests
Publisher’s Note
Author details
References
Management of the effects of psychological trauma experienced as a child
in clinical practice
According to van Dijke et al. (2017), traumatic events in childhood, such as neglect,
physical, sexual, and emotional abuse, have been associated with borderline personality
disorder (BPD). These traumatic events have been identified in many adults diagnosed with
BPD and manifest in affect and mood dysregulation, dissociation, poor interpersonal
effectiveness, and distress tolerance (van Dijke et al., 2017). The impacts of this diagnosis
are wide-ranging however, people with BPD tend to experience difficulties in daily
functioning due to heightened emotional sensitivity and struggle in managing impulsive
behaviours (Acres et al., 2021). BPD is also associated with high rates of self-harming
behaviours, and death by suicide rates are between 3% and 10% among this cohort of people
(Acres et al., 2021).
In order to address the psychological trauma and provide treatment, clinicians need to
have the knowledge and depth of skills founded on evidence-based practice when working
with people with BPD (Choi, 2018). Evidence-based practice combines patient experience
and values, current research, professional expertise, and clinical judgment to provide the best
available treatment options (Aveyard & Sharpe, 2009).
One of the most common treatments for BPD is Dialectical Behaviour Therapy (DBT),
which is an evidence-based form of skills therapy for people experiencing BPD or complex
behavioural difficulties due to past trauma (Dimeff & Linehan, 2001). DBT addresses
distress tolerance, emotion regulation and interpersonal relationships and can be used as an
effective form of psychotherapy that could strengthen individual resilience, psychological
wellbeing, and behaviours (Choi, 2018). Research evidence has also shown that DBT lessens
self-harming behaviours, inpatient admissions, suicidal acts, and an overall improvement in
mood, anger, dissociation, and global functioning (Dimeff & Linehan, 2011).
People with BPD tend to present frequently at emergency departments due to requiring
treatment for self-harming injuries, heightened state of distress and dysregulation and suicidal
ideation (Acres et al., 2021). Despite the evidence for treatment, patients with BPD diagnosis
and their carers are still reporting gaps in the care they receive, with restrictive practices such
as seclusion, physical and chemical restraint continue to be used in emergency departments
and inpatient units (Acres et al., 2021). These types of practices often intensify the distress
and previous experiences of trauma, thus leading to harmful impacts and poor treatment
trajectory (Acres et al., 2021). Additionally, patients with BPD typically access simplistic
forms of treatment or are treated under a medical model with medications to manage their
anxiety, mood fluctuations, emotional dysregulation, and anger (Choi, 2018). These
approaches might provide temporary relief and a sense of safety, however they do not address
the past and current trauma and circumstances that maintain the presenting problems (Choi,
2018).
These trends indicate a gap between evidence-based treatment and what patients and their
carers experience when accessing the healthcare system when seeking treatment for BPD.
Acres et al. (2021) found that health professionals, particularly nurses in emergency
departments, did not have the relevant training to identify and address the crisis patients with
BPD presented. Furthermore, due to this lack of training and awareness, there is a culture
perpetuated in emergency departments that people with BPD are not presenting with a
disease, therefore they do not require the same level of care and treatment as other ill people
(Acres et al., 2021).
Another gap identified in research by Giffin (2008) is the lack of communication and
understanding of the health service system and where to access evidence-based treatment.
Patients and their carers are often faced with calling multiple services to access assistance,
not understanding the role and capacity of each service (Giffin, 2008). This research also
highlighted inconsistency and contradictory advice health care professionals gave due to
diagnostic uncertainty or confusion (Giffin, 2008). As a result, patients and their carers faced
inconsistency, contradictory advice, incomplete and ad hoc treatment plans, and an overall lack of
support (Giffin, 2008).
Additionally, Linehan (2015) identified that DBT is a time-consuming process for the mental
health services due to DBT requiring adequate training, ongoing availability, and commitment for
over 12 months for weekly group and individual therapy and engaging in weekly consultation with
fellow DBT therapists. This level of commitment can prove to be a resource drain on public mental
health services, often resulting in DBT not being provided as a treatment option (Linehan, 2015).
In conclusion, this paper has demonstrated that BPD is a severe mental illness that causes a high
rate of suicide, self-harming behaviours, and poor global functioning. It is a condition that produces
heightened distress for patients and their carers and has a high burden of disease cost to the health
services. Although an evidence-based treatment is available to these patients, there are still
significant gaps where practice lags behind the evidence.
References
Acres, K., Loughhead, M., & Procter, N. (2021). From the community to the emergency
department: A study of hospital emergency department nursing practices from the
perspective of carers of a loved one with borderline personality disorder. Health &
Social Care in the Community (00), 1-9. https://doi.org/10.1111/hsc.13558
Aveyard, H., & Sharpe, P. (2009). Beginner’s guide to evidence-based practice in health
and social care professions: A beginner’s guide. McGraw-Hill Education.
Choi, H. (2018). Family systemic approaches for borderline personality disorder in acute
adult mental health care settings. Australian and New Zealand Journal of Family
Therapy, 39(2), 155–173. https://doi.org/10.1002/anzf.1308
Dimeff, L. & Linehan, M.M. (2001). Dialectical behaviour therapy in a nutshell. The
California Psychologist, 34, 10-13.
Giffin, J. (2008). Family experience of borderline personality disorder. Australian and New
Zealand Journal of Family Therapy, 29(3), 133–138.
https://doi.org/10.1375/anft.29.3.133
Linehan, M. (2015). DBT skills training manual (Second edition.). The Guilford Press.
Van Dijke, A., Hopman, J. A., and Ford, J. D. (2017). Affect dysregulation, adult attachment
problems, and dissociation mediate the relationship between childhood trauma and
borderline personality disorder symptoms in adulthood. European Journal of
Trauma & Dissociation 2(2), 91–99. https://doi.org/10.1016/j.ejtd.2017.11.002
https://doi.org/10.1111/hsc.13558
https://doi.org/10.1002/anzf.1308
https://doi.org/10.1375/anft.29.3.133
https://doi.org/10.1016/j.ejtd.2017.11.002
Order an Essay Now & Get These Features For Free:
Turnitin Report
Formatting
Title Page
Citation
Outline
