The translation of research into practices has been incomplete. Organizational readiness for change (ORC) is a potential facilitator of effective knowledge translation (KT). However we know little about the best way to assess ORC. Therefore, we sought to systematically review ORC measurement instruments.
We searched for published studies in bibliographic databases (Pubmed, Embase, CINAHL, PsychINFO, Web of Science, etc.) up to November 1st, 2012. We included publications that developed ORC measures and/or empirically assessed ORC using an instrument at the organizational level in the health care context. We excluded articles if they did not refer specifically to ORC, did not concern the health care domain or were limited to individual-level change readiness. We focused on identifying the psychometric properties of instruments that were developed to assess readiness in an organization prior to implementing KT interventions in health care. We used the Standards for Educational and Psychological Testing to assess the psychometric properties of identified ORC measurement instruments.
We found 26 eligible instruments described in 39 publications. According to the Standards for Educational and Psychological Testing, 18 (69%) of a total of 26 measurement instruments presented both validity and reliability criteria. The Texas Christian University –ORC (TCU-ORC) scale reported the highest instrument validity with a score of 4 out of 4. Only one instrument, namely the Modified Texas Christian University – Director version (TCU-ORC-D), reported a reliability score of 2 out of 3. No information was provided regarding the reliability and validity of five (19%) instruments.
Our findings indicate that there are few valid and reliable ORC measurement instruments that could be applied to KT in the health care sector. The TCU-ORC instrument presents the best evidence in terms of validity testing. Future studies using this instrument could provide more knowledge on its relevance to diverse clinical contexts.
Health-care organizations need to be ready prior to implement evidence-based interventions. In this study, we sought to achieve consensus on a framework to assess the readiness of health-care organizations to implement evidence-based interventions in the context of chronic care.
We conducted a web-based modified Delphi study between March and May 2013. We contacted 76 potentially eligible international experts working in the fields of organizational readiness (OR), knowledge translation (KT), and chronic care to comment upon the 76 elements resulting from our proposed conceptual map. This conceptual map was based on a systematic review of the existing frameworks of Organizational Readiness for Change (ORC) in health-care. We developed a conceptual map that proposed a set of core concepts and their associated 17 dimensions and 59 sub-dimensions. Experts rated their agreement concerning the applicability and importance of ORC elements on a 5-point Likert scale, where 1 indicates total disagreement and 5 indicates total agreement. Two rounds were needed to get a consensus from the experts. Consensus was a priori defined as strong (≥75%) or moderate (60-74%). Simple descriptive statistics was used.
In total, 14 participants completed the first round and 10 completed the two rounds. Panel members reached consensus on the applicability and importance of 6 out of 17 dimensions and 28 out of 59 sub-dimensions to assess OR for KT in the context of chronic care. A strong level of consensus (≥75%) was attained on the Organizational contextual factors, Leadership/participation, Organizational support, and Motivation dimensions. The Organizational climate for change and Change content dimensions reached a moderate consensus (60-74%). Experts also reached consensus on 28 out of 59 sub-dimensions to assess OR for KT. Twenty-one sub-dimensions reached a strong consensus (≥75%) and seven a moderate consensus (60-74%).
This study results provided the most important and applicable dimensions and sub-dimensions for assessing OR-KT in the context of chronic care. They can be used to guide the design of an assessment tool to improve knowledge translation in the field of chronic care.
Organizational readiness; Delphi study; Knowledge translation; Chronic care; Measurement
To demonstrate the benefit of defining operational management units in nursing homes and computing quality indicators on these units as well as on the whole facility.
Calculation of adjusted Resident Assessment Instrument – Minimum Data Set 2.0 (RAI–MDS 2.0) quality indicators for: PRU05 (prevalence of residents with a stage 2–4 pressure ulcer), PAI0X (prevalence of residents with pain) and DRG01 (prevalence of residents receiving an antipsychotic with no diagnosis of psychosis), for quarterly assessments between 2007 and 2011 at unit and facility levels. Comparisons of these risk-adjusted quality indicators using statistical process control (control charts).
A representative sample of 30 urban nursing homes in the three Canadian Prairie Provinces.
Explicit decision rules were developed and tested to determine whether the control charts demonstrated improving, worsening, unchanging or unclassifiable trends over the time period. Unit and facility performance were compared.
In 48.9% of the units studied, unit control chart performance indicated different changes in quality over the reporting period than did the facility chart. Examples are provided to illustrate that these differences lead to quite different quality interventions.
Our results demonstrate the necessity of considering facility-level and unit-level measurement when calculating quality indicators derived from the RAI–MDS 2.0 data, and quite probably from any RAI measures.
Quality of Care; Performance Measurement; Unit; Clinical Microsystem; Nursing Homes
To study the association between organizational context and research utilization in German residential long term care (LTC), we translated three Canadian assessment instruments: the Alberta Context Tool (ACT), Estabrooks’ Kinds of Research Utilization (RU) items and the Conceptual Research Utilization Scale. Target groups for the tools were health care aides (HCAs), registered nurses (RNs), allied health professionals (AHPs), clinical specialists and care managers. Through a cognitive debriefing process, we assessed response processes validity–an initial stage of validity, necessary before more advanced validity assessment.
We included 39 participants (16 HCAs, 5 RNs, 7 AHPs, 5 specialists and 6 managers) from five residential LTC facilities. We created lists of questionnaire items containing problematic items plus items randomly selected from the pool of remaining items. After participants completed the questionnaires, we conducted individual semi-structured cognitive interviews using verbal probing. We asked participants to reflect on their answers for list items in detail. Participants’ answers were compared to concept maps defining the instrument concepts in detail. If at least two participants gave answers not matching concept map definitions, items were revised and re-tested with new target group participants.
Cognitive debriefings started with HCAs. Based on the first round, we modified 4 of 58 ACT items, 1 ACT item stem and all 8 items of the RU tools. All items were understood by participants after another two rounds. We included revised HCA ACT items in the questionnaires for the other provider groups. In the RU tools for the other provider groups, we used different wording than the HCA version, as was done in the original English instruments. Only one cognitive debriefing round was needed with each of the other provider groups.
Cognitive debriefing is essential to detect and respond to problematic instrument items, particularly when translating instruments for heterogeneous, less well educated provider groups such as HCAs. Cognitive debriefing is an important step in research tool development and a vital component of establishing response process validity evidence. Publishing cognitive debriefing results helps researchers to determine potentially critical elements of the translated tools and assists with interpreting scores.
Cognitive debriefing; Translation; Alberta context tool; Estabrooks’ kind of research utilization items; Conceptual research utilization scale; Organizational context; Research utilization; Residential long term care
OPTIC is a mixed method Partnership for Health System Improvement (http://www.cihr-irsc.gc.ca/e/34348.html) study focused on improving care for nursing home (NH) residents who are transferred to and from emergency departments (EDs) via emergency medical services (EMS). In the pilot study we tested feasibility of concurrently collecting individual resident data during transitions across settings using the Transition Tracking Tool (T3).
The pilot study tracked 54 residents transferred from NHs to one of two EDs in two western Canadian provinces over a three month period. The T3 is an electronic data collection tool developed for this study to record data relevant to describing and determining success of transitions in care. It comprises 800+ data elements including resident characteristics, reasons and precipitating factors for transfer, advance directives, family involvement, healthcare services provided, disposition decisions, and dates/times and timing.
Residents were elderly (mean age = 87.1 years) and the majority were female (61.8%). Feasibility of collecting data from multiple sources across two research sites was established. We identified resources and requirements to access and retrieve specific data elements in various settings to manage data collection processes and allocate research staff resources. We present preliminary data from NH, EMS, and ED settings.
While most research in this area has focused on a unidirectional process of patient progression from one care setting to another, this study established feasibility of collecting detailed data from beginning to end of a transition across multiple settings and in multiple directions.
Transfers; Transitional care; Transition tracking; Nursing home; Emergency department
With the persistent gaps between research and practice in healthcare systems, knowledge translation (KT) has gained significance and importance. Also, in most industrialized countries, there is an increasing emphasis on managing chronic health conditions with the best available evidence. Yet, organizations aiming to improve chronic care (CC) require an adequate level of organizational readiness (OR) for KT.
Objectives: The purpose of this study is to review and synthesize the existing evidence on conceptual models/frameworks of Organizational Readiness for Change (ORC) in healthcare as the basis for the development of a comprehensive framework of OR for KT in the context of CC.
We conducted a systematic review of the literature on OR for KT in CC using Pubmed, Embase, CINAHL, PsychINFO, Web of Sciences (SCI and SSCI), and others. Search terms included readiness; commitment and change; preparedness; willing to change; organization and administration; and health and social services.
Study selection: The search was limited to studies that had been published between the starting date of each bibliographic database (e.g., 1964 for PubMed) and November 1, 2012. Only papers that refer to a theory, a theoretical component from any framework or model on OR that were applicable to the healthcare domain were considered. We analyzed data using conceptual mapping.
Data extraction: Pairs of authors independently screened the published literature by reviewing their titles and abstracts. Then, the two same reviewers appraised the full text of each study independently.
Overall, we found and synthesized 10 theories, theoretical models and conceptual frameworks relevant to ORC in healthcare described in 38 publications. We identified five core concepts, namely organizational dynamics, change process, innovation readiness, institutional readiness, and personal readiness. We extracted 17 dimensions and 59 sub-dimensions related to these 5 concepts.
Our findings provide a useful overview for researchers interested in ORC and aims to create a consensus on the core theoretical components of ORC in general and of OR for KT in CC in particular. However, more work is needed to define and validate the core elements of a framework that could help to assess OR for KT in CC.
Organizational readiness; Conceptual models; Frameworks; Knowledge translation; Chronic care; Healthcare system
Understanding the relationship between organizational context and research utilization is key to reducing the research-practice gap in health care. This is particularly true in the residential long term care (LTC) setting where relatively little work has examined the influence of context on research implementation. Reliable, valid measures and tools are a prerequisite for studying organizational context and research utilization. Few such tools exist in German. We thus translated three such tools (the Alberta Context Tool and two measures of research use) into German for use in German residential LTC. We point out challenges and strategies for their solution unique to German residential LTC, and demonstrate how resolving specific challenges in the translation of the health care aide instrument version streamlined the translation process of versions for registered nurses, allied health providers, practice specialists, and managers.
Our translation methods were based on best practices and included two independent forward translations, reconciliation of the forward translations, expert panel discussions, two independent back translations, reconciliation of the back translations, back translation review, and cognitive debriefing.
We categorized the challenges in this translation process into seven categories: (1) differing professional education of Canadian and German care providers, (2) risk that German translations would become grammatically complex, (3) wordings at risk of being misunderstood, (4) phrases/idioms non-existent in German, (5) lack of corresponding German words, (6) limited comprehensibility of corresponding German words, and (7) target persons’ unfamiliarity with activities detailed in survey items. Examples of each challenge are described with strategies that we used to manage the challenge.
Translating an existing instrument is complex and time-consuming, but a rigorous approach is necessary to obtain instrument equivalence. Essential components were (1) involvement of and co-operation with the instrument developers and (2) expert panel discussions, including both target group and content experts. Equivalent translated instruments help researchers from different cultures to find a common language and undertake comparative research. As acceptable psychometric properties are a prerequisite for that, we are currently carrying out a study with that focus.
Translation; Cultural adaptation; Alberta Context Tool; Estabrooks’ Kind of Research Utilization; Conceptual Research Use scale; Organizational context; Research utilization; Residential long term care
In recent years, improving the quality of care for nursing home residents has generated a considerable amount of attention. In response, quality indicators (QIs), based on available evidence and expert consensus, have been identified within the Resident Assessment Instrument – Minimum Data Set 2.0 (RAI-MDS 2.0), and validated as proxy measures for quality of nursing home care. We sought to identify practice sensitive QIs; that is, those QIs believed to be the most sensitive to clinical practice.
We enlisted two experts to review a list of 35 validated QIs and to select those that they believed to be the most sensitive to practice. We then asked separate groups of practicing physicians, nurses, and policy makers to (1) rank the items on the list for overall “practice sensitivity” and then, (2) to identify the domain to which the QI was most sensitive (nursing care, physician care, or policy maker).
After combining results of all three groups, pressure ulcers were identified as the most practice sensitive QI followed by worsening pain, physical restraint use, the use of antipsychotic medications without a diagnosis of psychosis, and indwelling catheters. When stratified by informant group, although the top five QIs stayed the same, the ranking of the 13 QIs differed by group.
In addition to identifying a reduced and manageable set of QIs for regular reporting, we believe that focusing on these 13 practice sensitive QIs provides both the greatest potential for improving resident function and slowing the trajectory of decline that most residents experience.
Nursing home; Performance measurement; RAI-MDS; Quality indicators
Bridging the research-practice gap is an important research focus in continuing care facilities, because the population of older adults (aged 65 years and over) requiring continuing care services is the fastest growing demographic among countries in the Organisation for Economic Co-operation and Development (OECD). Unlicensed practitioners, known as health care aides, provide the majority of care for residents living in continuing care facilities. However, little research examines how to sustain health care aide behavior change following initial adoption of current research evidence.
We will conduct a phase III, multicentre, cluster randomized controlled trial (RCT) using a stratified 2 × 2 additive factorial design, including an embedded process evaluation, in 24 supportive living facilities within the health zone of Edmonton, AB, Canada. We will determine which combination of frequency and intensity of reminders most effectively sustains the completion of the sit-to-stand activity by health care aides with residents. Frequency refers to how often a reminder is implemented; intensity refers to whether a reminder is social or paper-based. We will compare monthly reminders with reminders implemented every 3 months, and we will compare low intensity, paper-based reminders and high intensity reminders provided by a health care aide peer.
Using interviews, questionnaires, and observations, Sustaining Transfers through Affordable Research Translation (START) will evaluate the processes that inhibit or promote the mobility innovation’s sustainability among health care aides in daily practice. We will examine how the reminders are implemented and perceived by health care aides and licensed practical nurses, as well as how health care aides providing peer reminders are identified, received by their peers, and supported by their supervisors.
START will connect up-to-date innovation research with the practice of health care aides providing direct care to a growing population of older Albertans. The project’s reach extends to both supportive living and long-term care settings. Furthermore, START has the potential to introduce and sustain a broad range of innovations in various care areas, such as dementia care, wound care, and pain management – domains where the uptake and sustainability of innovations also encounter significant challenges. By identifying the optimal frequency and intensity of knowledge translation interventions, we hope to enable continuing care organizations to efficiently integrate care innovations into the day-to-day care of residents.
Knowledge translation; Reminders; Continuing care; Health care aides; Long-term care; Older adults; Sustainability; Mobility
Almost 90 percent of nursing home residents have some type of mobility limitation. Many spend most of their waking hours lying in bed or sitting. Such inactivity can negatively affect residents’ health and general well-being. This pilot study aimed to assess (1) the effect of the sit-to-stand activity on mobility outcomes of nursing home residents, (2) the effect of an audit-and-feedback intervention on uptake of the sit-to-stand activity by healthcare aides, and (3) the contextual factors influencing uptake of the sit-to-stand activity by healthcare aides.
This quasi-experimental pilot study was conducted in two nursing homes in western Canada. Twenty-six residents with dementia completed the sit-to-stand activity with 56 healthcare aides during daily care; separately, 71 healthcare aides completed a research use and context survey. Preliminary mobility feedback was presented to healthcare aides in one site. Resident mobility was measured using the 30-second sit-to-stand test. Healthcare aide uptake of the activity was measured using documentation flowsheets and a survey-based measure. Context was measured using the Alberta Context Tool. Mobility and uptake outcomes were analyzed over time and by site with analysis of covariance. Spearman and Pearson correlations were used to correlate context data with research use.
Residents who more frequently completed the sit-to-stand activity were more likely to maintain or improve mobility compared with those who completed it less frequently (F=4.46; p=0.046, after adjustment for age). Uptake for one site was significantly different from the other (t-score=2.67; p=0.01, after adjustment for resident covariates). The audit-and-feedback intervention was associated with increased uptake of the activity from pre-intervention to post-intervention (t-score=-2.48; p=0.02). More context domains correlated significantly with aides’ use of conceptual research and information sources in one site than the other.
The sit-to-stand activity is a promising means to maintain or improve transfer ability of nursing home residents with dementia. In the nursing home with initially weak uptake, strengthened uptake followed an audit-and-feedback intervention. Activity participation was higher in the site with stronger correlations between context and measured research use. Results are sufficiently promising to warrant proceeding with a full clinical trial.
Mobility; Knowledge translation; Healthcare aides; Context; Long-term care; Nursing home
Organizational context is recognized as an important influence on the successful implementation of research by healthcare professionals. However, there is relatively little empirical evidence to support this widely held view.
The objective of this study was to identify dimensions of organizational context and individual (nurse) characteristics that influence pediatric nurses’ self-reported use of research. Data on research use, individual, and contextual variables were collected from registered nurses (N = 735) working on 32 medical, surgical and critical care units in eight Canadian pediatric hospitals using an online survey. We used Generalized Estimating Equation modeling to account for the correlated structure of the data and to identify which contextual dimensions and individual characteristics predict two kinds of self-reported research use: instrumental (direct) and conceptual (indirect).
Significant predictors of instrumental research use included: at the individual level; belief suspension-implement, research use in the past, and at the hospital unit (context) level; culture, and the proportion on nurses possessing a baccalaureate degree or higher. Significant predictors of conceptual research use included: at the individual nurse level; belief suspension-implement, problem solving ability, use of research in the past, and at the hospital unit (context) level; leadership, culture, evaluation, formal interactions, informal interactions, organizational slack-space, and unit specialty.
Hospitals, by focusing attention on modifiable elements of unit context may positively influence nurses’ reported use of research. This influence of context may extend to the adoption of best practices in general and other innovative or quality interventions.
There is emerging evidence that context is important for successful transfer of research knowledge into health care practice. The Alberta Context Tool (ACT) is a Canadian developed research-based instrument that assesses 10 modifiable concepts of organizational context considered important for health care professionals’ use of evidence. Swedish and Canadian health care have similarities in terms of organisational and professional aspects, suggesting that the ACT could be used for measuring context in Sweden. This paper reports on the translation of the ACT to Swedish and a testing of preliminary aspects of its validity, acceptability and reliability in Swedish elder care.
The ACT was translated into Swedish and back-translated into English before being pilot tested in ten elder care facilities for response processes validity, acceptability and reliability (Cronbach’s alpha). Subsequently, further modification was performed.
In the pilot test, the nurses found the questions easy to respond to (52%) and relevant (65%), yet the questions’ clarity were mainly considered ‘neither clear nor unclear’ (52%). Missing data varied between 0 (0%) and 19 (12%) per item, the most common being 1 missing case per item (15 items). Internal consistency (Cronbach’s Alpha > .70) was reached for 5 out of 8 contextual concepts. Translation and back translation identified 21 linguistic- and semantic related issues and 3 context related deviations, resolved by developers and translators.
Modifying an instrument is a detailed process, requiring time and consideration of the linguistic and semantic aspects of the instrument, and understanding of the context where the instrument was developed and where it is to be applied. A team, including the instrument’s developers, translators, and researchers is necessary to ensure a valid translation. This study suggests preliminary validity, reliability and acceptability evidence for the ACT when used with nurses in Swedish elder care.
Questionnaire; Translation; Validity; Health care context; Research utilization; Nursing
Background and Purpose. In this paper, we present a protocol for advanced psychometric assessments of surveys based on the Standards for Educational and Psychological Testing. We use the Alberta Context Tool (ACT) as an exemplar survey to which this protocol can be applied. Methods. Data mapping, acceptability, reliability, and validity are addressed. Acceptability is assessed with missing data frequencies and the time required to complete the survey. Reliability is assessed with internal consistency coefficients and information functions. A unitary approach to validity consisting of accumulating evidence based on instrument content, response processes, internal structure, and relations to other variables is taken. We also address assessing performance of survey data when aggregated to higher levels (e.g., nursing unit). Discussion. In this paper we present a protocol for advanced psychometric assessment of survey data using the Alberta Context Tool (ACT) as an exemplar survey; application of the protocol to the ACT survey is underway. Psychometric assessment of any survey is essential to obtaining reliable and valid research findings. This protocol can be adapted for use with any nursing survey.
There is growing awareness of the role of information technology in evidence-based practice. The purpose of this study was to investigate the role of organizational context and nurse characteristics in explaining variation in nurses’ use of personal digital assistants (PDAs) and mobile Tablet PCs for accessing evidence-based information. The Promoting Action on Research Implementation in Health Services (PARIHS) model provided the framework for studying the impact of providing nurses with PDA-supported, evidence-based practice resources, and for studying the organizational, technological, and human resource variables that impact nurses’ use patterns.
A survey design was used, involving baseline and follow-up questionnaires. The setting included 24 organizations representing three sectors: hospitals, long-term care (LTC) facilities, and community organizations (home care and public health). The sample consisted of 710 participants (response rate 58%) at Time 1, and 469 for whom both Time 1 and Time 2 follow-up data were obtained (response rate 66%). A hierarchical regression model (HLM) was used to evaluate the effect of predictors from all levels simultaneously.
The Chi square result indicated PDA users reported using their device more frequently than Tablet PC users (p = 0.001). Frequency of device use was explained by ‘breadth of device functions’ and PDA versus Tablet PC. Frequency of Best Practice Guideline use was explained by ‘willingness to implement research,’ ‘structural and electronic resources,’ ‘organizational slack time,’ ‘breadth of device functions’ (positive effects), and ‘slack staff’ (negative effect). Frequency of Nursing Plus database use was explained by ‘culture,’ ‘structural and electronic resources,’ and ‘breadth of device functions’ (positive effects), and ‘slack staff’ (negative). ‘Organizational culture’ (positive), ‘breadth of device functions’ (positive), and ‘slack staff ‘(negative) were associated with frequency of Lexi/PEPID drug dictionary use.
Access to PDAs and Tablet PCs supported nurses’ self-reported use of information resources. Several of the organizational context variables and one individual nurse variable explained variation in the frequency of information resource use.
Health information technologies; Mobile technology; Personal digital assistant; Nursing informatics; Information-seeking; Organizational context; Evidence-based practice; PARIHS model
Changes in health status, triggered by events such as infections, falls, and geriatric syndromes, are common among nursing home (NH) residents and necessitate transitions between NHs and Emergency Departments (EDs). During transitions, residents frequently experience care that is delayed, unnecessary, not evidence-based, potentially unsafe, and fragmented. Furthermore, a high proportion of residents and their family caregivers report substantial unmet needs during transitions. This study is part of a program of research whose overall aim is to improve quality of care for frail older adults who reside in NHs. The purpose of this study is to identify successful transitions from multiple perspectives and to identify organizational and individual factors related to transition success, in order to inform improvements in care for frail elderly NH residents during transitions to and from acute care. Specific objectives are to:
1. define successful and unsuccessful elements of transitions from multiple perspectives;
2. develop and test a practical tool to assess transition success;
3. assess transition processes in a discrete set of transfers in two study sites over a one year period;
4. assess the influence of organizational factors in key practice locations, e.g., NHs, emergency medical services (EMS), and EDs, on transition success; and
5. identify opportunities for evidence-informed management and quality improvement decisions related to the management of NH – ED transitions.
This is a mixed-methods observational study incorporating an integrated knowledge translation (IKT) approach. It uses data from multiple levels (facility, care unit, individual) and sources (healthcare providers, residents, health records, and administrative databases).
Key to study success is operationalizing the IKT approach by using a partnership model in which the OPTIC governance structure provides for team decision-makers and researchers to participate equally in developing study goals, design, data collection, analysis and implications of findings. As preliminary and ongoing study findings are developed, their implications for practice and policy in study settings will be discussed by the research team and shared with study site administrators and staff. The study is designed to investigate the complexities of transitions and to enhance the potential for successful and sustained improvement of these transitions.
Seniors; Elderly; Transitions; Quality of care; Handovers; Communications; Emergency Departments; Emergency Medical Services; Nursing homes; Measurement
Researchers strive to optimize data quality in order to ensure that study findings are valid and reliable. In this paper, we describe a data quality control program designed to maximize quality of survey data collected using computer-assisted personal interviews. The quality control program comprised three phases: (1) software development, (2) an interviewer quality control protocol, and (3) a data cleaning and processing protocol. To illustrate the value of the program, we assess its use in the Translating Research in Elder Care Study. We utilize data collected annually for two years from computer-assisted personal interviews with 3004 healthcare aides. Data quality was assessed using both survey and process data. Missing data and data errors were minimal. Mean and median values and standard deviations were within acceptable limits. Process data indicated that in only 3.4% and 4.0% of cases was the interviewer unable to conduct interviews in accordance with the details of the program. Interviewers' perceptions of interview quality also significantly improved between Years 1 and 2. While this data quality control program was demanding in terms of time and resources, we found that the benefits clearly outweighed the effort required to achieve high-quality data.
In Canada, healthcare aides (also referred to as nurse aides, personal support workers, nursing assistants) are unregulated personnel who provide 70-80% of direct care to residents living in nursing homes. Although they are an integral part of the care team their contributions to the resident care planning process are not always acknowledged in the organization. The purpose of the Safer Care for Older Persons [in residential] Environments (SCOPE) project was to evaluate the feasibility of engaging front line staff (primarily healthcare aides) to use quality improvement methods to integrate best practices into resident care. This paper describes the process used by teams participating in the SCOPE project to select clinical improvement areas.
The study employed a collaborative approach to identify clinical areas and through consensus, teams selected one of three areas. To select the clinical areas we recruited two nursing homes not involved in the SCOPE project and sampled healthcare providers and decision-makers within them. A vote counting method was used to determine the top five ranked clinical areas for improvement.
Responses received from stakeholder groups included gerontology experts, decision-makers, registered nurses, managers, and healthcare aides. The top ranked areas from highest to lowest were pain/discomfort management, behaviour management, depression, skin integrity, and assistance with eating.
Involving staff in selecting areas that they perceive as needing improvement may facilitate staff engagement in the quality improvement process.
Quality improvement; Healthcare providers; Quality care; Long-term care
Engaging end-users of research in the process of disseminating findings may increase the relevance of findings and their impact for users. We report findings from a case study that explored how involvement with the Translating Research in Elder Care (TREC) study influenced management and staff at one of 36 TREC facilities. We conducted the study at ‘Restwood’ (pseudonym) nursing home because the Director of Care engaged actively in the study and TREC data showed that this site differed on some areas from other nursing homes in the province. The aims of the case study were two-fold: to gain a better understanding of how frontline staff engage with the research process, and to gain a better understanding of how to share more detailed research results with management.
We developed an Expanded Feedback Report for use during this study. In it, we presented survey results that compared Restwood to the best performing site on all variables and participating sites in the province. Data were collected regarding the Expanded Feedback Report through interviews with management. Data from staff were collected through interviews and observation. We used content analysis to derive themes to describe key aspects related to the study aims.
We observed the importance of understanding organizational routines and the impact of key events in the facility’s environment. We gleaned additional information that validated findings from prior feedback mechanisms within TREC. Another predominant theme was the sense that the opportunity to engage in a research process was reaffirming for staff (particularly healthcare aides)—what they did and said mattered, and TREC provided a means of having one’s voice heard. We gained valuable insight from the Director of Care about how to structure and format more detailed findings to assist with interpretation and use of results.
Four themes emerged regarding staff engagement with the research process: sharing feedback reports from the TREC study; the meaning of TREC to staff; understanding organizational context; and using the study feedback for improvement at Restwood. This study has lessons for researchers on how to share research results with study participants, including management.
This report is an introduction to a series of three research papers that describe the evolution of the approaches taken by the Translating Research in Elder Care (TREC) research team during its first four years to feed back the research findings to study participants. TREC is an observational multi-method health services research project underway in 36 nursing homes in the prairie provinces of Canada. TREC has actively involved decision makers from the sector in all stages from initial planning, through data collection to dissemination activities. However, it was not planned as a fully integrated knowledge translation project. These three papers describe our progress towards fully integrated knowledge translation—with respect to timely and requested feedback processes. The first paper reports on the process and outcomes of creating and evaluating the feedback of research findings to healthcare aides (unregulated health professionals). These aides provide over 80% of the direct care in our sample and actively requested the feedback as a condition of their continued cooperation in the data acquisition process. The second paper describes feedback from nursing home administrators on preliminary research findings (a facility annual report) and evaluation of the reports’ utility. The third paper discusses an approach to providing a more in-depth form of feedback (expanded feedback report) at one of the TREC nursing homes.
Survey and interview feedback from healthcare aides is presented in the first paper. Overall, healthcare aides’ opinions about presentation of the feedback report and the understand ability, usability, and usefulness of the content were positive. The second paper describes the use of telephone interviews with facility administrators and indicates that the majority of contextual areas (e.g., staff job satisfaction) addressed in facility annual report to be useful, meaningful, and understandable. More than one-half of the administrators would have liked to have received information on additional areas. The third paper explores how a case study that examined how involvement with the TREC study influenced management and staff at one of the TREC nursing homes. The importance of understanding organizational routines and the impact of corporate restructuring were key themes emerging from the case study. In addition, the Director of Care suggested changes to the structure and format of the feedback report that would have improved its usefulness.
We believe that these findings will inform others undertaking integrated knowledge translation activities and will encourage others to become more engaged in feedback processes.
This project occurred during the course of the Translating Research in Elder Care (TREC) program of research. TREC is a multilevel and longitudinal research program being conducted in the three Canadian Prairie Provinces of Alberta, Saskatchewan, and Manitoba. The main purpose of TREC is to increase understanding about the role of organizational context in influencing knowledge use in residential long-term care settings. The purpose of this study was to evaluate healthcare aides’ (HCAs) perceptions of a one-page poster designed to feed back aggregated data (including demographic information and perceptions about influences on best practice) from the TREC survey they had recently completed.
A convenience sample of 7 of the 15 nursing homes participating in the TREC research program in Alberta were invited to participate. Specific facility-level summary data were provided to each facility in the form of a one-page poster report. Two weeks following delivery of the report, a convenience sample of HCAs was surveyed using one-to-one structured interviews.
One hundred twenty-three HCAs responded to the evaluation survey. Overall, HCAs’ opinions about presentation of the feedback report and the understandability, usability, and usefulness of the content were positive. For each report, analysis of data and production and inspection of the report took up to one hour. Information sessions to introduce and explain the reports averaged 18 minutes. Two feedback reports (minimum) were supplied to each facility at a cost of CAN$2.39 per report, for printing and laminating.
This study highlights not only the feasibility of producing understandable, usable, and useful feedback reports of survey data but also the value and importance of providing feedback to survey respondents. More broadly, the findings suggest that modest strategies may have a positive and desirable effect in participating sites.
This project is part of the Translating Research in Elder Care (TREC) program of research, a multi-level and longitudinal research program being conducted in 36 nursing homes in three Canadian Prairie Provinces. The overall goal of TREC is to improve the quality of care for older persons living in nursing homes and the quality of work life for care providers. The purpose of this paper is to report on development and evaluation of facility annual reports (FARs) from facility administrators’ perspectives on the usefulness, meaningfulness, and understandability of selected data from the TREC survey.
A cross sectional survey design was used in this study. The feedback reports were developed in collaboration with participating facility administrators. FARs presented results in four contextual areas: workplace culture, feedback processes, job satisfaction, and staff burnout. Six weeks after FARs were mailed to each administrator, we conducted structured telephone interviews with administrators to elicit their evaluation of the FARs. Administrators were also asked if they had taken any actions as a result of the FAR. Descriptive and inferential statistics, as well as content analysis for open-ended questions, were used to summarize findings.
Thirty-one facility administrators (representing thirty-two facilities) participated in the interviews. Six administrators had taken action and 18 were planning on taking action as a result of FARs. The majority found the four contextual areas addressed in FAR to be useful, meaningful, and understandable. They liked the comparisons made between data from years one and two and between their facility and other TREC study sites in their province. Twenty-two indicated that they would like to receive information on additional areas such as aggressive behaviours of residents and information sharing. Twenty-four administrators indicated that FARs contained enough information, while eight found FARs ‘too short’. Administrators who reported that the FAR contained enough information were more likely to take action within their facilities than administrators who reported that they needed more information.
Although the FAR was brief, the presentation of the four contextual areas was relevant to the majority of administrators and prompted them to plan or to take action within their facility.
Research evidence underpins best practice, but is not always used in healthcare. The Promoting Action on Research Implementation in Health Services (PARIHS) framework suggests that the nature of evidence, the context in which it is used, and whether those trying to use evidence are helped (or facilitated) affect the use of evidence. Urinary incontinence has a major effect on quality of life of older people, has a high prevalence, and is a key priority within European health and social care policy. Improving continence care has the potential to improve the quality of life for older people and reduce the costs associated with providing incontinence aids.
This study aims to advance understanding about the contribution facilitation can make to implementing research findings into practice via: extending current knowledge of facilitation as a process for translating research evidence into practice; evaluating the feasibility, effectiveness, and cost-effectiveness of two different models of facilitation in promoting the uptake of research evidence on continence management; assessing the impact of contextual factors on the processes and outcomes of implementation; and implementing a pro-active knowledge transfer and dissemination strategy to diffuse study findings to a wide policy and practice community.
Setting and sample
Four European countries, each with six long-term nursing care sites (total 24 sites) for people aged 60 years and over with documented urinary incontinence
Methods and design
Pragmatic randomised controlled trial with three arms (standard dissemination and two different programmes of facilitation), with embedded process and economic evaluation. The primary outcome is compliance with the continence recommendations. Secondary outcomes include proportion of residents with incontinence, incidence of incontinence-related dermatitis, urinary tract infections, and quality of life. Outcomes are assessed at baseline, then at 6, 12, 18, and 24 months after the start of the facilitation interventions. Detailed contextual and process data are collected throughout, using interviews with staff, residents and next of kin, observations, assessment of context using the Alberta Context Tool, and documentary evidence. A realistic evaluation framework is used to develop explanatory theory about what works for whom in what circumstances.
Current Controlled Trials ISRCTN11598502.
Almost 90% of residents living in long-term care facilities have limited mobility which is associated with a loss of ability in activities of daily living, falls, increased risk of serious medical problems such as pressure ulcers, incontinence and a significant decline in health-related quality of life. For health workers caring for residents it may also increase the risk of injury. The effectiveness of rehabilitation to facilitate mobility has been studied with dedicated research assistants or extensively trained staff caregivers; however, few investigators have examined the effectiveness of techniques to encourage mobility by usual caregivers in long-term care facilities.
This longitudinal, quasi-experimental study is designed to demonstrate the effect of the sit-to-stand activity carried out by residents in the context of daily care with health care aides. In three intervention facilities health care aides will prompt residents to repeat the sit-to-stand action on two separate occasions during each day and each evening shift as part of daily care routines. In three control facilities residents will receive usual care. Intervention and control facilities are matched on the ownership model (public, private for-profit, voluntary not-for-profit) and facility size. The dose of the mobility intervention is assessed through the use of daily documentation flowsheets in the health record. Resident outcome measures include: 1) the 30-second sit-to-stand test; 2) the Functional Independence Measure; 3) the Health Utilities Index Mark 2 and 3; and, 4) the Quality of Life - Alzheimer's Disease.
There are several compelling reasons for this study: the widespread prevalence of limited mobility in this population; the rapid decline in mobility after admission to a long-term care facility; the importance of mobility to quality of life; the increased time (and therefore cost) required to care for residents with limited mobility; and, the increased risk of injury for health workers caring for residents who are unable to stand. The importance of these issues is magnified when considering the increasing number of people living in long-term care facilities and an aging population.
This clinical trial is registered with ClinicalTrials.gov (trial registration number: NCT01474616).
There are few validated measures of organizational context and none that we located are parsimonious and address modifiable characteristics of context. The Alberta Context Tool (ACT) was developed to meet this need. The instrument assesses 8 dimensions of context, which comprise 10 concepts. The purpose of this paper is to report evidence to further the validity argument for ACT. The specific objectives of this paper are to: (1) examine the extent to which the 10 ACT concepts discriminate between patient care units and (2) identify variables that significantly contribute to between-unit variation for each of the 10 concepts.
859 professional nurses (844 valid responses) working in medical, surgical and critical care units of 8 Canadian pediatric hospitals completed the ACT. A random intercept, fixed effects hierarchical linear modeling (HLM) strategy was used to quantify and explain variance in the 10 ACT concepts to establish the ACT's ability to discriminate between units. We ran 40 models (a series of 4 models for each of the 10 concepts) in which we systematically assessed the unique contribution (i.e., error variance reduction) of different variables to between-unit variation. First, we constructed a null model in which we quantified the variance overall, in each of the concepts. Then we controlled for the contribution of individual level variables (Model 1). In Model 2, we assessed the contribution of practice specialty (medical, surgical, critical care) to variation since it was central to construction of the sampling frame for the study. Finally, we assessed the contribution of additional unit level variables (Model 3).
The null model (unadjusted baseline HLM model) established that there was significant variation between units in each of the 10 ACT concepts (i.e., discrimination between units). When we controlled for individual characteristics, significant variation in the 10 concepts remained. Assessment of the contribution of specialty to between-unit variation enabled us to explain more variance (1.19% to 16.73%) in 6 of the 10 ACT concepts. Finally, when we assessed the unique contribution of the unit level variables available to us, we were able to explain additional variance (15.91% to 73.25%) in 7 of the 10 ACT concepts.
The findings reported here represent the third published argument for validity of the ACT and adds to the evidence supporting its use to discriminate patient care units by all 10 contextual factors. We found evidence of relationships between a variety of individual and unit-level variables that explained much of this between-unit variation for each of the 10 ACT concepts. Future research will include examination of the relationships between the ACT's contextual factors and research utilization by nurses and ultimately the relationships between context, research utilization, and outcomes for patients.
In healthcare, a gap exists between what is known from research and what is practiced. Understanding this gap depends upon our ability to robustly measure research utilization.
The objectives of this systematic review were: to identify self-report measures of research utilization used in healthcare, and to assess the psychometric properties (acceptability, reliability, and validity) of these measures.
We conducted a systematic review of literature reporting use or development of self-report research utilization measures. Our search included: multiple databases, ancestry searches, and a hand search. Acceptability was assessed by examining time to complete the measure and missing data rates. Our approach to reliability and validity assessment followed that outlined in the Standards for Educational and Psychological Testing.
Of 42,770 titles screened, 97 original studies (108 articles) were included in this review. The 97 studies reported on the use or development of 60 unique self-report research utilization measures. Seven of the measures were assessed in more than one study. Study samples consisted of healthcare providers (92 studies) and healthcare decision makers (5 studies). No studies reported data on acceptability of the measures. Reliability was reported in 32 (33%) of the studies, representing 13 of the 60 measures. Internal consistency (Cronbach's Alpha) reliability was reported in 31 studies; values exceeded 0.70 in 29 studies. Test-retest reliability was reported in 3 studies with Pearson's r coefficients > 0.80. No validity information was reported for 12 of the 60 measures. The remaining 48 measures were classified into a three-level validity hierarchy according to the number of validity sources reported in 50% or more of the studies using the measure. Level one measures (n = 6) reported evidence from any three (out of four possible) Standards validity sources (which, in the case of single item measures, was all applicable validity sources). Level two measures (n = 16) had evidence from any two validity sources, and level three measures (n = 26) from only one validity source.
This review reveals significant underdevelopment in the measurement of research utilization. Substantial methodological advances with respect to construct clarity, use of research utilization and related theory, use of measurement theory, and psychometric assessment are required. Also needed are improved reporting practices and the adoption of a more contemporary view of validity (i.e., the Standards) in future research utilization measurement studies.