|Home | About | Journals | Submit | Contact Us | Français|
Due to fragmentation of care, continuity of care is often limited in the care provided to frail older people. Further, frail older people are not always enabled to become involved in their own care. Therefore, we developed the Health and Welfare Information Portal (ZWIP), a shared Electronic Health Record combined with a communication tool for community-dwelling frail older people and primary care professionals. This article describes the process evaluation of its implementation, and aims to establish (1) the outcomes of the implementation process, (2) which implementation strategies and barriers and facilitators contributed to these outcomes, and (3) how its future implementation could be improved.
Mixed methods study, consisting of (1) a survey among professionals (n=118) and monitoring the use of the ZWIP by frail older people and professionals, followed by (2) semi-structured interviews with purposively selected professionals (n=12).
290 frail older people and 169 professionals participated in the ZWIP. At the end of the implementation period, 55% of frail older people and informal caregivers, and 84% of professionals had logged on to their ZWIP at least once. For professionals, the exposure to the implementation strategies was generally as planned, they considered the interprofessional educational program and the helpdesk very important strategies. However, frail older people’s exposure to the implementation strategies was less than intended. Facilitators for the ZWIP were the perceived need to enhance interprofessional collaboration and the ZWIP application being user-friendly. Barriers included the low computer-literacy of frail older people, a preference for personal communication and limited use of the ZWIP by other professionals and frail older people. Interviewees recommended using the ZWIP for other target populations as well and adding further strategies that may help frail older people to feel more comfortable with computers and the ZWIP.
This study describes the implementation process of an innovative e-health intervention for community-dwelling frail older people, informal caregivers and primary care professionals. As e-health is an important medium for overcoming fragmentation of healthcare and facilitating patient involvement, but its adoption in everyday practice remains a challenge, the positive results of this implementation are promising.
Our current healthcare system is not well equipped to meet the needs of the growing population of frail older people . Due to fragmentation, care for a single frail older person is often provided by a number of professionals who work for a variety of organizations and services, which results in discontinuity of care [2,3]. Also, the healthcare system is not designed to facilitate the incorporation of patient perspectives in decision making, which becomes even more difficult when care is delivered by an interprofessional team [4,5]. This is unfortunate, as involving patients in their care is mandatory and can improve patient outcomes [6,7].
Information Technology has the potential to diminish these problems, by means of a multidisciplinary shared Electronic Health Record that is accessible to patients as well [1,8]. Therefore, we developed a program which included such a record: the Health and Welfare Information Portal (ZWIP). The program aimed to facilitate shared decision-making and self-management by frail older people and informal caregivers, as well as to reduce fragmentation of care by improving collaboration among professionals involved.
As the program consisted of a number of interacting components and was delivered to several different general practices, conducting a process evaluation to study its implementation was considered critical . First, because information concerning barriers and facilitators experienced during the implementation may be able to guide improvements to the implementation plan and the intervention itself [10,11]. Second, because it can help identify critical factors for the intervention’s implementation in other settings or by other research groups . Such process evaluations are especially useful for e-health interventions, as the number of studies concerning their implementation is limited , while their adoption remains a challenge . Therefore, this article aims to establish (1) the outcomes of the implementation process of the ZWIP, (2) which implementation strategies and experienced barriers and facilitators contributed to these outcomes, and (3) how its future implementation could be improved.
We evaluated the implementation of the ZWIP with a mixed methods study, consisting of (1) a quantitative evaluation by means of a survey and data collected during the implementation and use of the ZWIP, followed by (2) a qualitative evaluation by means of semi-structured interviews with purposively selected participants. The local ethics committee, the Committee on Research involving Human Subjects Arnhem-Nijmegen, reviewed the study and stated that no formal approval was required.
Participants of the study were community-dwelling frail older people, who were patients of participating general practices in the province of Gelderland or Noord-Brabant, the Netherlands; their informal caregivers; and healthcare and welfare professionals involved in their care. They participated in the ZWIP during its implementation phase, which started in September 2010 and ended on the first of July 2011. We monitored the use of the ZWIP and its implementation strategies for both frail older people and professionals. Further, professionals were surveyed and interviewed. We chose not to survey or interview frail older people and informal caregivers as the project had already been time-consuming for them, and they would be surveyed and interviewed as part of the project’s future effect evaluation as well.
The ZWIP was developed by means of Intervention Mapping , a method for the systematic development of evidence-informed interventions. Throughout this development, future users, i.e. primary care professionals and geriatricians (n=15), as well as (frail) older people and informal caregivers (n=14), were involved extensively trough their participation in working groups. These working groups started with participants specifying which problems related to interprofessional collaboration and self-management by frail older people should be solved by the ZWIP, for example not knowing which professionals are involved in the care of a particular frail older person and professionals not being able to contact each other. Then, theories matching the identified determinants of these problems were used to support the development of the intervention. These included Social Cognitive Theory, Goal Setting Theory  and elements of organizational change theories . The involvement of the target populations continued during the iterative development process of the ZWIP.
The ZWIP can be considered a combination of an Electronic Health Record accessible to the frail older person, informal caregiver and all professionals involved, with a tool for interprofessional and patient-professional communication. The ZWIP consists of (1) information about the frail older person’s health, functioning and social situation, contact information about professionals involved in their care, and care-related goals formulated by or with the frail older person, (2) a secure messaging system for communication between the frail older person and one or more professionals or between professionals, and (3) tailored educational materials for the frail older person and informal caregiver. The frail older persons hold the key to the ZWIP, as they decide which professionals are granted access to their personal ZWIP. The ZWIP can be entered by logging on to a website which conforms to Dutch security regulations. This website, which runs in Dutch, can be accessed from any computer. Frail older persons and their informal caregivers can log on by means of a shared user name and password, while professionals need a security token for logging on. Additional file 1 presents a movie which demonstrates the use of the ZWIP by a frail older person and her informal caregiver.
The ZWIP implementation team consisted of the project manager, physicians, a nurse, a nurse scientist experienced with implementation, and research assistants working for the department of Geriatric Medicine of the Radboud University Nijmegen Medical Centre. They implemented the ZWIP using tailored implementation strategies for each target population, i.e. frail older people and informal caregivers, professionals and the employing organizations of professionals. An overview of the implementation strategies used is provided in Table Table11.
We invited general practices affiliated with this University Hospital or involved in the program’s development to participate in the ZWIP, which was made available to them at no charge for the duration of the study. The participating practices invited local primary care professionals from all relevant disciplines involved in the care of frail older people, such as physiotherapists, district nurses and social workers, to take part in the programs’ interprofessional educational program. This program, for which continuing medical education credits were available, addressed screening for frailty, self-management support, interprofessional collaboration, and use of the ZWIP during three three-hour workshop meetings. In addition, professionals received coaching in specific components of the program, and were supported by a telephonic helpdesk. Further, financial compensation was provided for the time invested in the program.
The general practices screened their populations of ≥70years for frailty using a two-step screening questionnaire (Easycare-TOS). In the first step, the general practitioner (GP) selected patients which were considered (possibly) frail. In the second step, the thus selected patients were screened for frailty on the physical, psychological and social domain during a home visit by a nurse or gerontological social worker. During a second home visit, all people who were frail were invited to participate in the ZWIP. If they gave informed consent, a ZWIP was installed for them.
We supported frail older people and informal caregivers in using the ZWIP by a number of strategies, such as offering a visit by a volunteer who could demonstrate the ZWIP, having a telephonic helpdesk available, and making the ZWIP available in print when this was preferred.
The quantitative evaluation of the implementation of the ZWIP consisted of a survey for professionals, and an evaluation of the data collected about the use of the implementation strategies and the data from the ZWIP itself. Data collected included the numbers of older people screened, the number of participants, the number of messages sent, the number of professionals participating in a frail older person’s ZWIP, the number of participants who logged on to the ZWIP, the number of calls to the telephonic helpdesk and the number of visits to frail older people by volunteers.
The survey for professionals was sent out at the beginning of July 2011. The survey was developed building on existing questionnaires used previously to evaluate the implementation of complex interventions and on previous experience of the authors. The survey included questions concerning demographics, time spent on using the ZWIP, perceived value of the implementation strategies, and barriers and facilitators for the use of the ZWIP. We used separate questionnaires for GPs, for nurses and gerontological social workers conducting the screening, and for other professionals. Participants were asked to fill out the survey online, those who did not respond were sent a paper version.
The qualitative evaluation consisted of semi-structured interviews about experiences with the implementation process and perceived barriers and facilitators for the use of the ZWIP. A topic list for these interviews was developed by members of the research group and was adjusted until consensus was reached. We conducted these interviews with 12 purposively selected professionals, who had a variety of experiences with the implementation process of the ZWIP. This was arranged by selecting professionals from several disciplines and with different roles in the implementation process, who came from three general practices with varying levels of adoption of the ZWIP. In addition, we interviewed members of the implementation team, who were not involved in conducting its evaluation. Interviews were conducted by members of the research group (LvN, MP, SR) and were transcribed verbatim by a research assistant.
We used descriptive statistics to describe baseline characteristics of participants, data collected about the implementation and the actual use of the ZWIP, and data derived from the survey for professionals. The qualitative data gathered in the semi-structured interviews were analyzed by two members of the research group (MP, SR) using content analysis . Interviews were conducted in parallel with data analysis, using Atlas.ti to support this. We conducted interviews until theoretical saturation was achieved.
Fourteen general practices were invited to participate in the study, seven of these (50%) agreed to participate. The characteristics of these practices are shown in Table Table2.2. In total, 290 frail older people and 169 professionals participated in the ZWIP.
A total of 158 professionals received a survey, i.e. 39 GPs, 26 nurses and gerontological social workers conducting the screening, and 93 other professionals. Eleven professionals could not be reached. Hundred-eighteen professionals (75%) returned the questionnaire, 34 were GP’s, 22 were nurses and social workers who conducted the screening and connected frail older persons to the ZWIP, and 62 were other professionals, such as physiotherapists, pharmacists, nurses and social workers.
Twelve purposively selected professionals participated in the semi-structured interviews. Three of them were GPs, three were nurses and gerontological social workers conducting the screening at the homes of participants, three were other professionals, i.e. pharmacists (n=2) or physiotherapist (n=1) and three were members of the implementation team, i.e. project manager, nurse providing coaching, or research assistant.
The 290 frail older people who had a ZWIP account installed constituted 49% of all frail older people invited. The percentage of frail older people agreeing to participate varied over the separate general practices. Interviewees suggested that this may have been caused by variation in local professionals’ attitudes towards the ZWIP, as well as by variation in computer literacy due to social-economic differences between the general practices. An overview of the outcomes of the implementation process is provided in Table Table33.
Most interviewees described being involved in the ZWIP of at least some frail older people, varying from one to tens of people. Some used the ZWIP quite often for a limited number of frail older people, whereas others rarely used it. Interviewees described that most frail older people and their informal caregivers made limited use of the ZWIP. On the other hand, they also gave examples of frequent users of the ZWIP. One interviewee described that the use of the ZWIP was limited when the frail older person was in good health, but that its use increased when the frail older person became ill. Table Table44 provides some illustrative examples of quotes by interviewees.
All but one of the planned implementation strategies targeting professionals (Table (Table1)1) had been available during the implementation period. The development of E-learning for professionals took longer than expected and was therefore not used during the implementation phase. We added one implementation strategy for professionals during the implementation period, i.e. the designation of one key person in each general practice who coordinated the required activities and helped colleagues with questions, as coordinating everything from one central point became too demanding for the implementation team. For frail older people and informal caregivers all planned implementation strategies had been available.
Exposure of professionals and frail older people and informal caregivers to the separate implementation strategies varied over the participating general practices (Table (Table5).5). For instance, professionals’ participation in the educational program varied between 60% and 100%. Their overall exposure to coaching was 47%; it was 95% (20 of 21) for professionals conducting the program’s screening for frailty. Of the participating frail older people and informal caregivers, only 62 had used the offered but not obligatory visits by a volunteer to explain the ZWIP, 63% of GPs (19 of 30) had always or often called their frail older patients themselves to ask them to participate in the screening.
Of the surveyed professionals who had participated in the educational program, 70% (63 of 89) considered it (very) necessary for being able to work with the ZWIP. Interviewees confirmed this, as they felt that meeting each other and gaining knowledge about each others’ expertise during the educational program facilitated collaboration, and they appreciated the opportunity to ask questions and to practice working with the ZWIP. However, they did feel that the educational program could have been shorter, and that too much time had elapsed between the educational program and the first frail older persons having a ZWIP. As for the coaching, only 26% (14 of 53) of surveyed professionals who had received coaching felt they would not be able to work with the ZWIP without it. However, interviewees did consider coaching necessary, as they appreciated the assistance of someone experienced in conducting the screening and entering data in the ZWIP. The helpdesk was considered (very) necessary by 77% (41 of 53) of surveyed professionals who had contacted it, and interviewees agreed that it was useful (Tables4 and and66).
Interviewees stated that for all target populations experiencing problems with interprofessional communication or contacting professionals had been an important incentive for participating in the ZWIP. Additional facilitators for professionals were appreciating that the ZWIP could be used at a time of their choosing, sympathizing with the idea of the ZWIP, enjoying participating in something new, the ZWIP application being user-friendly and receiving sufficient support in working with the ZWIP. Additional facilitators for frail older people were wanting to keep control of their own care, appreciating that their message in the ZWIP was directly and quickly answered by their own GP instead of by the medical assistant, participation of an informal caregiver and the GP being involved.
On the other hand, preferring to have face-to-face contact presented an important barrier to the use of the ZWIP for all target populations. Barriers specific for professionals were considering the ZWIP too early for the current generation of older people whose computer-literacy is limited and having doubts about whether the ZWIP is the best way to improve care. In addition, interviewees considered time constraints an important barrier, even though 67% (64 of 96) of surveyed professionals considered the time spent on using the ZWIP (very) limited. Further, limited use of the ZWIP by both professionals and frail older people presented a barrier to respectively 52% (50 of 94) and 60% (56 of 94) of surveyed professionals (Table (Table7).7). Interviewees agreed with this, as they considered not being invited into the ZWIP of frail older people much, receiving few messages, and not all professionals in their work area being familiar with the ZWIP an important barrier for its use. A final barrier described were the start-up problems experienced by professionals, which included the ZWIP application not working correctly, lack of clarity about the eligibility criteria for older people, and receiving financial compensation too late. Interviewees felt that at the start of the project, the implementation team had struggled with translating the ideas behind ZWIP into everyday practice, sometimes causing support to be lacking or too late. Interviewed members of the implementation team acknowledged these start-up problems, and explained the problems with the ZWIP application by the limited time available for its initial development, resulting in improvements of the ZWIP continuing alongside its implementation. Further, interviewees of the implementation team described that the need for local professionals and organizations to get ready to work together first, and the obligated but time-consuming population-based screening had slowed down the implementation of the ZWIP. Barriers specific for frail older people included considering the ZWIP not useful or quite a fuss, and considering the ZWIP something for professionals. In addition, frail older people were not always invited to participate by a motivated professional or were not considered eligible to participate by professionals. However, the main barriers for frail older people related to computers, i.e. not having a computer, not being comfortable with or capable of working with a computer, concerns about the security of the ZWIP and not yet being familiar with the ZWIP. Although we did direct several implementation strategies at these barriers, such as asking an informal caregiver to use the ZWIP for the frail older person and offering a visit by a volunteer to explain the ZWIP, one interviewee remarked that sometimes, although explicitly offered, frail older people did not want to use these strategies as they did not want to burden them or did not want yet another unknown person in their house (Table (Table44).
Interviewees made several recommendations for improving future implementations of the ZWIP. These included shortening the educational program, having an e-learning and a website available and using early adopters as advocates for the program. For frail older people, interviewees suggested the use of an additional implementation strategy, i.e. organising meetings for all frail older people in which they can learn about the ZWIP and practice its use, in order to familiarise them with the ZWIP within a more comfortable context. Interviewees considered the ZWIP useful for other populations as well, e.g. for frail people younger than 70years, non-frail older people, psychiatric patients, palliative patients, patients with diabetes or COPD and for problematic family situations. Last, interviewees shared some considerations for improvements of the ZWIP itself, which included linking the ZWIP to their own electronic health records, and enabling professionals to use the ZWIP more flexibly, i.e. to use only those parts needed (Table (Table44).
This study describes the implementation of the ZWIP. By the end of the implementation period, 290 frail older people and 169 professionals were involved in the ZWIP. Their use of the ZWIP varied. The implementation strategies were generally delivered as planned for professionals. However, the exposure of frail older people and informal caregivers to some of the implementations strategies, such as their use of the optional instruction about the ZWIP application by volunteers was less than intended. Professionals were generally positive about the implementation process, especially about the interprofessional educational program and the helpdesk. Factors that facilitated the implementation of the ZWIP were frail older people and professionals feeling the need to enhance interprofessional collaboration and the ZWIP application being user-friendly, barriers were the low computer-literacy of frail older people, start-up problems, a preference for personal contact and limited use of the ZWIP by others. Interviewees recommended adapting the implementation strategies to make them more efficient, for example by shortening the educational programme, to use the ZWIP for other target populations as well, and to add new strategies that may help frail older people to feel more comfortable with computers and the ZWIP.
Overall, the results of the implementation were positive. First, the frail older people who were the target population of the ZWIP are likely to be among the most difficult populations to engage in an e-health intervention, as they feel less comfortable and competent with computers than younger populations [19,20]. Therefore, the recruitment of 290 frail older participants (49% of those invited to participate), who were not previously selected for having computer skills, is quite a positive outcome. Of course, their actual use of the ZWIP during the implementation period varied, but those who never or rarely used the ZWIP may not have had a reason to use it, as all went well. In addition, for several frail older people a ZWIP was created near the end of the implementation period, which resulted in them having had limited time to use it. However, participants could continue to use the ZWIP for one year following the end of the implementation period. Additional positive results of the implementation are that the interviewed professionals recommended using the ZWIP for other target populations recommended also using the ZWIP for other target populations, and that professionals and frail older people not yet involved in the current research project had approached us to ask whether they could use the ZWIP as well.
Important factors that contributed to these outcomes were the involvement of future target populations throughout the development process; the implementation strategies such as the interprofessional educational program and the helpdesk, which were considered particularly useful by professionals; and the widely acknowledged need to improve the care for the growing number of frail older people and to improve the communication and collaboration among professionals [21,22]. As the ZWIP incorporates many components of the currently advocated introduction of the patient-centred medical home , e.g. patient-centeredness, care-coordination and the use of e-health such as shared Electronic Health Records to improve quality of care , it fits very well in the improvements currently recommended for primary care.
To our knowledge, the ZWIP is one of the first e-health interventions to combine a multidisciplinary shared Electronic Health Record with interprofessional and patient-professional communication. This, added to the limited number of publications concerning the results of the implementation of e-health interventions , makes comparisons to other studies difficult. However, there have been several articles published concerning the barriers for the implementation of e-health interventions such as electronic medical records or electronic communication. These largely agree with the barriers found in this study, i.e. preferring personal contact, being worried about security of data and time constraints [13,24-26], even though the results on this latter barrier were somewhat mixed in our study, with about two-third of professionals reporting that they considered the time spent on using the ZWIP limited, whereas interviewees reported that time constraints did present a barrier to using the ZWIP. Additional barriers specific to this study were the low computer-literacy of frail older people and the experienced start-up problems.
Our study had some limitations. First, as a result of a deliberate choice, frail older people and informal caregivers’ experiences with the implementation process were only evaluated indirectly. We do acknowledge the limitations of receiving such indirect information. However, we expect that we have been able to give an overall impression of their experiences with the projects’ implementation. Second, the implementation was evaluated by members of the project team who were involved in its implementation. Although we used objective quantitative data sources, and ensured that the interviews with primary care professionals were conducted by an objective research assistant, this may have affected our results. Third, the study was conducted in the Netherlands, which healthcare system is characterized by each patient having their own GP, usually over an extended time period. This may limit the generalisability of our findings to healthcare systems which do not have such a strong primary care foundation. On the other hand, the study had several important strengths, which include the large number of participants, including a large number of frail older people who are usually more difficult to recruit for such projects and a large group of professionals of whom a large majority (87%) had not been involved in the development process themselves, the use of a mixed-methods design which combined multiple data sources, and the ZWIP being implemented directly in everyday practice for use in regular care.
This study has described the implementation of an innovative e-health intervention for community-dwelling frail older people, informal caregivers and primary care professionals, which had positive results. The implementation strategies for professionals, especially the involvement of the target populations in its development, the educational program which resulted in professionals getting to know each other personally and the helpdesk, as well as the experienced need for improving care for frail older people contributed to this. However, the strategies intended to improve frail older people’s computer literacy did not always succeed as exposure to these strategies was limited. Therefore, both additional strategies targeting frail older people’s computer literacy and reviewing the ZWIP to optimise its user-friendliness are needed. Nevertheless, as e-health is an important medium for overcoming healthcare fragmentation and facilitating patient involvement, but its adoption in everyday practice remains a challenge, the results of this implementation are promising.
Based on the current study, several recommendations can be made for the implementation of comparable e-health interventions. First, when e-health innovations are directed at populations who currently have limited computer literacy, such as frail older people, implementation efforts should focus on improving this by e.g. a comprehensive training program . Piloting the implementation strategies selected for this aim, to ensure that they are able to meet the needs of the target population, is highly recommendable. Second, as a preference for personal contact continues to be an important barrier for the use of e-health by both patients and professionals, it should be addressed during the implementation, e.g. by emphasizing that e-health is meant to be an addition to and not a replacement of the existing spectrum of communication methods and by providing professionals engaging in the use of electronic communication the opportunity to get acquainted with each other during the implementation. Last, although some ongoing development is probably unavoidable with many innovative e-health interventions, the resulting inconvenience for professionals should be restricted to a minimum, as the start-up problems caused by working with an application under development are likely to deter participants who were hesitant to adopt these techniques to begin with.
The authors declare that they have no competing interests.
SR was involved in the design of the study, conducted the interviews, conducted the data analysis and drafted the manuscript; MP was involved in the design of the study, conducted the interviews, conducted the data analysis and critically reviewed the manuscript; MH was involved in the design of the study and critically reviewed the manuscript; LvN conducted the interviews and critically reviewed the manuscript; HS was involved in the design of the study and critically reviewed the manuscript; CvW critically reviewed the manuscript; MOR critically reviewed the manuscript; TvA was involved in the design of the study and critically reviewed the manuscript; MMH was involved on the design of the study and critically reviewed the manuscript; RM was involved in the design of the study and critically reviewed the manuscript. All authors read and approved the final manuscript.
The pre-publication history for this paper can be accessed here:
Movie demonstrating the use of the Health and Welfare Information Portal by a frail older person and her informal caregiver. (WMV 16069 kb)
We thank Wilma Derksen-Driessen, Mark Kuster, Charlotte Neger, Jean Nielen and Joep Scheltinga for their assistance with the implementation of the ZWIP.
This work was supported by a grant from the Dutch National Care for the Elderly Programme, coordinated and sponsored by ZonMw, organization for health research and development, The Netherlands [grant number 60-61900-98-129]. The funding agency was not involved in the design and interpretation of the data, the writing of the manuscript or in the decision to submit the manuscript for publication.