|Home | About | Journals | Submit | Contact Us | Français|
Integrating electronic referral systems into clinical practices may increase use of web-accessible tobacco interventions. We report on our feasibility evaluation of using theory-driven implementation science techniques to translate an e-referral system (ReferASmoker.org) into the workflow of 137 community-based medical and dental practices, including system use, patient registration, implementation costs, and lessons learned. After 6 months, 2,376 smokers were e-referred (medical, 1,625; dental, 751). Eighty-six percent of the medical practices [75/87, mean referral = 18.7 (SD=17.9), range 0–105] and dental practices [43/50, mean referral = 15.0 (SD=10.5), range 0–38] had e-referred. Of those smokers e-referred, 25.3 registered [mean smoker registration rate—medical 4.9 (SD=7.6, range 0–59), dental 3.6 (SD=3.0, range 0–10)]. Estimated mean implementation costs are medical practices, US$429.00 (SD=85.3); and dental practices, US$238.75 (SD=13.6). High performing practices reported specific strategies to integrate ReferASmoker.org; low performers reported lack of smokers and patient disinterest in the study. Thus, a majority of practices e-referred and 25.3 % of referred smokers registered demonstrating e-referral feasibility. However, further examination of the identified implementation barriers is important as of the estimated 90,000 to 140,000 smokers seen in the 87 medical practices in 6 months, only 1,625 were e-referred.
The online version of this article (doi:10.1007/s13142-013-0230-3) contains supplementary material, which is available to authorized users.
Smoking is the most important preventable cause of premature death in the USA [1–5]. The Center for Disease Control (CDC) estimates that the medical costs of treating smokers is more than US$96 billion a year including another US$97 billion a year from lost productivity . While smoking cessation efforts have achieved some success, cessation rates are not as high as desired . To impact national cessation rates, smoking cessation support must be widely disseminated and readily available . Easily accessible public health interventions, like quitlines and quit smoking Internet systems, may improve smoking cessation rates [3, 9–15], but are under-utilized . New methods are needed to proactively recruit smokers to these available, effective smoking cessation systems.
Having providers directly refer smokers to these public health interventions can increase their use . Providers have reported several barriers to referring smokers, including lack of time due to competing demands and lack of referral feedback ; however, some providers have suggested that a referral system offering a single source with feedback and support would facilitate referrals. Fliers and fax-referrals have been used by providers; fax-referrals, a proactive handoff from provider to quitline, have been more successful in engaging smokers than fliers . Proactive referrals have mainly been used with quitlines and not with web-assisted tobacco interventions. For these Internet systems, automated electronic referrals (e-referrals) could directly link patients to the intervention. In an e-referral system , healthcare providers, at the point of care, can advise patients to use an online behavioral intervention and, with patient agreement, provide the online behavioral system a contact identifier (e.g., the patient’s email). The system can then proactively use this patient contact to send persuasive messages encouraging participation. In addition to being proactive, e-referral systems also have the ability to provide real-time feedback to providers. Patients can be engaged electronically (e.g., via emails) rather than by phone, which increases the efficiency of the system. While the use of paper and fax-referral systems to increase access to quitlines has been studied, much less is known about using e-referral systems to increase use of an evidence-based, web-assisted tobacco intervention.
The term implementation research refers to the “scientific study of methods to promote the systematic uptake of research findings and other evidence-based practices into routine practice and, hence, to improve the quality and effectiveness of health services and care .” Implementation science is the corresponding body of knowledge regarding those methods and the irrespective characteristics . Implementation science is a critical component of translational behavioral medicine and is increasingly supported by the National Institutes of Health through new program announcements, an annual meeting, and a summer training institute . Consistent with these goals and guided by the Promoting Action on Research Implementation in Health Services (PARIHS) framework , we have evaluated the feasibility of theory-driven implementation science techniques to translate an e-referral system (ReferASmoker.org) into the workflow of these clinical practices (87 medical and 50 dental) recruited nationwide. In this paper, we report on the results of our feasibility evaluation, including the variations of use of the system by providers to refer smokers, the proportion of e-referred smokers, and the lessons learnt implementing the system.
As part of a prospective translational behavioral medicine feasibility evaluation, we used implementation research methods to translate an electronic referral to a patient web-assisted tobacco intervention into a cohort of community-based medical and dental practices recruited nationwide. This study was conducted within the context of two larger randomized trials evaluating the effectiveness of the e-referral system . This study was approved by the University of Alabama at Birmingham and University of Massachusetts Medical School Institutional Review Boards.
Eighty-seven primary care medical practices from 28 US states were recruited from a registered database of internal medicine and family practice clinics. Fifty dental practices from seven US states were recruited from state lists of registered dentists. Practices were recruited using nationwide mass mailing. These practices were recruited for two federally funded pragmatic trials evaluating Decide2Quit.org, a patient-centered web-assisted tobacco intervention [QUIT-PRIMO (Quality Improvement in Tobacco Provider Referrals & Internet-delivered Microsystem Optimization) is a study of our patient-centered, web-assisted tobacco intervention Decide2Quit.org in primary care medical practices ; the HI-QUIT (Hygienists to Internet Quality Improvement in Tobacco) study is a similar study of Decide2Quit.org in dental practices]. Recruitment continued until recruitment goals for evaluation of the patient-level intervention were met. In this report, we focus on the implementation of the practice-level implementation of “e-referrals” as described below.
The ReferASmoker.org e-referral system facilitated referrals to Decide2Quit.org, an evidence-based, web-delivered smoking cessation self-management system [20, 25, 26]. ReferASmoker.org included a web form where providers entered smokers’ email addresses (Appendix 1). This form was designed to be easily completed by nursing or front office staff as the smoker was discharged from the visit. Once referred, smokers received up to 10 emails encouraging registration at Decide2Quit.org. The system also provided real-time feedback to the providers about their smokers’ use of the Decide2Quit.org system.
A multi-pronged, facilitation-based implementation program, guided by the elements of the PARIHS framework, was instituted to support the e-referral system. The PARIHS framework posits that the success of a particular implementation effort is a function of three basic elements: the evidence associated with the implementation, the context in which implementation will occur, and the facilitation of that process . Successful implementation is most likely in cases when evidence is high and reflects clinical experience and patient preferences, a clinical context is receptive to change and characterized by accommodating culture and appropriate evaluation mechanisms, and suitable processes and resources are available to facilitate and sustain change . When used as a guiding framework, PARIHS can highlight important evidence and contextual factors that may influence an implementation effort, and can inform the development of a suitable facilitation-based implementation strategy to address those factors .
Our previously reported pre-implementation evaluation work  identified a range of contextual factors to address in our implementation program, including communication and ownership issues, limited training on the e-referral system and corresponding registration challenges, lack of motivation to use the system and a tendency for practices to forget to refer patients for participation, as well as a key evidence-related factor, perceived potential for the system to positively affect care. Based on these insights, we designed an implementation program comprised of the following supports to facilitate program uptake and sustained use. First, individualized telephone/Internet trainings using an academic detailing approach were provided to up to two implementation coordinators (physicians, nurses, dentist, hygienist, or other staff) at each practice. Training included hands-on demonstrations of the ReferASmoker.org website, including initial registration; practice e-referring a “test” smoker; and exploring Decide2Quit.org, the web system for smokers. As a guide, a detailed training script was developed (Appendix 2). This training was equivalent in both studies, but two practice participants were trained in physician practices, and only one was trained in dental practices (due to the smaller size of these practices). Incentives were provided for completing training but not for subsequent use of the system. To support implementation, we provided workflow support materials, including information prescription pads to aid referrals and fliers to act as visual cues to remind providers to refer smokers to the study.
We also provided a booster program to support implementation. Again, this varied in the dental and medical practices. In both practices, a series of four motivational booster emails were sent over a 6-week period. The medical practices had a total of six booster calls: three were 15 min long, and two were 30 min long. There were a total of three booster calls to the dental practices: two were 15 min long, and one was 30 min long.
During initial training and booster calls, we used Motivational Interviewing techniques  including accurate empathy, reflective listening, and overcoming ambivalence. For example, if a study staff member had attempted to contact a particular Implementation Coordinator on multiple occasions and failed to do so, we focused on positive reflective statements to increase the likelihood of the practice continuing to be active in the study.
Consistent with the goals of an implementation science study, we did not have strict guidelines on who to refer or the number of referrals required of practices, nor did we provide referrals incentives. Practices were told that the web-assisted tobacco intervention had content for all smokers, regardless of their readiness to quit. We provided them the system and training, but the adoption and use of the system was left to them.
Practice characteristics were collected through Internet and paper-based surveys. All our interactions with the practices (training, booster calls, etc.) were logged by our research staff. A qualitative survey was also conducted with the practices at the end of 3 months assessing the benefits of the various components of the implementation program and reported barriers to implementation. This survey was completed by the person identified as the “implementation coordinator” for the practice, most often a nurse or dental hygienist.
The primary measure of practice participation, or provider performance, was the number of e-referrals completed by end of follow-up. E-referrals by providers, smoker information, and activity were tracked through web analytics. We also tracked whether an e-referred smoker registered with the system. Each e-referred smoker’s email was logged by ReferASmoker.org. Once a smoker registered with that email, they were then linked in the database with their practice of origin. We report the proportion of smokers who registered overall and the rate of referral and smoker registration per practice in the first 6 months. When we use the term “rates,” we are referring to a count over unit time (6-month period). Referral rate was defined as the number of referrals per practice per 6-month time period. Registration rate was defined as the number of registrations per practice per 6-month time period.
We then estimated marginal costs to implement the e-referral system at each practice, including the mean cost of implementing the system per registered smoker. Costs were estimated from an organizational perspective (our costs); we included projects costs, not additional time costs for providers or smokers. There were two costs: (1) baseline training and setup, and (2) follow-up booster calls. The baseline training and setup costs included (a) project staff time for call attempts to setup training, training sessions, and preparation for mailing project supplies and incentives, and (b) reimbursement paid to practices for time spent on training. We used the following estimated times: each training call attempt (5 min), training call sessions (30 min), and mailing preparation (15 min). The reimbursement for the training was US$150/session. The follow-up booster calls costs included the project staff time for call attempts to setup booster calls and the time for booster call sessions. Project staff costs were calculated at US$25/hour based on mean salary for a person with master’s-level public health training.
To identify barriers and facilitators to referral success, we conducted a case-comparison study. Based on measured provider performance, we identified the top and bottom five referring practices, both medical and dental. Two members of our team reviewed notes from the training and booster calls and the qualitative survey to identify themes. The themes were then reviewed by the larger investigator group, and we created a summary of key points.
The 87 medical practices were 41 % internal medicine and 59 % family practice. The 50 dental practices were mostly solo practices (77 %). Medical practices reported seeing a median of 125 smokers per week, while dental practices reported seeing 100 smokers per week. Providers were mostly female and white (Table 1). Overall, 151 medical and 50 dental providers were trained.
After 6 months of follow-up, 75 of the 87 medical practices and 43 of the 50 dental practices used the system to refer smokers. Overall, practices referred 2,376 smokers; medical practices referred 1,625 smokers while dental practices referred 751 smokers. The mean referral rate in the 6-month evaluation period at the medical practices was 18.7 (SD=17.9), range 0–105, and the mean referral rate at dental practices was 15.0 (SD=10.5), range 0–38.
Overall, 601 smokers (25.3 %, 95 % CI 23.6 % to 27.1 %) registered on Decide2Quit.org. Of smokers referred by medical practices, 422 registered (26.0 %, 95 % CI 23.9 % to 28.2 %); the mean registration rate per practice was 4.9 (SD=7.6), range 0–59. The registration rate of smokers referred by dental practices was 23.8 % (N=179, 95 % CI 20.8 % to 27.1 %); the mean registration rate per practice was 3.6 (SD=3.0), range 0–10. Registered smokers were predominantly female (64.0 %), white (87.6 %), and thinking of quitting (76.0 %) (Table 2).
As the implementation intensity varied, so did the cost. For the medical practices, the mean estimated cost for the training sessions was US$374.50 (SD=76.7); the mean estimated cost for the booster calls was US$54.40 (SD=19.9). The mean estimated total cost was US$429.00 (SD=85.3). The total cost to implement the e-referral system in 87 medical practices was US$36,890.80; the cost per smoker was US$87.40.
For the dental practices, the mean estimated cost for the training sessions was US$201.20 (SD=12.7); the mean estimated cost for the booster calls was US$37.54 (SD=6.2). The mean estimated total cost was US$238.75 (SD=13.6). The total cost to implement the e-referral system in 50 dental practices was US$11,937.50; the cost per patient was US$66.69.
Based on field notes, our top five referring practices during training showed enthusiasm for the project, were inquisitive about the various components of the system, and had preliminary ideas about how the system could be integrated into their workflow. Based on qualitative data from booster calls, these practices could best be described as proactive in their integration of the system within their workflow. For example, one practice kept information prescription pads at the nurses’ station and used it to begin a discussion with the patient about quitting smoking and the study. The physician would further discuss the study with the patient and if the patient agreed, forward the pad to the nurses for referring the smoker. Our highest referring practice integrated the information prescription pad into their electronic medical record, including setting up reminders to refer smokers.
In comparison to these, our bottom five referring practices were notably less proactive in their approach. They also reported low numbers of smokers in their practice and that smokers were not interested in their system. Many of these low-performing practices also had multiple staff turnovers, making it difficult to implement the system.
During the training, practices identified several barriers and facilitators to implementing the e-referral system. Barriers manifested at both the patient and practice levels. At the patient level, reports of lack of interest in quitting, uncertainty about using the website, and possible repercussions from participating in the smoking cessation project were common. For instance, practices received feedback from long-time smokers, such as “Why stop now?”, and an overall negative attitude towards smoking cessation efforts. In addition, some practices had problems with patients with little experience with technology or having no access to email or Internet. One practice stated that patients had “no interest if they [the patient] had no Internet access,” and another had patients say “they thought they would get more ‘spam’” and had an initial distrust of the website. In addition, one practice had a patient express the concern that “their insurance would be notified if they did [the program] and their rates would go up.” These patient-expressed barriers all presented pressing challenges to implementation. Barriers were also observed at the practice level and included hesitations about the e-referral system, difficulty with the use of materials, and staffing issues. For instance, one practice wanted to wait and see “how many are referred before implementing in the whole practice,” which contributed to less staff members implementing the project. In other instances, practices reported difficulty tearing pages from the prescriptions pads as well as staff turnover resulting in limiting their ability to designate and delegate responsibilities related to referral. While barriers towards implementation existed, some facilitators were identified among the practices. Many practices reported feeling that the training was helpful (i.e., “the process was simple” and “easy to use”) and that incorporating all staff members into the training eased implementation. In addition, many practices took the initiative to tailor the timing of the program to the needs of their practice, for instance, identifying the appropriate time to discuss the referral as “while waiting for the dentist.” Other practices “built in reminders into the system,” or “kept the pad within view to remember to discuss with the patient,” or “scanned the InfoRx into the electronic medical record.” Practices also utilized many communication channels to build awareness about the program among patients and staff. These included poster displays and posting information on the office Facebook page, office website, and/or office monitors. The high level of awareness not only generated interest from patients but also assisted in reminding staff about the project. Most practices, though, felt implementation was easy to do because the website “covered all information needed” and was patient-centered and appropriate (i.e., “not invasive”).
Previous studies have reported difficulties in engaging smokers in public health interventions [30–32]. Only about 5–14 % of smokers follow through with treatment referrals after being advised to quit and less than 7 % of smokers in the USA enroll in clinic-based cessation programs [33–37]. We tested the feasibility of implementing the ReferASmoker.org e-referral into the clinical workflow of 136 medical and dental practices recruited nationwide using theory-grounded implementation research methods. The finding that 86 % of both the medical and dental practices used the system and 25.3 % of smokers subsequently registered with the system is highly encouraging and suggests the feasibility of implementing such a system in diverse, real-world practice settings.
However, the number of smokers e-referred was only a small fraction of the total smokers seen. We estimated that medical practices had 282,750 clinic visits made by smokers in 6 months (125/week times 26 weeks times 87 practices), while dental practices had 130,000 (100/week times 26 weeks times 50 practices). These represent referral opportunities, but not necessarily unique individuals. As smokers often have chronic medical comorbidities, we estimate that many smokers will be seen more than once. If the practice saw each smoker twice, then that would indicate a total of 141,375 and 65,000 dental smokers. If each smoker was seen an average of three times, then there would be 94,250 medical smokers and 43,333 dental smokers. A more nuanced examination of the kinds of barriers and facilitators to implementation that we identified across our cohort of practices would be an important next step towards improving the reach of e-referral systems for smoking cessation. While we understand from our work that the higher referring practices were more proactive in the integration of the e-referral system into their workflow and the lower referring practices struggled with patient resistance and the integration of e-referrals into the context of a visit, continued study could identify additional strategies to address the most pressing patient- and practice-level barriers. The methods of formative evaluation fieldwork, particularly in-depth interviews and workflow mapping, seem particularly promising in this regard.
Other, less proactive provider referral approaches have had only limited effect in recruiting smokers. Gordon et al.  used a multi-pronged recruitment approach including direct-to-patient mailings and provider referrals using brochures to recruit to an online smoking cessation system. Tobacco control and healthcare professionals (n=1,244) were mailed print brochures, and display units were also sent upon request to health care organizations. Only 95 (3.8 %) of all participating smokers were recruited from provider referrals out of a total 2,523 users. The National Colorectal Cancer Research Alliance (NCCRA) and OncoLink distributed fliers through major pharmacies to recruit smokers into an online intervention . Only 7.3 % out of 2,162 smokers were registered through this method. Our rate of registration (25.3 %) far exceeded these prior studies and is likely due to the detailed provider training that we offered as part of our implementation program as well as proactive email follow-up after e-referral.
Fax-referrals with proactive call-backs have been more successful. Secondary analysis of fax-referrals from Arkansas Children’s Hospital revealed that out of 749 smokers, 157 (21.0 %) enrolled in a treatment program and received one or more treatment sessions . In an observational study of faxed referrals to the Ohio Tobacco Quit Line, 23 % (n=1,616) were able to be enrolled out of a total of 6,951 faxed referrals . The success rate of fax-referrals shows that proactive approaches are superior to passive (e.g., brochure-based) approaches in recruiting smokers. The e-referral method in our study in which proactive emails were sent following referrals had similar results to fax-referrals.
Effective smoking cessation programs have the potential to dramatically decrease health care utilization and costs [41–44]. A cost–benefit analysis found that the ratio of benefits to cost varies from US$0.80 to US$2.40 saved per dollar spent on smoking cessation programs, depending upon the type of intervention . Quitters have reduced heath care costs and utilization within a 5- to 6-year period post-cessation. Quitters without chronic conditions have health care costs comparable to never smokers’ within 5 years of quitting; quitters with chronic conditions have health care costs comparable to never smokers’ within 10 years [45, 46]. The cost per registered smoker using our e-referral approach was medical practices, US$87.40, and dental practices, US$66.69, and as the number of patients recruited increases, this will decline. For comparison, in the Gordon et al. study described above , the recruitment cost of smokers using provider referrals was calculated to be US$597 per participant based on the cost of mailing the fliers to the providers.
In addition to further analysis of barriers and facilitators as noted above, increasing the adoption of e-referral system will require understanding how the implementation of these systems will be funded. There is a push toward a payment model linking provider reimbursement with patient outcomes. The U.S. Department of Health and Human Services recently announced that practices will be paid for smoking cessation sessions under Medicare Parts A and B . Practices can expect to receive US$10 to US$15 for shorter smoking cessation sessions and US$25 to US$30 for longer ones. A trained office member is expected to provide this smoking cessation counseling; however, most practices do not have office members trained in smoking cessation counseling. These practices might be willing to refer smokers outside using systems like ours and then have feedback reports that can be used as documentation to obtain reimbursements. We expect the demand for smoking cessation referral services to continue to grow with the implementation of the Accountable Care Organization (ACO) , a newer payment model to be formed as part of heath care reform tying provider reimbursements to quality metrics and reductions in the total cost of care for an assigned patient population.
There are multiple strengths to our study. To our knowledge, our paper is the first to report on the results of using rigorous implementation research methods to translate an innovative e-referral system into the workflow of clinical and dental practices. Proactive e-referral systems, if integrated effectively into clinical settings, can be a powerful additional method to recruit smokers to public health interventions. Thus, our study is an important contribution to the field of implementation science. We implemented this system in a large number of heterogeneous, non-academic, community-based clinical settings across the USA, enhancing the value of this feasibility assessment. While the overall number of patient referred was low, we have demonstrated feasibility, but have also identified barriers to address in future work.
One limitation of our study is that our practice-level rates of referral may be underestimated. Although we have comprehensive user tracing logs, we did have some patients who registered but did not identify a provider practice. Also, our cost analysis is only from the perspective of the system implementers, not the individual practices. We do not have detailed estimates of the cost required by practices to refer a smoker. Although we studied the e-referral practice in multiple practices settings (medicine and dentistry) across multiple states and found similar results, we have not assessed the potential of e-referrals for other patient behaviors, such as medication adherence, diet adherence, or weight control. Further research is needed to generalize these findings.
Our findings demonstrated that there is potential to increase use of web-assisted tobacco interventions through medical and dental practices e-referrals, but barriers remain. E-referral systems have significant potential for the future of eHealth, but more work needs to be done to understand how to maximize referrals and subsequent participation.
Funding for these studies were received from the National Cancer Institute, grant 1 R01 CA129091, and the National Institute of Dental and Craniofacial Research, grant U01 DE16746, U01 DE16747, and U19 DE22516. Research reported in this publication was also supported by the National Center for Advancing Translational Sciences of the National Institutes of Health under award number UL1TR000161. The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH.
The authors have no conflict of interest to disclose.
Trial registration: Web-delivered Provider Intervention for Tobacco Control (QUIT-PRIMO)—a randomized controlled trial: NCT00797628.
The Quit Primo and National Dental PBRN Collaborative Group comprises practitioners, faculty, and staff who contributed to this activity. A list of these persons is at http://www.dpbrn.org/users/publications/Default.aspx
Policy: E-referral systems have the potential to increase access to public health interventions
Practice: Patients will access public health interventions if referred by providers
Research: Providers will e-refer at the point-of-care, but will need to be guided by a detailed implementation program