Few previous studies have examined changes in referral patterns following implementation of nursing BPGs. We assessed changes in referrals from the perspective of both nurses and patients, as well as using chart audits. The patterns of reported referrals from these information sources offer important insights regarding the BPG implementation process and continuity of care.
Various factors may have influenced the uptake of referral recommendations. First, BPG recommendations for referrals are stated in rather general terms, written so as to be applicable to nurses working in various sectors of the health system. This lack of specificity may have limited their uptake. Second, content knowledge about the health issue is an important aspect of incorporating guideline recommendations into practice. For example the practice recommendations for delirium-depression-dementia in older adults require nurses to identify symptoms of delirium, dementia and depression in order to make a referral for medical diagnosis. The larger pre- versus post- intervention differences in nurses' familiarity scores relative to nurses' self-reported referrals was expected, as awareness about a referral source must precede decisions to make referrals to that source. Third, health care providers need more than knowledge to make a referral. They also need an appreciation of what services a referral source can offer before being convinced that a referral is worthwhile for the client. This was demonstrated, in part, by the pattern of referrals we noted. For three BPGs (asthma, delirium-depression-dementia, venous leg ulcers), referrals to sources that were "in-house" (i.e. offered in the institution where the nurses were employed) were more frequent than referrals to external agencies or resources. For the venous leg ulcer BPG, a significant increase in referrals to specialist nurses may indicate an increased likelihood of referrals when there is a clear horizontal referral mechanism for nurses to refer to other nurses.
CRNs were attempting to implement many BPG recommendations in a relatively short time. Recommendations pertaining to referrals may not have been a consistent priority for all sites. We do not know the extent to which CRNs stressed the importance of referrals in their educational sessions. Previous research has demonstrated the importance of training nurses and other professionals on how to make referrals and how to motivate patients to actually keep referral appointments [20
]. These topics may not have been adequately or consistently addressed during educational sessions, possibly contributing to the lack of positive findings for some BPGs.
Other studies have shown a marked increase in appropriate referrals when this is an explicit and primary intervention focus. For example the introduction of a program aimed at increasing referrals for asthma education of patients consulting at the emergency department for acute asthma found that the number of referrals increased more than 10-fold over four months [21
]. Each Asthma Education Centre kept statistics on the number of patients recruited at the emergency department and referred to the center. This program involved education of nurses and respiratory therapists working in the emergency department and in hospital units. Training was focused on a number of key areas including asthma and its treatment, the role of emergency department staff, key messages to provide to patients, services offered at the asthma education centre, and how to make referrals and skills for approaching and motivating patients. The Robichaud et al. (2004) [21
] study illustrated the importance of addressing the process of making a referral. This may highlight a weakness of the BPG recommendations used by the implementation sites. The RNAO BPG recommendations describe the importance of referrals and identify the types of referrals that should be made. However, they do not emphasize effective strategies for making referrals such as ways to approach and motivate patients to follow-up on referral recommendations.
Several factors may have contributed to the differential patterns of referrals to clinical services within the agency, referral resources external to the agency and Internet sources. Referrals in-house may have led to positive feedback to the nurses on whether patients were finding the referrals helpful. In part, this may have accounted for the observation that referrals within the agency were more likely to increase than other types of referrals. Nurses may also have been more familiar with in-house referral resources and information about these referral resources may have received more emphasis during education sessions. Nurses and patients alike may not have considered suggestions to seek information on the Internet as a "referral" per se. This may have altered their responses to questions about referrals to the Internet. However, it may also reflect the computer skills of participating nurses and their lack of up-to-date information on credible Internet sites. Nurses have been found to prefer more interactional sources of knowledge compared to e-mails and the Internet [22
]. Finally, teams of nurses care for patients. This provides an increased chance of "slippage" if one nurse assumes that another nurse has initiated a referral or discussed a potential agency where the patient can self-refer. Without a formalized means to chart information about the status of patient referrals, the exchange of information about referrals to different resources may be uneven among nurses and between nurses and other health providers.
The focus of this study was on patient referrals that were initiated following introduction of the BPG recommendations. Since making a referral often requires an inter-disciplinary approach, future inquiries should also assess changes in familiarity with referral sources and in reported referrals by other members of the interdisciplinary health care team.
The type of clinical setting where BPG recommendations were implemented may also have influenced referral patterns. Some guidelines are more suited to recommending referrals that would be within nurses' scope of practice. Furthermore, the breastfeeding and smoking cessation guidelines recommend community organizations that are open to self-referrals. However, asthma, venous leg ulcers and delirium-depression-dementia BPGs have a predominance of referral recommendations with a medical focus. Thus, depending on the BPG and the setting, initiating a referral may require a physician's order. However, knowledge of referral sources, increased awareness of the value of appropriate referrals, and familiarity with the means to support patients in following up referrals may all influence effectiveness of the referral process. Differences in intra- and inter-organizational context (e.g. patterns of service delivery within an organization and established inter-organizational partnerships to facilitate referrals), and diverse organizational expectations of nurses' practice (e.g. job descriptions for nurses) may have influenced referral patterns. These factors create different conditions for new referral processes to take hold. However, it is not possible to separate out the relative influence of these contextual factors on referral processes within the participating agencies in this study. Future studies should aim to examine the contribution of these factors on referral processes, thereby providing critical information on the mechanisms by which new referral processes take hold within and between agencies and further guidance on the transferability of findings.
We recommend that health care agencies interested in helping their clients access community resources work collaboratively with agencies in their community to develop strong documentation systems including the use of electronic or web-based formats for referral. While identifying referral sources for inclusion in our tools, CRNs identified many community and Internet-based sources for referral that had been previously untapped by nurses in their setting. Anecdotal evidence through comments of participants indicated that better collaboration between hospital and community sectors is also needed. Prompts on the chart and a rapid web-based or email system would support systematic tracking of referral recommendations and their uptake by patients.
This study examined referrals made by the same nurses prior to and following implementation of the BPGs. Although our study was prospective, there was no concurrent control group of agencies where BPGs were not being implemented. A concurrent control group would be an important addition to future studies to adjust for factors other than BPG implementation that may have changed nurses' referral patterns. In addition, future studies need to have larger numbers of patients for the measurement of longer-term referral outcomes.
The timing of patient interviews may have introduced a recall bias. Response rates for patient interviews were lower than for nurses and thus, patient interview data is likely from a non-representative sample. This is especially likely for the post-implementation patient interviews for smoking cessation and diabetes BPGs. Thus, pre- and post-comparisons of patient reports of referrals should be made cautiously. There was a difference in the nurses' and patients' reports of referral patterns. Since we were not able to match patients to assigned nurses, we cannot determine whether the patients' and nurses' reports of referrals are discrepant, or whether the patients' experiences were with nurses other than those who completed the questionnaire. We did not ask nurses to indicate how many referrals they had actually made in the past month. This might have provided a better gauge of any shift in the nurses' actual referral practices.
A mixed methods study would reveal further insights into what constitutes a referral from the perspectives of patients, families and nurses and how this perspective influences responses to questions about referral practices.