PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of nihpaAbout Author manuscriptsSubmit a manuscriptHHS Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
 
J Am Med Dir Assoc. Author manuscript; available in PMC 2010 March 29.
Published in final edited form as:
PMCID: PMC2846624
NIHMSID: NIHMS99413

Provider Management of and Satisfaction with Laboratory Testing in the Nursing Home Setting: Results of a National Internet-Based Survey

Brian H. Shirts, MD, PhD,1,2 Subashan Perera, PhD,3,4 Joseph T. Hanlon, PharmD, MS,3,5,6,7 Yazan F. Roumani, MS, MBA,3 Stephanie A. Studenski, MD, MPH,3,5 David A. Nace, MD, MPH,3 Michael J. Becich, MD, PhD,8,9 and Steven M. Handler, MD, MS, CMD3,5,8

Abstract

Objectives

To describe the management of and satisfaction with laboratory testing, and desirability of laboratory health information technology in the nursing home setting.

Design

Cross-sectional study using an Internet-based survey.

Participants and Setting

National sample of 426 nurse practitioners and 308 physicians who practice in the nursing home setting.

Measurements

Systems and processes available for ordering and reviewing laboratory tests, laboratory test result management satisfaction, self-reported delays in laboratory test result review, and desirability of computerized laboratory test result management features in the nursing home setting.

Results

A total of 96 participants (48 physicians and 48 nurse practitioners) completed the survey, for an overall response rate of 13.1% (96/734). Of the survey participants, 77.1% had worked in the nursing home setting for more than 5 years. Over half of clinicians (52.1%) reported three or more recent delays in receiving laboratory test results. Only 43.8% were satisfied with their laboratory test results management. Satisfaction was associated with keeping a list of laboratory orders and availability of computerized laboratory test order entry. In the nursing home, 35.4% of participants reported the ability to electronically review laboratory test results, 12.5% and 10.4% respectively had computerized ordering of chemistry/hematology and microbiology/pathology tests. The following three features were rated most desirable in a computerized laboratory test result management system: showing abnormal results first, warning if a test result was missed, and allowing electronic acknowledgment of test results.

Conclusion

Delays in receiving laboratory test results and dissatisfaction with the management of laboratory test result information are commonly reported among physicians and nurse practitioners working in nursing homes. Test result management satisfaction was associated with computerized order entry and keeping track of ordered lab tests, suggesting that implementation of certain health information technology could potentially improve quality of care.

Keywords: Laboratory techniques and procedures, laboratories, nursing homes, medication monitoring

INTRODUCTION

In ambulatory care and hospital settings, a large proportion of physicians express dissatisfaction with how they receive laboratory results and communicate this information to patients and other clinicians.13 Delays in receiving laboratory test results are common and have been found to be correlated with levels of dissatisfaction with laboratory test result handling.4 Failure to follow-up on completed laboratory results represents about 20% of all laboratory testing errors in both inpatient and outpatient settings.1, 5 The majority of these problems involve delays or failure in reporting laboratory results to physicians, or failure to appropriately respond to abnormal laboratory results when they are made available.5 Errors and delays in laboratory test result follow-up have also been shown to be a major cause of malpractice liability.6 Although laboratory test result follow-up has been studied in hospital and ambulatory care settings,3, 4 it has not been well-characterized in the nursing home setting.

Laboratory testing and follow-up are particularly important in nursing homes because residents receive on average 8.8 medications per day, placing them at considerably increased risk for developing adverse drug reactions (ADRs).7, 8 ADRs are defined as unintended or noxious responses to a drug given in a dosage intended for prophylaxis, diagnosis, or therapy.9 In the nursing home setting, ADRs represent the most common form of medication-related adverse events and have an incidence ranging from 1.19 to 7.26 per 100 resident-months.10 The majority of ADRs are preventable and occur at the monitoring stage of the medication use process, which frequently involves laboratory testing.11, 12 Potentially reducing ADRs by better understanding and improving the laboratory ordering and reporting processes may represent an important way to improve medical care among nursing home residents.

The nursing home setting can present unique challenges to laboratory test result follow-up. Physicians and nurse practitioners commonly care for residents at multiple facilities that often have different laboratory test ordering and reporting protocols.13 There is often insufficient nursing staff and difficulty recruiting and retaining qualified staff.14, 15 Moreover, nursing staff assigned to review incoming laboratory test results may have inadequate training to recognize and communicate normal, abnormal, and critical values.16, 17 Laboratory test results are frequently reported after hours to an on-call physician not familiar with the resident’s medications or medical problems.18, 19 Furthermore, laboratory specimens may be sent to multiple testing facilities from a single nursing home to comply with differing insurance plan regulations, further complicating test management. Because the majority of nursing home residents are cognitively unable to check-up on laboratory test results, the safety net of direct patient vigilance is often absent.20

In our cross-sectional study, we designed and administered an Internet-based survey to a national sample of physicians and nurse practitioners working in the nursing home setting. The objectives were to determine the methods available for ordering and reviewing laboratory tests, laboratory test result management satisfaction, and desirability of computerized laboratory test result management features in the nursing home setting.

METHODS

Study Participants and Setting

We surveyed physicians and nurse practitioners because these professionals are directly involved in the process of ordering laboratory tests and reviewing subsequent results in the nursing home setting. Potential physician participants were identified from a list of 327 American Medical Directors Association (AMDA) Foundation long-term care Research Network member email addresses. The AMDA Foundation long-term care Research Network provides a mechanism to meet the needs of both community-based medical directors and established investigators, and to advance the research agenda for long-term care. Potential nurse practitioner participants were identified by obtaining e-mail addresses of a random sample of 500 of the approximately 1,000 members of the National Conference of Gerontological Nurse Practitioners (NCGNP). The NCGNP is the primary organization for advanced practice nurses who want to pursue continuing education in gerontological care and who seek peer support from experienced clinicians.

Survey tool

For the purposes of this study, we defined laboratory testing to include chemistry, hematology, microbiology, and anatomic pathology tests. In the survey, we asked participants to indicate their profession (physician or nurse practitioner), gender, facility characteristics in which they worked (nonprofit, for-profit, government, number of beds), and duration of time working in the nursing home setting. To determine the presence of laboratory health information technology (HIT) currently available to participants, we used five “yes or no” questions, two questions about ordering (i.e., computerized provider order entry [CPOE]), two about reviewing (i.e., electronically view) laboratory test results, and one question about computerized decision support for either task.

The remainder of the survey contained questions about laboratory test result management systems adapted from the Poon et al survey.4 This survey assesses systems used for laboratory test result tracking, self-reported delays in reviewing laboratory results, satisfaction with test result management, and the desirability of certain features in an electronic laboratory results management system. Our survey included minimal modifications to the Poon et al 2004 survey to reflect situations more common in the nursing home setting. Modifications included changing questions to reflect the potential role that family and caregivers play in the management of nursing home resident’s laboratory test results. Also, questions were reordered so that provider satisfaction was assessed before asking questions about potential communication difficulties and late test results (Appendix).

When describing systems of test result management, clinicians were able to select multiple methods of communication (e.g., phone, pager, fax, and flagging in the chart). To measure delays, clinicians were asked, “How many times over the last 2 months have you reviewed nursing home laboratory test results that you wish you had known about earlier?” Satisfaction with laboratory result management was graded on a five-point scale from “very satisfied” to “very dissatisfied.” To assess desirability of laboratory health information technology (HIT), participants were asked to rate the utility of eight potential features of a computerized system for laboratory test result management on a seven-point scale from “useless” to “indispensable.”

We asked two physicians and two nurse practitioners to pilot-test the survey before we administered it to others. The University of Pittsburgh Institutional Review Board approved the study as exempt, and informed consent was not needed for study participation.

Survey Administration

Between September and October 2007, we contacted potential survey participants using the email addresses provided by their professional organization (the AMDA Foundation Research Network or the NCGNP). The initial e-mail described the purpose of the study and invited those that were interested to click on a link that would take them to an Internet-based survey. If there was no response, potential participants were sent a reminder email after 2 weeks. Upon entering the website, participants were required to enter their email address, and choose to receive a $5 gift certificate or donate $5 to their professional organization as an incentive for completing the survey.

Statistical analysis

For purposes of calculating response rates, we defined potential participants as individuals who had a valid e-mail address. If an individual’s e-mail address was invalid (i.e., if it did not allow the e-mail to be delivered within a 1-week period), the individual was removed from the list of potential participants. To calculate the overall response rate, we divided the number of actual participants by the number of potential participants.

We used descriptive statistics to summarize participant responses to each specific question and demographic data. Chi-square, Fisher’s exact, and Kruskal-Wallis tests were used to make comparisons between professions.21 We used Kendall’s tau and Cochran-Armitage trend test to examine association between laboratory HIT availability, delays in laboratory test result reporting, and test result management satisfaction.21 We used SAS® version 9 for Windows (SAS Institute, Inc., Cary, NC) for all statistical analyses.

RESULTS

Response Rates and Participant Characteristics

Of the 827 emails, 93 did not reach the intended recipient. Of the 734 individuals with valid e-mail addresses, 96 (13.1%) completed the survey. Half of the 96 participants were physicians, and response rates were slightly higher among physicians (15.6%) than among nurse practitioners (11.3%) (Table 1). Most physicians worked at nonprofit facilities, while most nurse practitioners worked at for-profit facilities (p=0.004). The vast majority (77.1%) of participants had worked in the nursing home setting for more than 5 years, with physicians reporting more nursing home work experience than nurse practitioners (p<0.001). Nurse practitioners spent a larger percentage of their clinical time providing nursing home services (p=0.004), and worked in smaller nursing homes (p=0.033).

Table 1
Characteristics of respondents, stratified by profession

Current Management of Laboratory Tests

There was substantial variability in both normal and abnormal laboratory test result reporting mechanisms. Participants were informed of normal laboratory results via flagging results on chart (49.0%), by phone (41.6%), and by fax (35.4%). Sixteen participants (16.7%) reported that normal results were not routinely communicated to them. Participants were usually informed about abnormal results via pager (37.5%), flagging results on chart (30.2%), fax (30.2%), and phone (24.0%). Critical results were most often reported by phone (81.2%) and by pager (51.0%). Few (4.2%) participants reported failing to hear about abnormal results, and no participants reported failing to hear about critical results.

In the nursing home setting, 52.1% of participants reported three or more delays in receiving laboratory test results in the past 2 months, while 80.2% reported at least one delay. There was no significant difference between physicians and nurse practitioners in incidence of self-reported delays (p=0.178).

Twenty percent of participants reported spending over one hour a day reviewing and managing laboratory test results. Physicians spent less time than nurse practitioners on average (16–30 versus 31–45 minutes; p=0.02). Fewer physicians than nurse practitioners kept a record of ordered tests (6.3% versus 41.7%; p<0.001), and were able to detect when a test result had not been received (25.0% versus 39.6%; p=0.127).

Few of the nursing homes in which participants worked had laboratory HIT resources available (Figure 1). Nursing homes were more likely to have the capability to electronically view laboratory test results (35.4%) then have CPOE for generating new laboratory test orders (11.5%). A small fraction of nursing homes had CPOE with computerized decision support (6.3%).

Figure 1
Percent of survey respondents with laboratory health information technology in the nursing home.

Satisfaction with Current Laboratory Result Management

Overall, 43.8% of participants reported being somewhat or very satisfied with their laboratory test result management. Fewer physicians than nurse practitioners were satisfied with their current reporting system (37.5% versus 50.0%), but the difference was not significant (p=0.386). Satisfaction was significantly associated with having CPOE for chemistry/hematology orders (p<0.001), microbiology/pathology orders (p=0.002), or having a record of tests ordered (p=0.04). There was no significant association between satisfaction and having computerized decision support for CPOE (p=0.07), experiencing test result delays (p=0.184), or being able to electronically view chemistry/hematology (p=0.43) or microbiology/pathology results (p=0.36).

Desirability of Laboratory Health Information Technology

The three features of laboratory HIT systems rated most valuable were: showing abnormal results first, warning the appropriate clinician that a resident missed a requested test, and providing electronic acknowledgment of completed results (Figure 2). Participants also considered it useful for laboratory HIT systems to provide decision support to clinicians about abnormal test results and to provide various methods to facilitate communication of test results to residents, their family members, and other clinicians.

Figure 2
Desirability of features in an electronic laboratory results management system rated on 7 point scale from useless (0) to indispensable (7)

DISCUSSION

Our study demonstrates that the majority of physicians and nurse practitioners who work in nursing homes are generally not satisfied with the management of laboratory test result information, and experience delays in receiving laboratory test results. We also found that relatively few nursing homes have the laboratory HIT resources that are commonly found in ambulatory care and hospital settings (e.g., electronic viewing of laboratory test results or computerized order entry for ordering laboratory tests), even though these resources are considered desirable for laboratory test management.

The survey administered in nursing homes was a minimal modification of a survey that Poon et al,4 developed for use in hospital-based ambulatory care clinics. Similar to the original survey, our results showed that comparable proportions of healthcare professionals experienced delays in receiving laboratory test results and were not satisfied with the handling of these results. However, our survey found no association between delays and lack of satisfaction, whereas the original survey did find an association between these factors.4 This difference between survey results may be attributable to the fact that the original survey was administered to clinicians in a hospital-based ambulatory care clinic in which the use of laboratory HIT was uniform and thus accounted for less of the variance in satisfaction. Alternatively, it may be attributable to differences in the number of clinicians that keep records of tests ordered in hospitals compared to nursing homes.

In our survey, the nursing home characteristics that were most closely associated with satisfaction levels were having a record of ordered laboratory tests and having CPOE for laboratory tests. These characteristics are not independent factors because CPOE systems often allow clinicians to view a list of ordered laboratory tests. It is not surprising that HIT was associated with satisfaction because such systems have been shown to improve quality, efficiency, and cost in both ambulatory care clinics and hospitals.22 Specifically, laboratory HIT systems have been shown to significantly reduce order and result delivery times, medication error rates, and ADRs in hospital and ambulatory care settings.2325 We did not find HIT to be significantly associated with delays in our study, which may be partly due to not objectively measuring order delivery times, as assessed in other studies.

Our survey confirmed results of previous studies that the adoption of HIT in nursing homes lags behind other healthcare settings such as hospitals and ambulatory care.2628 A study by Subramanian et al,29 suggests that it may be more difficult to implement HIT systems in nursing homes, partly because the parties incurring the costs may not directly reap the benefits. Furthermore, recent research suggests that use of nursing home HIT systems is related to specific facility and staff-level characteristics, such as facility location, nursing home administrator tenure, and number of services offered.30

Study Strengths and Limitations

Our study had several strengths. First, this was a national survey of physicians and nurse practitioners recruited from major professional organizations, working in nursing homes. Unlike prior studies, we surveyed doctors and nurse practitioners, since both groups of healthcare professionals are directly involved in the process of ordering laboratory tests and reviewing subsequent results. In an attempt to improve the survey response rate and reduce the possibility of non-response bias, we employed multiple methods, including university sponsorship, monetary incentives, and sending email reminders to potential participants.31

The study also had several potential limitations. The response rate was low, despite the brevity of the survey and the provision of a nominal incentive. The low response rate may limit the generalizability of the findings due to non-response bias. Nevertheless, our response rate was similar to that of other investigators who conducted Internet-based surveys, which appear to generate a somewhat lower response rate then that associated with traditional mail surveys.32 Using a combination of both Internet and paper-based techniques may have contributed to a higher response rate.33 Response bias may have been introduced because this was an Internet-based survey, and individuals who responded are potentially more likely to be computer literate and predisposed to using laboratory HIT.

Implications for Future Work

Further research is needed in several areas. Additional research needs to be conducted to validate the findings of this study with a wider range of nursing home healthcare professionals, including geriatric psychiatrists, psychiatric/mental-health nurse practitioners, and physician assistants. Research is also needed to determine if designing and implementing laboratory HIT systems that include the functionality rated as most desirable by survey participants would: (1) decrease time spent managing normal and abnormal laboratory data; (2) decrease delays in laboratory test result reporting; (3) decrease ADRs related to laboratory monitoring errors; and, (4) improve clinician satisfaction with laboratory test management.

CONCLUSION

Delays in receiving laboratory test results and dissatisfaction with the management of laboratory test result information are commonly reported by physicians and nurse practitioners working in nursing homes. Implementation of certain laboratory health information technology, such as computerized order entry and tracking of laboratory test orders, could increase satisfaction and reduce delays in laboratory result management, thereby contributing to improved quality of care.

Acknowledgments

This study was supported in part by NIH grants F30MH074265, 1 KL2 RR024154-01 (NIH Roadmap Multidisciplinary Clinical Research Career Development Award Grant), P30AG024827, R01AG027017, F30MH074265, a Merck/AFAR Junior Investigator Award in Geriatric Clinical Pharmacology, and a Veteran’s Administration Health Services Research and Development Service Merit Award (IIR06-062). The authors acknowledge no conflicts of interest.

Appendix

Part I: Systems for ordering and reviewing laboratory results

An external file that holds a picture, illustration, etc.
Object name is nihms99413u1.jpg

An external file that holds a picture, illustration, etc.
Object name is nihms99413u2.jpg

Part II: Satisfaction with test result management and reporting

An external file that holds a picture, illustration, etc.
Object name is nihms99413u3.jpg

An external file that holds a picture, illustration, etc.
Object name is nihms99413u4.jpg

Part III: Respondent and Facility Demographic Information

An external file that holds a picture, illustration, etc.
Object name is nihms99413u5.jpg

An external file that holds a picture, illustration, etc.
Object name is nihms99413u6.jpg

Footnotes

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

References

1. Boohaker EA, Ward RE, Uman JE, McCarthy BD. Patient notification and follow-up of abnormal test results. A physician survey. Arch Intern Med. 1996 Feb 12;156(3):327–331. [PubMed]
2. Murff HJ, Gandhi TK, Karson AK, et al. Primary care physician attitudes concerning follow-up of abnormal test results and ambulatory decision support systems. Int J Med Inform. 2003 Sep;71(2–3):137–149. [PubMed]
3. Lin JJ, Dunn A, Moore C. Follow-up of outpatient test results: a survey of house-staff practices and perceptions. Am J Med Qual. 2006 May-Jun;21(3):178–184. [PubMed]
4. Poon EG, Gandhi TK, Sequist TD, Murff HJ, Karson AS, Bates DW. “I wish I had seen this test result earlier!”: Dissatisfaction with test result management systems in primary care. Arch Intern Med. 2004 Nov 8;164(20):2223–2228. [PubMed]
5. Plews-Ogan ML, Nadkarni MM, Forren S, et al. Patient safety in the ambulatory setting. A clinician-based approach. J Gen Intern Med. 2004 Jul;19(7):719–725. [PMC free article] [PubMed]
6. Karcz A, Holbrook J, Burke MC, et al. Massachusetts emergency medicine closed malpractice claims: 1988–1990. Ann Emerg Med. 1993 Mar;22(3):553–559. [PubMed]
7. Doshi JA, Shaffer T, Briesacher BA. National estimates of medication use in nursing homes: findings from the 1997 medicare current beneficiary survey and the 1996 medical expenditure survey. Journal of the American Geriatrics Society. 2005 Mar;53(3):438–443. [PubMed]
8. Field TS, Gurwitz JH, Avorn J, et al. Risk factors for adverse drug events among nursing home residents. Archives of Internal Medicine. 2001 Jul 9;161(13):1629–1634. [PubMed]
9. Edwards IR, Aronson JK. Adverse drug reactions: definitions, diagnosis, and management. Lancet. 2000 Oct 7;356(9237):1255–1259. [PubMed]
10. Handler SM, Wright RM, Ruby CM, Hanlon JT. Epidemiology of medication-related adverse events in nursing homes. American Journal Geriatric Pharmacotherapy. 2006 Sep;4(3):264–272. [PubMed]
11. Gurwitz JH, Field TS, Avorn J, et al. Incidence and preventability of adverse drug events in nursing homes. American Journal of Medicine. 2000;109(2):87–94. [PubMed]
12. Gurwitz JH, Field TS, Judge J, et al. The incidence of adverse drug events in two large academic long-term care facilities. American Journal of Medicine. 2005 Mar;118(3):251–258. [PubMed]
13. Levy C, Palat SI, Kramer AM. Physician practice patterns in nursing homes. J Am Med Dir Assoc. 2007 Nov;8(9):558–567. [PubMed]
14. Decker F, Gruhn P, Matthews-Martin L, Dollard K, Dollard, Tucker A, Bizette L. American Health Care Association. February 12, 2003. Results of the 2002 AHCA Survey of Nursing Staff Vacancy and Turnover in Nursing Homes.
15. Castle NG, Engberg J, Men A, Castle NG, Engberg J, Men A. Nursing home staff turnover: impact on nursing home compare quality measures. Gerontologist. 2007 Oct;47(5):650–661. [PubMed]
16. Levenson SA, Vance J. Assessing and collecting data on nursing facility patients: a guide for nurse on effective communication with physicians. Columbia, MD: American Medical Directors Association; 2004.
17. Cadogan MP, Franzi C, Osterweil D, Hill T. Barriers to effective communication in skilled nursing facilities: differences in perception between nurses and physicians. Journal of the American Geriatrics Society. 1999 Jan;47(1):71–75. [PubMed]
18. Hastings SN, Whitson HE, White HK, et al. After-hours calls from long-term care facilities in a geriatric medicine training program. J Am Geriatr Soc. 2007 Dec;55(12):1989–1994. [PubMed]
19. Whitson HE, Hastings SN, McConnell ES, Lekan-Rutledge DA. Inter-disciplinary focus groups on telephone medicine: a quality improvement initiative. Journal of the American Medical Directors Association. 2006 Sep;7(7):407–411. [PubMed]
20. Keren R, Muret-Wagstaff S, Goldmann DA, Mandl KD. Notifying emergency department patients of negative test results: pitfalls of passive communication. Pediatr Emerg Care. 2003 Aug;19(4):226–230. [PubMed]
21. Rosner B. Fundamentals of Biostatistics. 6. Boston, Massachusetts: Duxbury Press; 2005.
22. Chaudhry B, Wang J, Wu S, et al. Systematic review: impact of health information technology on quality, efficiency, and costs of medical care. Annals of Internal Medicine. 2006 May 16;144(10):742–752. [PubMed]
23. Bates DW, Leape LL, Cullen DJ, et al. Effect of computerized physician order entry and a team intervention on prevention of serious medication errors. Jama. 1998 Oct 21;280(15):1311–1316. [PubMed]
24. Westbrook JI, Georgiou A, Dimos A, Germanos T. Computerised pathology test order entry reduces laboratory turnaround times and influences tests ordered by hospital clinicians: a controlled before and after study. J Clin Pathol. 2006 May;59(5):533–536. [PMC free article] [PubMed]
25. Raebel MA, Chester EA, Newsom EE, et al. Randomized trial to improve laboratory safety monitoring of ongoing drug therapy in ambulatory patients. Pharmacotherapy. 2006 May;26(5):619–626. [PubMed]
26. Kaushal R, Blumenthal D, Poon EG, et al. The costs of a national health information network. Ann Intern Med. 2005 Aug 2;143(3):165–173. [PubMed]
27. Poon EG, Jha AK, Christino M, et al. Assessing the level of healthcare information technology adoption in the United States: a snapshot. BMC Med Inform Decis Mak. 2006;6:1. [PMC free article] [PubMed]
28. DesRoches CM, Campbell EG, Rao SR, et al. Electronic health records in ambulatory care--a national survey of physicians. New England Journal of Medicine. 2008 Jul 3;359(1):50–60. [PubMed]
29. Subramanian S, Hoover S, Gilman B, Field TS, Mutter R, Gurwitz JH. Computerized physician order entry with clinical decision support in long-term care facilities: costs and benefits to stakeholders. J Am Geriatr Soc. 2007 Sep;55(9):1451–1457. [PubMed]
30. Chan S. Factors associated with the use of electronic information systems for drug dispensing and medication administration records in nursing homes. Journal of the American Medical Directors Association. 2008;9(6):414–421. [PubMed]
31. Edwards P, Roberts I, Clarke M, et al. Increasing response rates to postal questionnaires: systematic review. BMJ. 2002 May 18;324(7347):1183. [PMC free article] [PubMed]
32. Aitken C, Power R, Dwyer R. A very low response rate in an on-line survey of medical practitioners. Australian & New Zealand Journal of Public Health. 2008 Jun;32(3):288–289. [PubMed]
33. Beebe TJ, Locke GR, 3rd, Barnes SA, Davern ME, Anderson KJ. Mixing web and mail methods in a survey of physicians. Health Services Research. 2007 Jun;42(3 Pt 1):1219–1234. [PMC free article] [PubMed]