Four hundred eighty physicians were recruited to participate, representing 380 physician offices. Of the 463 recruited physicians, 445 were successfully contacted by email using the addresses they had furnished at the time of recruitment. Of these 445 recruited physicians, 210 (47.2%) physicians from 190 offices logged on to at least one of the educational modules. Of the 210 physicians who logged on at least once during the 45-week study period, one hundred twenty-four (59%) returned again to log-on for Module 2, eighty-seven (41%) logged on for Module 3, and forty-four (21%) logged on to Module 4.
Two hundred and ten physicians of 445 logged on at least once to the website, leaving 235 physicians as non-participants. Figure represents total physician log-on by week. Analysis of log-on days indicated that participants were most likely to log-on on Monday or Thursday (see Figure ). Log-on times were also examined and findings indicate that physicians were most likely to log-on to a module between the hours of 3 P.M. and 7 P.M (15:00–19:00). Other common times for participants to log-on included times earlier in the day, or between the hours of 8 P.M. and midnight (20:00–24:00) (see Figure ).
Cumulative internet engagement by physician over time
Number of logins by day of the week.
Number of logins by time of day.
The participant characteristics (n = 210) were compared with those of recruited but non-participating physicians (n = 235) (Table ). Age was not significant, however there were significant differences by gender, degree, and country of medical training. By race, 81.5% of the participants were Caucasian, 8.8% were Asian, 3.0% were African American, 3.5% were Hispanic and 3.0% were listed as other with no significant differences. The largest percentage of recruited physicians was family practitioners (41.3%), followed by general internists (29.4%), pediatricians (9.6%), and general practitioners (1.5%), with no significant differences. Female physicians were significantly more likely to participate than males (p = . 0001), Medical Doctors (MDs) were significantly more likely than Doctors of Osteopathy (DOs) to participate (p = . 01), and graduates of U.S. medical schools significantly more likely than graduates of international medical schools to participate (p = . 01). In addition to demographic characteristics, chlamydia screening rates of participant and non-participant physicians were compared. Baseline screening rates of nonparticipants were significantly lower than those of participants. Non-participant chlamydia screening rates were 14.6% at baseline, compared to 17.4% for participants (p < .006).
Participant* versus recruited** but non-participant characteristics
From follow-up evaluation question data, 100% of responding DOs felt that the course email reminders were effective in reminding them about educational modules compared to 92.6% of MDs; 95.9% of US medical graduates rated the reminders as useful, while only 77.8% of graduates from non-US medical schools found them useful. Concerning preferences for delivery of CME, no DOs reported web-based activities as their preferred method for lifelong learning; they preferred local (50.0%) and national meetings (50.0%). MDs were more likely to prefer web-based CME (37.0%) to local (35.1%) or national (16.6%) meetings. Female physicians reported a preference for web-based CME (50.0%), whereas male physicians preferred local (35.0%) and national (27.5%) meetings over web-based CME (25.0%).
The results of this study indicate that email reminders may be useful in engaging nearly half of a group of practicing primary care physicians recruited to participate in an online women's health educational series. This data is consistent with McMahon et al.'s findings in comparing the use of email, fax and mail, finding that email reminders were more useful to increase response rates [12
]. The study participation rate, (47.2%) is also consistent with the work of Flanagan et. al.'s study of participation in web decision support tools for the management of pneumonia [6
]. However, the gap of up to 3 months between recruitment and the initiation of the online educational activities may have contributed to a lower participation rate. It is possible that by decreasing the gap in time between recruitment and announcement of the availability of the online educational activity, participation rates could be increased. Future study designs using email reminders should consider beginning the intervention immediately following agreement to participate or shortly thereafter. Two current studies using email reminders to promote educational courses in the prevention of glucocorticoid-induced osteoporosis and in the secondary prevention of cardiovascular disease in patients following a myocardial infarction have been designed to deliver the intervention immediately following agreement to participate [13
Findings from follow-up evaluation question data indicate that CME providers interested in targeting specific groups of physicians may benefit from using alternative methods of CME recruitment and delivery. CME providers targeting DOs may want to explore ways to engage DOs in web-based learning activities or consider focusing activities that target DOs to local or national meetings. Providers of CME who are interested in engaging male physicians and graduates of U.S. medical schools may find email reminders useful, but they may also want to explore additional methods of recruitment.
Persistent email reminders did increase physicians' response rates to online education, but response rate decreased with the number of reminders. The first three reminders produced the largest responses, with decline after the 10th reminder. Based on our experience, it would seem reasonable that providers and researchers with limited resources consider focusing their announcements/reminders on the first 3–10 encounters. Data from time of log-on underscores the advantages of asynchronous online interventions for busy clinicians. Traditional "live" online symposia scheduled for the middle of the day might appeal to the physicians who logged on between 10 AM and 2 PM (10:00–14:00), but data from this intervention suggest that many physicians have more available time later in the day for online educational activity. The investigators findings of Monday being a frequent day for log-on was unexpected, but may offer an opportunity for future study designs to include weekend email broadcasting rather than a Thursday broadcast.
The topic of the online educational activity may influence response to email announcements and reminders. While baseline Chlamydia screening rates were relatively low in both groups, the significant difference between the groups may indicate that those who are likely to perform better according to clinical practice guidelines are more likely to participate in online educational strategies that reinforce their use. Or the higher screening rates may be associated with a higher degree of interest in the overall topic area of women's health.
Also related to the topic addressed in this online study, previous studies of preventive practices of female physicians have indicated they are more likely than males to promote preventive practices [14
] and screening [15
] including Chlamydia screening among their female patients than are male physicians. The advertised educational topic for this study was an online women's health course. More female physicians than male physicians responded to the email course reminders, but the topic may have had more appeal to female physicians than to male physicians, leaving the issue of whether there are gender differences among physicians in response to email reminders unresolved.
Comparisons of characteristics of participant physicians and non-participant recruited physicians may be useful to those designing online recruitment and engagement strategies for future studies. Those using email reminders to communicate with physician populations including large numbers of DOs, however, may benefit from considering blended methods of CME recruitment and delivery. Using various methods of reaching providers, may also enhance DO participation. CME providers targeting DOs may want to explore additional ways to engage DOs in web-based learning activities or consider focusing activities that target DOs at local or national meetings.