Background

Online medical education curricula offer new tools to teach and evaluate learners. The effect on educational outcomes of using learner feedback to guide curricular revision for online learning is unknown.

Methods

In this study, qualitative analysis of learner feedback gathered from an online curriculum was used to identify themes of learner feedback, and changes to the online curriculum in response to this feedback were tracked. Learner satisfaction and knowledge gains were then compared from before and after implementation of learner feedback.

Results

37,755 learners from 122 internal medicine residency training programs were studied, including 9437 postgraduate year (PGY)1 residents (24.4 % of learners), 9864 PGY2 residents (25.5 %), 9653 PGY3 residents (25.0 %), and 6605 attending physicians (17.0 %). Qualitative analysis of learner feedback on how to improve the curriculum showed that learners commented most on the overall quality of the educational content, followed by specific comments on the content. When learner feedback was incorporated into curricular revision, learner satisfaction with the instructive value of the curriculum (1 = not instructive; 5 = highly instructive) increased from 3.8 to 4.1 (p < 0.001), and knowledge gains (i.e., post test scores minus pretest scores) increased from 17.0 % to 20.2 % (p < 0.001).

Conclusions

Learners give more feedback on the factual content of a curriculum than on other areas such as interactivity or website design. Incorporating learner feedback into curricular revision was associated with improved educational outcomes. Online curricula should be designed to include a mechanism for learner feedback and that feedback should be used for future curricular revision.

Background

Distance learning may be useful for building health research capacity. However, evidence that it can improve knowledge and skills in health research, particularly in resource-poor settings, is limited. We compared the impact and acceptability of teaching two distinct content areas, Biostatistics and Research Ethics, through either on-line distance learning format or traditional on-site training, in a randomized study in India. Our objective was to determine whether on-line courses in Biostatistics and Research Ethics could achieve similar improvements in knowledge, as traditional on-site, classroom-based courses.

Methods

Subjects: Volunteer Indian scientists were randomly assigned to one of two arms.

Intervention: Students in Arm 1 attended a 3.5-day on-site course in Biostatistics and completed a 3.5-week on-line course in Research Ethics. Students in Arm 2 attended a 3.5-week on-line course in Biostatistics and 3.5-day on-site course in Research Ethics. For the two course formats, learning objectives, course contents and knowledge tests were identical.

Main Outcome Measures: Improvement in knowledge immediately and 3-months after course completion, compared to baseline.

Results

Baseline characteristics were similar in both arms (n = 29 each). Median knowledge score for Biostatistics increased from a baseline of 49% to 64% (p < 0.001) 3 months after the on-site course, and from 48% to 63% (p = 0.009) after the on-line course. For the on-site Research Ethics course, median score increased from 69% to 83% (p = 0.005), and for the on-line Research Ethics course from 62% to 80% (p < 0.001). Three months after the course, median gains in knowledge scores remained similar for the on-site and on-line platforms for both Biostatistics (16% vs. 12%; p = 0.59) and Research Ethics (17% vs. 13%; p = 0.14).

Conclusion

On-line and on-site training formats led to marked and similar improvements of knowledge in Biostatistics and Research Ethics. This, combined with logistical and cost advantages of on-line training, may make on-line courses particularly useful for expanding health research capacity in resource-limited settings.

Background

Several residency program characteristics have been suggested as measures of program quality, but associations between these measures are unknown. We set out to determine associations between these potential measures of program quality.

Methods

Survey of internal medicine residency programs that shared an online ambulatory curriculum on hospital type, faculty size, number of trainees, proportion of international medical graduate (IMG) trainees, Internal Medicine In-Training Examination (IM-ITE) scores, three-year American Board of Internal Medicine Certifying Examination (ABIM-CE) first-try pass rates, Residency Review Committee-Internal Medicine (RRC-IM) certification length, program director clinical duties, and use of pharmaceutical funding to support education. Associations assessed using Chi-square, Spearman rank correlation, univariate and multivariable linear regression.

Results

Fifty one of 67 programs responded (response rate 76.1%), including 29 (56.9%) community teaching and 17 (33.3%) university hospitals, with a mean of 68 trainees and 101 faculty. Forty four percent of trainees were IMGs. The average post-graduate year (PGY)-2 IM-ITE raw score was 63.1, which was 66.8 for PGY3s. Average 3-year ABIM-CE pass rate was 95.8%; average RRC-IM certification was 4.3 years. ABIM-CE results, IM-ITE results, and length of RRC-IM certification were strongly associated with each other (p < 0.05). PGY3 IM-ITE scores were higher in programs with more IMGs and in programs that accepted pharmaceutical support (p < 0.05). RRC-IM certification was shorter in programs with higher numbers of IMGs. In multivariable analysis, a higher proportion of IMGs was associated with 1.17 years shorter RRC accreditation.

Conclusions

Associations between quality indicators are complex, but suggest that the presence of IMGs is associated with better performance on standardized tests but decreased duration of RRC-IM certification.

Background

The Internet provides a means of disseminating medical education curricula, allowing institutions to share educational resources. Much of what is published online is poorly planned, does not meet learners' needs, or is out of date.

Discussion

Applying principles of curriculum development, adult learning theory and educational website design may result in improved online educational resources. Key steps in developing and implementing an education website include: 1) Follow established principles of curriculum development; 2) Perform a needs assessment and repeat the needs assessment regularly after curriculum implementation; 3) Include in the needs assessment targeted learners, educators, institutions, and society; 4) Use principles of adult learning and behavioral theory when developing content and website function; 5) Design the website and curriculum to demonstrate educational effectiveness at an individual and programmatic level; 6) Include a mechanism for sustaining website operations and updating content over a long period of time.

Summary

Interactive, online education programs are effective for medical training, but require planning, implementation, and maintenance that follow established principles of curriculum development, adult learning, and behavioral theory.