PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of springeropenLink to Publisher's site
Advances in Health Sciences Education
 
Adv Health Sci Educ Theory Pract. 2010 August; 15(3): 349–356.
Published online 2009 September 23. doi:  10.1007/s10459-009-9201-3
PMCID: PMC2940026

Significant increase in factual knowledge with web-assisted problem-based learning as part of an undergraduate cardio-respiratory curriculum

Abstract

In recent years, increasing attention has been paid to web-based learning although the advantages of computer-aided instruction over traditional teaching formats still need to be confirmed. This study examined whether participation in an online module on the differential diagnosis of dyspnoea impacts on student performance in a multiple choice examination of factual knowledge in cardiology and pneumology. A virtual problem-based learning environment for medical students supervised by postgraduate teachers was created. Seventy-four out of 183 fourth-year medical students volunteered to use the online module while attending a 6-week cardio-respiratory curriculum in summer 2007. Of these, 40 were randomly selected to be included (intervention group); the remaining 34 served as an internal control group. Analysis of all written exams taken during the preceding term showed that both groups were comparable (86.4 ± 1.1 vs. 85.9 ± 1.1%; p = 0.751). Students in the intervention group scored significantly higher in the final course assessment than students allocated to the control group (84.8 ± 1.3 vs. 79.5 ± 1.4%; p = 0.006; effect size 0.67). Thus, additional problem-based learning with an online module as part of an undergraduate cardio-respiratory curriculum lead to higher students’ scores in an exam testing factual knowledge. Whether using this teaching format increases overall student motivation to engage in the learning process needs to be further investigated.

Keywords: Factual knowledge, Motivation, Multiple choice questions, Problem-based learning, Web-based learning

Introduction

The use of computers was introduced in medical education more than 40 years ago (Lipkin et al. 1961). Although computer-assisted instruction (CAI) would naturally lend itself to a more complex presentation of content matter (Tegtmeyer et al. 2001), early attempts mainly aimed at a linear transmission of knowledge. Since then, the developmental focus has moved from mere technical aspects to the design of software solutions suited for specific teaching objectives (Valcke and De Wever 2006). In recent years, increasing attention has been paid to web-based learning (WBL) due to its flexible scheduling, economies of scale, ease in updating, and the potential for the delivery of individualized teaching using novel instructional methods (Cook 2006, 2007).

The combination of WBL with a problem-based teaching format results in a virtual learning environment which can be expected to foster the structured acquisition of both knowledge and skills related to clinical reasoning (Hammoud et al. 2006; Kamin et al. 2002). Although WBL cannot replace bed-side teaching, the case studies presented in a WBL module should resemble real clinical settings as far as possible (Glick and Moore 2001). A small number of studies suggest that collaborative learning via the internet is feasible and might yield advantages depending on the circumstances under which it is implemented (De Wever et al. 2008; Kamin et al. 2002; Nathoo et al. 2005; Stromso et al. 2004; Taradi et al. 2005; Wehrs et al. 2007). Regarding the acquisition of factual knowledge, previous studies did not detect an advantage of CAI over traditional teaching formats (Schwartz and Griffin 1993). However, the impact of participating in a problem-based WBL module on students’ gain of factual knowledge has not been studied in great detail.

This article describes students’ enrolment in and satisfaction with a newly set-up online module on the differential diagnosis of chronic shortness of breath. In addition, the students’ achievements in a multiple choice examination of factual knowledge were assessed with regard to their interest and participation in this online module.

Methods

Description of the online module

In summer 2007, 183 fourth-year medical students were given the opportunity to sign up for using and evaluating a collaborative online module while attending the six-week cardio-respiratory curriculum (blended learning) at Göttingen University Hospital (Germany). A web-based learning management system (Clix®, Information Multimedia Communication AG, Saarbrücken, Germany) was used in order to facilitate asynchronous group discussions and the exchange of documents as suggested by previous studies (Spinello and Fischbach 2004). The module was created according to criteria outlined by Hilty et al. (2006) and Greenhalgh (2001). Postgraduate teachers who were specifically trained to assist the students using the module served as online tutors. Course content and the didactic concept underlying the online module were developed by the first author of this manuscript and subsequently discussed with respiratory specialists and faculty staff as recommended (Hudson 2004); During the online module, students were asked to diagnose a 64-year old man who presented to his general practitioner with chronic shortness of breath. The differential diagnosis of shortness of breath is one of the learning objectives to be covered in the six-week cardio-respiratory curriculum. In order to find the correct diagnosis, students needed to have diagnostic procedures carried out on their virtual patient, the results of which were provided by their online tutors. In this regard, the online module did not represent true “problem-based learning” as defined by Barrows and Mitchell (1975). However, the module did comprise distinct steps towards the diagnosis:

  • Step (1) Information on the presenting complaint and past medical history of the patient was provided along with a table indicating costs for 46 standard diagnostic procedures in internal medicine. Students were asked to discuss the value of these procedures in the management of “their” patient and to choose three initial tests they thought most helpful for finding the correct diagnosis. Students were encouraged to retrieve information on standard diagnostic procedures from books and lectures embedded in the six-week cardio-respiratory curriculum.
  • Step (2) Test results were uploaded by the online tutor. Students were now given the opportunity to order as many further test results (just one at a time) as they needed in order to reach a final diagnosis.
  • Step (3) As soon as students indicated they were ready to make a diagnosis, they were asked to submit the relevant code according to the International Classification of Diseases (version 10) which was available on the platform.
  • Step (4) Students received feedback on their “guess” and were given the correct diagnosis. In order to complete the module, they were asked to search the “Guidelines on diagnosis and treatment of pulmonary arterial hypertension” issued by the European Society of Cardiology in 2004 (Galie et al. 2004) and upload it on the online platform.

We expected students to spend on average two hours per week on the online module.

Student enrolment, data acquisition and analysis

In March 2007, an invitation to sign up for the supplemental online module was mailed to all 183 students. Due to a limitation of resources, participation in the module could only be offered to 40 students. Students who responded to the invitation were randomly assigned to either the intervention group (participation in the online module) or the control group (interest in the online module but no participation possible).

Students’ performance levels before entering the study were determined on the basis of scores achieved in written multiple choice exams taken in their previous semester (winter term 2006/07). For each student, an individual percent score (total number of correct answers/total number of questions asked in all written exams × 100) was computed. On the last day of the six-week cardio-respiratory curriculum, all 183 students took a summative examination containing 68 multiple-choice questions assessing factual knowledge. These questions covered the whole range of cardiology and pneumology while the online module was focused on the differential diagnosis of chronic shortness of breath. Student achievements in the final multiple choice examination were compared between students in the intervention and the control group.

Students who had been assigned a place in the online module were asked to complete online questionnaires (EvaSys®, ElectricPaper, Lüneburg, Germany) on subject characteristics and attitudes towards computer-based learning in general and the new online module in particular. Personal agreement with specific statements was assessed using six-point Likert scales anchored by 1 (“fully agree”) and 6 (“completely disagree”) on the first and the last day of the six-week curriculum.

Data are presented as median ± SE of the mean. Statistical analyses were carried out using SPSS 12.0.1 (SPSS Inc., Chicago, Illinois, USA); p values (significance level 5%) were derived from χ2 tests and T-tests. Effect size was calculated according to Cohen (1992).

Results

Enrolment phase

Of 183 students entering their fourth year in summer 2007, 74 volunteered to participate in the supplemental online module. As is displayed in Table 1, these students had achieved higher scores in previous exams than those who did not sign up for the online module (86.2 ± 0.8 vs. 84.1 ± 0.6%; p = 0.035). However, there was no significant difference in previous exam scores between the 40 students randomly allocated to the intervention group and the 34 students who were assigned to participate as the internal control group (p = 0.751). Among those interested in the online module, the proportion of female students (66.2%) was significantly greater than in those who did not sign up for participation (51.4%; p = 0.046).

Table 1
Examination scores

Achievements in end-of-course examinations

Students who had signed up for the online module continued to score higher in the final course examination (82.4 ± 1.0%) than those not interested in using the online platform (78.6 ± 1.0%; p = 0.009). Although there had been no difference in summative exam scores between the intervention and the control group before entering the fourth year, students in the intervention group achieved significantly higher scores in the final multiple choice (84.8 ± 1.3%) test than students in the control group (79.5 ± 1.4%; p = 0.006, effect size 0.67). In fact, exam scores of students in the control group were no different from those of students who had not signed up for the online module at all (p = 0.636).

Student satisfaction, perceived learning outcome and evaluation of the online module

Before starting the online module, participants indicated that they were interested in CAI (2.11 ± 0.14 on a six-point Likert scale) and that they felt well-trained to use computers and the internet (2.24 ± 0.19). Perceived knowledge on the differential diagnosis of dyspnea (3.50 ± 0.12) and the components of a lung function test (3.66 ± 0.16) was low, and students did not feel competent to devise a diagnostic management plan for a patient complaining of dyspnoea (4.61 ± 0.09). In addition, they did not know the costs of standard diagnostic procedures in internal medicine (4.24 ± 0.15; Table 2).

Table 2
Students’ perceptions of knowledge and skills

The second survey carried out at the end of the 6-week course yielded favourable ratings regarding technical aspects of the online module (average of 2 on a six-point Likert scale). On a six-point scale with 1 being the best possible mark, students’ overall rating of the online module was 2.2 ± 0.13. Students had spent an average of 2.7 ± 0.3 h per week using the learning platform. All participants stated that their knowledge on cardio-respiratory diseases had increased during the 6 weeks during which the module was online (Table 2).

Discussion

Although the acquisition of higher-order cognitive skills such as clinical reasoning can be viewed as the ultimate goal of medical education, most examinations at medical school as well as many high stakes exit examinations (Fischer et al. 2005) still contain a considerable amount of multiple choice questions assessing factual knowledge (Palmer and Devitt 2007). While problem-based learning is primarily effective in promoting skills related to clinical reasoning (Norman and Schmidt 1992), it is also capable of imparting factual knowledge as has been shown in large cohort studies (Van der Vleuten et al. 1996). The present study investigated whether this effect of the teaching format is preserved when problem-based learning is transferred to a virtual environment. The two main findings were that students on a higher achievement level were more willing to explore innovative teaching formats and that students randomly allocated to using the online module scored significantly higher in a final multiple choice test than those who signed up but could not be allocated due to a lack of resources. Importantly, there was no difference in achievement level between the two groups before entering the study.

A recent meta-analysis reviewing a total of 201 published articles on the effectiveness of internet-based learning for students, postgraduate trainees and practitioners in a profession directly related to human or animal health concluded that, when compared to no intervention, internet-based interventions have large and significant effects on the acquisition of knowledge, skills and behaviours (Cook et al. 2008). The effect size of 0.67 between achievements in the intervention and the control group found in our study population is well within the range of studies included in this systematic review. The primary outcome of most of these studies was matched to the original learning objective underlying the teaching intervention (Kern et al. 1998); for examples see (Pederson et al. 2006; Sisson et al. 2006; Walsh et al. 2007); thus the finding of a significant effect of internet-based learning over no intervention is not surprising. However, in our study we found a significant effect in knowledge increase even though final exam questions were not closely matched to the content of the online module, i.e., the module did not specifically prepare students for the final exam. Instead, it might have increased students’ motivation to learn about cardiology and pneumology in general since shortness of breath is a common presenting complaint in both disciplines.

Interestingly, motivation to use innovative teaching and learning formats has been identified as an important confounder in non-randomized trials (Hallgren et al. 2002). All 74 students signing up for the online module expressed their motivation by responding to the mailed invitation to participate. Thus it can be assumed that ‘baseline motivation’ was comparable in the two groups. Although motivation was not formally assessed, it can be hypothesized that participation in the online module further increased student motivation to learn. This notion is corroborated by the fact that students in the intervention group spent an extra amount of time (almost 3 h per week) on the online module. This increase in learning time is within the range of findings from comparable WBL projects (Nathoo et al. 2005). The impact of web-assisted problem-based learning on overall learning motivation has not been studied in great detail and should be addressed in future trials as this aspect was not covered in the systematic review mentioned above (Cook et al. 2008).

Students’ self-assessment of knowledge and skills related to the management of dyspnea increased significantly over time; due to the lack of a completely randomized design with mutual exclusion of either web-based or contact teaching, the contribution of the online module to this increase cannot be measured. We cannot exclude an improvement in students’ rating on their self assessment for knowledge and diagnostic skills in the control group similar to the one observed in the intervention group since we only obtained self-assessments from members of the latter. In any case, the true impact of WBL on any of the learning objectives pursued by this teaching format (e.g., factual knowledge gain, clinical reasoning) can only be determined in prospective randomized trials (Letterie 2003; Raupach et al. 2009). However, media-comparative research is being heavily criticized due to methodological limitations (Cook 2005) one of which has once again become apparent in this study: It might well be that only motivated students will benefit from using WBL modules. Thus, medical educators and students could face an ever-increasing demand for individualization regarding the teaching formats used to arrive at the desired outcome.

During the enrolment phase, female students showed greater interest in using the web-based learning module. This observation is surprising in view of the fact that women have previously been reported to be more hesitant regarding the use of computers (Broos 2005). However, this aspect needs to be addressed in future research since the present study was not designed to specifically assess gender aspects.

Conclusion

This study examined the effect of virtual collaborative learning in respiratory medicine on student achievements in a multiple choice test covering the whole range of cardiology and pneumology. Students randomly assigned to using the problem-based online module on the differential diagnosis of chronic shortness of breath scored higher in a test of factual knowledge than students not included in the study despite comparable achievement levels before entering the study. The online module is likely to have increased students’ motivation to learn, and subsequent learning was not restricted to the content of the online module. However, it remains to be elucidated whether all students would benefit from using innovative teaching and learning methods regardless of prior motivation. In addition, future research will have to further assess gender aspects related to computer-assisted learning.

Acknowledgments

We would like to thank all tutors and students who devoted their time and effort to this study.

Open Access This article is distributed under the terms of the Creative Commons Attribution Noncommercial License which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited.

References

  • Barrows HS, Mitchell DL. An innovative course in undergraduate neuroscience. Experiment in problem-based learning with ‘problem boxes’ British Journal of Medical Education. 1975;9:223–230. [PubMed]
  • Broos A. Gender and information and communication technologies (ICT) anxiety: Male self-assurance and female hesitation. CyberPsychology & Behavior. 2005;8:21–31. doi: 10.1089/cpb.2005.8.21. [PubMed] [Cross Ref]
  • Cohen J. A power primer. Psychological Bulletin. 1992;112:155–159. doi: 10.1037/0033-2909.112.1.155. [PubMed] [Cross Ref]
  • Cook DA. The research we still are not doing: An agenda for the study of computer-based learning. Academic Medicine. 2005;80:541–548. doi: 10.1097/00001888-200506000-00005. [PubMed] [Cross Ref]
  • Cook DA. Where are we with web-based learning in medical education? Medical Teacher. 2006;28:594–598. doi: 10.1080/01421590601028854. [PubMed] [Cross Ref]
  • Cook DA. Web-based learning: Pros, cons and controversies. Clinical Medicine. 2007;7:37–42. [PubMed]
  • Cook DA, Levinson AJ, Garside S, Dupras DM, Erwin PJ, Montori VM. Internet-based learning in the health professions: A meta-analysis. JAMA. 2008;300:1181–1196. doi: 10.1001/jama.300.10.1181. [PubMed] [Cross Ref]
  • De Wever, B., Van Winckel, M., & Valcke, M. (2008). Discussing patient management online: The impact of roles on knowledge construction for students interning at the paediatric ward. Advances in Health Sciences Education Theory and Practice, 13, 25–42. [PubMed]
  • Fischer MR, Herrmann S, Kopp V. Answering multiple-choice questions in high-stakes medical examinations. Medical Education. 2005;39:890–894. doi: 10.1111/j.1365-2929.2005.02243.x. [PubMed] [Cross Ref]
  • Galie, N., Torbicki, A., Barst, R., Dartevelle, P., Haworth, S., Higenbottam, T., et al. (2004). Guidelines on diagnosis and treatment of pulmonary arterial hypertension. The task force on diagnosis and treatment of pulmonary arterial hypertension of the European Society of Cardiology. European Heart Journal, 25, 2243–2278 [PubMed]
  • Glick TH, Moore GT. Time to learn: The outlook for renewal of patient-centred education in the digital age. Medical Education. 2001;35:505–509. doi: 10.1046/j.1365-2923.2001.00935.x. [PubMed] [Cross Ref]
  • Greenhalgh T. Computer assisted learning in undergraduate medical education. BMJ. 2001;322:40–44. doi: 10.1136/bmj.322.7277.40. [PMC free article] [PubMed] [Cross Ref]
  • Hallgren RC, Parkhurst PE, Monson CL, Crewe NM. An interactive, web-based tool for learning anatomic landmarks. Academic Medicine. 2002;77:263–265. doi: 10.1097/00001888-200203000-00018. [PubMed] [Cross Ref]
  • Hammoud M, Gruppen L, Erickson SS, Cox SM, Espey E, Goepfert A, et al. To the point: Reviews in medical education online computer assisted instruction materials. American Journal of Obstetrics and Gynecology. 2006;194:1064–1069. doi: 10.1016/j.ajog.2005.08.023. [PubMed] [Cross Ref]
  • Hilty DM, Hales DJ, Briscoe G, Benjamin S, Boland RJ, Luo JS, et al. APA summit on medical student education task force on informatics and technology: Learning about computers and applying computer technology to education and practice. Academic Psychiatry. 2006;30:29–35. doi: 10.1176/appi.ap.30.1.29. [PubMed] [Cross Ref]
  • Hudson JN. Computer-aided learning in the real world of medical education: Does the quality of interaction with the computer affect student learning? Medical Education. 2004;38:887–895. doi: 10.1111/j.1365-2929.2004.01892.x. [PubMed] [Cross Ref]
  • Kamin C, Deterding R, Lowry M. Student’s perceptions of a virtual PBL experience. Academic Medicine. 2002;77:1161–1162. doi: 10.1097/00001888-200211000-00028. [PubMed] [Cross Ref]
  • Kern DE, Thomas PA, Howard DM, Bass EB. Curriculum development for medical education—a six-step approach. Baltimore and London: The John Hopkins University Press; 1998.
  • Letterie GS. Medical education as a science: The quality of evidence for computer-assisted instruction. American Journal of Obstetrics and Gynecology. 2003;188:849–853. doi: 10.1067/mob.2003.168. [PubMed] [Cross Ref]
  • Lipkin M, Engle RL, Jr, Davis BJ, Zworykin VK, Ebald R, Sendrow M, et al. Digital computer as aid to differential diagnosis. Use in hematologic diseases. Archives of Internal Medicine. 1961;108:56–72. [PubMed]
  • Nathoo AN, Goldhoff P, Quattrochi JJ. Evaluation of an interactive case-based online network (ICON) in a problem based learning environment. Advances in Health Sciences Education Theory and Practice. 2005;10:215–230. doi: 10.1007/s10459-005-7851-3. [PubMed] [Cross Ref]
  • Norman GR, Schmidt HG. The psychological basis of problem-based learning: A review of the evidence. Academic Medicine. 1992;67:557–565. doi: 10.1097/00001888-199209000-00002. [PubMed] [Cross Ref]
  • Palmer EJ, Devitt PG. Assessment of higher order cognitive skills in undergraduate education: Modified essay or multiple choice questions? Research paper. BMC Medical Education. 2007;7:49. doi: 10.1186/1472-6920-7-49. [PMC free article] [PubMed] [Cross Ref]
  • Pederson LL, Blumenthal DS, Dever A, McGrady G. A web-based smoking cessation and prevention curriculum for medical students: Why, how, what, and what next. Drug and Alcohol Review. 2006;25:39–47. doi: 10.1080/09595230500459503. [PubMed] [Cross Ref]
  • Raupach, T., Münscher, C., Anders, S., Steinbach, R., Pukrop, T., Hege, I., et al. (2009). Web-based collaborative training of clinical reasoning: A randomized trial. Medical Teacher, 31, e431–e437 [PubMed]
  • Schwartz S, Griffin T. Comparing different types of performance feedback and computer-based instruction in teaching medical students how to diagnose acute abdominal pain. Academic Medicine. 1993;68:862–864. doi: 10.1097/00001888-199311000-00017. [PubMed] [Cross Ref]
  • Sisson SD, Rastegar D, Rice TN, Prokopowicz G, Hughes MT. Physician familiarity with diagnosis and management of hypertension according to JNC 7 guidelines. Journal of Clinical Hypertension (Greenwich) 2006;8:344–350. doi: 10.1111/j.1524-6175.2006.05335.x. [PubMed] [Cross Ref]
  • Spinello E, Fischbach R. Problem-based learning in public health instruction: A pilot study of an online simulation as a problem-based learning approach. Education for Health (Abingdon) 2004;17:365–373. doi: 10.1080/13576280400002783. [PubMed] [Cross Ref]
  • Stromso HI, Grottum P, Hofgaard Lycke K. Changes in student approaches to learning with the introduction of computer-supported problem-based learning. Medical Education. 2004;38:390–398. doi: 10.1046/j.1365-2923.2004.01786.x. [PubMed] [Cross Ref]
  • Taradi SK, Taradi M, Radic K, Pokrajac N. Blending problem-based learning with web technology positively impacts student learning outcomes in acid–base physiology. Advances in Physiology Education. 2005;29:35–39. doi: 10.1152/advan.00026.2004. [PubMed] [Cross Ref]
  • Tegtmeyer K, Ibsen L, Goldstein B. Computer-assisted learning in critical care: From ENIAC to HAL. Critical Care Medicine. 2001;29:N177–N182. doi: 10.1097/00003246-200108001-00006. [PubMed] [Cross Ref]
  • Valcke M, Wever B. Information and communication technologies in higher education: Evidence-based practices in medical education. Medical Teacher. 2006;28:40–48. doi: 10.1080/01421590500441927. [PubMed] [Cross Ref]
  • Vleuten C, Verwijnen GM, Wijnen WHFW. Fifteen years of experience with progress testing in a broblem-based learning curriculum. Medical Teacher. 1996;18:103–109. doi: 10.3109/01421599609034142. [Cross Ref]
  • Walsh K, Rafiq I, Hall R. Online educational tools developed by heart improve the knowledge and skills of hospital doctors in cardiology. Postgraduate Medical Journal. 2007;83:502–503. doi: 10.1136/pgmj.2007.059501. [PMC free article] [PubMed] [Cross Ref]
  • Wehrs VH, Pfafflin M, May TW. E-learning courses in epilepsy-concept, evaluation, and experience with the e-learning course “genetics of epilepsies” Epilepsia. 2007;48:872–879. doi: 10.1111/j.1528-1167.2007.01029.x. [PubMed] [Cross Ref]

Articles from Springer Open Choice are provided here courtesy of Springer