Few articles on computer assisted learning in medical education have been published. A search of Medline and ERIC databases using the Mesh term “medical education” and free text terms “computer based” and “computer assisted” turned up around 200 potentially relevant studies, of which only 12 were prospective randomised studies with objective, predefined outcome criteria (table). These studies represent a range of different settings, interventions, and outcomes and are therefore not directly comparable. Most studies have methodological problems, including lack of statistical power, potential contamination between intervention and control groups, and attrition of the sample.
As the table shows, the randomised controlled trials had mixed but generally positive results. These suggested that the efficacy (the “can it work?” question described by Haynes17
) of high quality programmes in medical education is reasonably well established, a finding that is in keeping with meta-analyses of computer assisted learning in non-medical education.18
However, the effectiveness and cost effectiveness of these initiatives remain in doubt.
In the mid-1990s, at least two UK medical schools supplied all first year students with laptop computers and enhanced access to a range of networked multimedia applications. One project was never formally evaluated, but anecdotal reports suggested that many students found the computers expensive, impractical, and difficult to integrate with the mainstream curriculum (P Booton, personal communication). Results of the other project were published. The authors bravely admitted that some students made no use of their computers at all, technical glitches and incompatibility problems were common, staff were ill prepared for the change in learning medium, and “there was no academic organisational structure to shape a coherent response to the rapid increase in computer use.”19
Lack of engagement
Failure of students to engage with newly introduced technology is a recurring theme in reports on non-medical education. Perceived barriers include inadequate planning, poor integration with other forms of learning, and cultural resistance from staff. One ethnographic study in which students were closely observed while taking part in online courses showed that considerable frustration and time wasting arose from poor course design, technical glitches, “dead” hypertext links, poorly coordinated real time seminars, and ambiguous instructions.20
The only study of computer assisted learning in medical education that used comparable, in depth, qualitative methods found few such problems, but it was restricted to students' use of computers in a supervised classroom setting.21
Transferability and evaluation
Three important conclusions can be drawn from the reports. Firstly, innovators who have developed apparently successful products should be guarded about claiming that their systems are transferable, even when the efficacy of these systems has been shown in the research setting. Secondly, the evaluation of all educational technologies should include observation of unsupervised students attempting to gain access from remote sites and follow online links and instructions. Thirdly, neither course materials nor teaching skills are directly transferable from the traditional lecture theatre to the virtual campus. We should recognise, and take systematic steps to guard against, the danger of allowing inadequately trained tutors and lecturers to “go virtual.”
The differences in learning culture between computer based and traditional learning should not be underestimated, especially for the novice. As Reingold argues, “Fear is an important element in every novice computer user's first attempts to use a new machine or new software: fear of destroying data, fear of hurting the machine, fear of seeming stupid in comparison to others, or even to the machine itself.”22
One author has distinguished between students who “lose themselves” and those who “find themselves” in the virtual environment of email discussions,23
and another found that whereas some students perceived their virtual seminar group as part of a warm, friendly, and supportive online community, others perceived themselves facing a whole sea of strangers, perhaps reflecting different stages in the development of online learning skills (box ) or different learning styles.24
Box 4 : Stages of competence in online learning (adapted from Salmon24)
Issues of costs and training
The cost of hardware and software, and telephone line charges, often prove a more important barrier to accessing web based materials than the course organisers initially assume. The amount of training needed to become comfortable with specialised software packages is often underestimated; students on a course that relies heavily on computer work may spend most of their first term getting to grips with the technology. Few students learn all the essential technical skills at the outset of the course. Rather, they tend to use “just in time learning”—that is, most of them make no attempt to get to grips with a feature of the software until they actually need to use that feature. This suggests that too much initial training may not be popular or effective