The use of this approach enabled us to identify 1124 articles dealing with telemedicine. One hundred and thirty-three full-text articles were obtained for closer inspection. Of these, 50 were deemed to fulfill the inclusion criteria of the review and are listed and briefly described in .10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59
Fourteen of the studies considered the application of telemedicine to medical consultation of various types, and 7 dealt with patient monitoring or counselling. A further 13 were concerned with teleradiology, and the remainder with emergency department care, psychiatry, dermatology, cardiology, ophthalmology and pathology.
Thirty-four of the articles assessed at least some clinical outcomes; the remaining 16 were mainly economic analyses. Some kind of economic analysis was included in 30 (60%) of the studies. In terms of study design, the quality of the clinical studies ranged in most cases from fair to poor. According to the Jovell and Navarro-Rubio classification,9
6 were randomized controlled trials (RCTs), corresponding to levels 2 or 3, 4 were level 4 or 5, 13 level 6, 6 level 7 and 5 level 8. Conditions of scientific rigour varied considerably. In many papers, procedures for the selection of patients and for the reading and interpretation of clinical findings were not adequately described. The outcome measures used were sometimes vaguely defined or clinically not very relevant.
Although RCTs provide the strongest study design, the strength of the evidence obtained will also be dependent on the quality of the study. The 6 papers that were located that described RCTs provide an illustration of the variation in study quality and reported outcomes. Two of the larger RCTs29,52
were well described in terms of the randomization and subsequent procedures. The first of these, which considered automated telephone calls and management of diabetes, showed improvement in glycemic control and other benefits through the use of a telemedicine approach. However, no effect on health-related quality of life (HRQOL) was demonstrated. The second, which considered real-time teledermatology, indicated that there was no significant clinical difference from conventional consultations. A linked economic analysis indicated that teledermatology was not cost-effective under the conditions of the trial.
A further report of a large RCT28
had a more limited description of randomization but showed that telelemedicine using a telephone-based system improved compliance with medication and led to a significant decrease in blood pressure.
Of the papers about smaller RCTs, that by Brennan and colleagues45
appeared to have been well performed; the authors found that clinical outcomes were similar for telemedicine and for the alternative approach in an emergency department setting. Another paper, which described a pilot project for a larger RCT, indicated time savings for patients as a result of video consultations, but no significant difference in HRQOL between groups.17
This study appeared to be more limited in quality, with substantial dropping out of patients and possibly insensitive outcome measures. The third small RCT found no significant difference between tele–exercise monitoring and a hospital-based program.24
The power of the study was low, and further investigations would be needed to assess this application.
The nonrandomized clinical studies also varied in their quality, as judged by the descriptions in the articles, and in their outcomes. Some would have provided useful indications to decision-makers in the health systems concerned. For example, the study by Trippi and colleagues44
showed that 72% of patients scheduled for hospital admission had normal results in dobutamine stress tele-echocardiography and could, therefore, be discharged instead of being admitted to hospital. Giovas and colleagues46
reported that pre-hospital diagnosis by electrocardiography, using a telemedicine link to the ambulance, took place 25 minutes before hospital diagnosis for a control group. Other studies indicated important clinical benefits through avoiding the unnecessary transfer of patients. For example, Goh and colleagues37
reported that the use of teleradiology in the management of neurosurgical patients reduced both numbers of transfers and adverse events during transfer and also increased the number of therapeutic measures before transfer was undertaken.
In most of the studies, effectiveness was defined in clinical terms. Only 2 studies17,29
included standardized HRQOL measures. No studies employed quality-adjusted life-year (QALY) calculations. Given the diversity of the studies in terms of design, topics covered, populations and health care settings, calculation of a notional average for effect size was not feasible. Indications of effect sizes for some of the studies are given in .
Few comprehensive economic analyses were included in the articles. The analyses mainly measured direct medical costs, although some kind of estimation of transportation costs was included in 25 studies. Indirect costs were assessed in 4 studies,22,34,53,59
incremental cost analysis was performed in one18
and cost-effectiveness ratios were also calculated in one.28
Discounting of costs was included in 7 studies,18,20,22,34,39,52,58
and 14 included some kind of sensitivity analysis or break-even analysis of the study results.
Most of the economic analyses were variants of cost analysis. Cost–benefit analysis was said to have been carried out in 3 studies.13,41,52
However, these were methodologically more like cost-analysis studies, because the benefits were estimated as savings (mainly the cost of travel) compared with the conventional alternative.
Demonstrated savings in costs of transportation varied considerably among the different health care situations described in the papers, from a 40% reduction to no savings as a result of telemedicine. Three of 4 studies of the transmission of diagnostic images indicated that telemedicine was more costly than the cheapest alternative.42,54,59
Economic analyses have mostly shown that teleradiology, especially transmission of CT images, can be cost-saving, although one of the studies, which was of good quality, did not find this to be the case.34
An important contribution to the discussion about the cost-effectiveness of teleradiology is the study by Bergmo,39
which explicitly provides a measure of the workload that has to be exceeded in order to achieve cost savings by using teleradiology (break-even analysis). A similar study, also undertaken by Bergmo, has shown that specialist consultations in the field of otorhinolaryngology can be performed in a cost-saving way when the workload exceeds a certain number of patients.18
Pilot projects in telepsychiatry, the provision of orthopedic and dermatology services via telemedicine and the evaluation of the costs and benefits of a prison telemedicine program used a similar approach.22,23,47
Such studies that give a clear number needed to treat by the telemedicine option are helpful for decision-makers when faced with the question of whether or not to start a new telemedicine service. Teledermatology, with short distances (26 km) between sites, appeared not to be cost-saving in one study.52
The quality of the economic analysis in the papers was relatively low, with a few exceptions. The papers by Bergmo,18,39
Agha and colleagues,59
Stensland and colleagues,22
Halvorsen and colleagues34
and Wootton and colleagues52
provide examples of better-quality economic studies. The costs included varied significantly among studies, so that comparison of the cost estimates may not be feasible in many cases. There were also several economic studies that did not give detailed information about the empirical background of the costs or benefits, or both, included in the calculations. These studies were excluded from the review. For example, we excluded a teleradiology cost–benefit analysis,60
because the theoretically good economic model did not make use of the empirical cost and benefit estimations made at specific sites by the study group.