We found no effect of a computerised decision support system as a vehicle for implementing evidence based guidelines for the management of two chronic diseases in primary care. We addressed the complexities of such management where clinicians provide ongoing care for patients with complex conditions and for extended periods. At the time of its deployment the system was embedded in two of the most widely used practice computer systems in the United Kingdom and was more sophisticated than any other commercially available to primary care facilities in the United Kingdom. We were, however, unable to show any incremental effect over the distribution of paper versions of the guidelines across a comprehensive range of measures, almost certainly due to the low levels of use.
A systematic review suggested caution in interpreting the results of identified studies of computerised decision support systems because of flaws in their design or analysis, a common situation in studies on changing professional behaviour.3,18
However, we addressed all of the important issues for design, conduct, and analysis.4,19
Asthma and angina are common chronic diseases in primary care. Both are of low incidence so dimensions of care such as initial investigations would be infrequent and less likely to be affected by any method of guideline implementation. Much of both guidelines dealt with the ongoing management of established cases, and the low incidence of both conditions should not have affected this. None the less, there are areas of care that an interactive computerised decision support system may be less able to influence, such as the issuing of routine repeat prescriptions by administrative staff. However, good clinical practice suggests that most patients with angina or asthma should receive an annual review, and within the trial almost all the patients consulted sufficiently frequently for this to have been considered. Although the guidelines reflected much of current practice—and performance of some actions may have been close to optimal (for example, use of short acting β2 agonists)—they both made recommendations for management that were not routine at the time of the study. Therefore it is unlikely that the guidelines merely enshrined all of current practice, and the data on process of care before the intervention show that this was not the case.
Although the study practices were selected on the basis of the extensive use of their computer systems (and thus were most likely to use a computerised decision support system), the staff had limited training in the functioning and use of the system. Limiting the amount of training in our study to one day was a pragmatic decision based on resources but was not that far removed from what is routine computer training within primary care in the United Kingdom.
For most general practitioners in the study the computerised system functioned in the context of routine surgeries (as opposed to settings such as clinics dedicated to disease management). Patients could present with any clinical problem such as arthritis or depression and, despite having asthma or angina, might not wish to discuss this, even though the computerised system might suggest this was appropriate. Given the range and complexity of problems that patients present to general practitioners it is demanding for any system to function in an unobtrusive yet helpful manner. The negative findings and low levels of use in our trial are similar to those observed by Hetlevik et al, who evaluated a computerised decision support system for the management of patients with hypertension or diabetes in primary care.20,21
They found that the guideline was used in the management of only 12% of patients with diabetes.21
Assuming that the technical challenges of producing a system that truly supports the management of complex disease can be overcome there remains the problem of how such systems function within clinical encounters where patients with complex conditions are managed. “To be widely accepted by practising clinicians, computerised support systems for decision making must be integrated into the clinical workflow. They must present the right information, in the right format, at the right time, without requiring special effort.”22
It is at least possible that for some or all of the reasons discussed above the low levels of use are all that can be reasonably expected of a computerised decision support system for the management of chronic disease.21
Certainly, in terms of implementing evidence based care, computerisation seems unlikely ever to be the “magic bullet” that answers all questions, and the current system could not be recommended.23
Although an increasing number of studies show that computerised decision support systems can function in a variety of circumstances, the challenge still remains to show how far this is possible, desirable, and efficient.3