The findings of this study indicate that the form of prescriber feedback used here—mailed, unsolicited, centralised, government sponsored, and involving aggregate data—is unlikely to have any impact on the prescribing activities of general practitioners. This rigorous randomised controlled design showed no impact of the interventions, either on median prescribing rates or on variability in prescribing rates. The trial had sufficient statistical power to detect any important effect. It should be noted from the graphs that for some drug groups a decline in prescribing rate seemed to follow the intervention; in the absence of a valid control group (which in this study displayed the same trends), we might have concluded that the intervention was having some impact.
A recent Cochrane review of 37 randomised controlled trials concluded that audit and feedback of data on practice activity can sometimes be effective in changing the behaviour of healthcare professionals.6
There have, however, been few rigorous evaluations of the impact of feedback on prescribing activity.7–14
Of these eight randomised controlled trials only four concerned prescribing by general practitioners or family physicians in private practice, and only one evaluated feedback from a government department or authoritative agency.7
Other reviews have concluded that to be effective, feedback should offer clear alternatives to current practices; be part of an overall strategy for changing behaviour; and be presented close to the time of decision making.8,15,16
The authority of the “messenger” is important, and the “message” needs to be repeated at intervals to sustain any effect.16
In addition, the process should be active rather than passive, and the participants should have agreed previously to review their practices.16,17
Reasons for ineffectiveness of feedback
There are several possible reasons why the form of feedback described here was ineffective. The first is the ambiguity of the message. Though very high prescribing of non-steroidal anti-inflammatory drugs and antibiotics is undesirable, prescribing rates alone cannot be used to judge the quality of the use of other classes of drugs such as angiotensin converting enzyme inhibitors and lipid lowering drugs. Consequently, it is difficult for prescribers to respond to graphical displays of their own prescribing rates for these latter groups of drugs. The educational messages that accompanied the feedback in this trial were of a general nature and not individualised according to the profiles of individual general practitioners.
So the intervention studied here, although easy to implement on a national scale, lacked some of the features that are thought necessary to effect important behaviour change. The feedback was not close enough to the time of prescribing; it was seen to come from a national agency with no local input or ownership; there was no opportunity for discussion of the data in a problem based format; and the intervention did not offer alternatives to the drugs being highlighted in the feedback. Lastly, there were no real incentives to the participants to change their behaviour.
As a small (but important) impact might still have been expected we were surprised that the intervention had no discernible effect on prescribing behaviour. This is an important message, particularly for government agencies planning to conduct similar feedback in the belief that it will have a modest impact on prescribers or possibly sensitise them to other educational messages. In Australia payment for drugs and medical services takes place under a national health insurance programme (Medicare). The Health Insurance Commission, which is a statutory authority, has responsibility for the operational aspects and for investigating professional fraud and overservicing. The ambiguity in its role—as investigator rather than an educator—may have had a negative impact.
Limitations of study
There are some limitations to this study. The methods were rigorous, but there were weaknesses in some of the outcome measures. Australia has a complex system of reimbursement of pharmacists for cost of pharmaceutical products.18
The Health Insurance Commission will hold a record of a transaction only if the cost of the drug is higher than the contribution from the patient ($A18.00 at the time of the study). In the case of pensioners or holders of concession cards, however, the government covers virtually the full cost of all the study drugs. Data capture for non-steroidal anti-inflammatory drugs and antibiotics will have been incomplete, but this should have been equal in the intervention and control groups and therefore not a source of bias. Data capture for the other drug classes was complete, and there were no qualitative or quantitative differences in the response to the interventions between drug classes.
In conclusion, we believe that centralised, government sponsored prescriber feedback is not worth while and should not be seen as a high priority by government agencies. Prompt detailed feedback of individualised prescribing data in a clinical setting with an effective educational programme, however, may be effective. In Australia we believe these interventions should be carried out at the local level, possibly by divisions of general practice. The prescribing data that we used in this study could be useful but will need to be augmented by more clinically relevant datasets.