We identified relevant articles in the Cochrane Effective Practice and Organisation of Care register and pending file in January 2004. We also examined the reference lists of retrieved articles.
We included randomised controlled trials involving healthcare professionals. Audit and feedback was defined as “any summary of clinical performance of healthcare over a specified time period”. We included only those studies that objectively measured provider performance in a healthcare setting or healthcare outcomes.
Two reviewers (GJ and JMY) independently selected studies for inclusion, extracted data and assessed the quality of the study.7
An overall quality rating (high, moderate, low protection against bias) was assigned on the basis of the following criteria: concealment of allocation, blinded or objective assessment of primary outcome(s), and completeness of follow‐up (mainly related to follow‐up of professionals) and no important concerns in relation to baseline measures, reliable primary outcomes or protection against contamination. We assigned a rating of high protection against bias if the first three criteria were scored as done, and there were no important concerns related to the last three criteria, moderate if one or two criteria were scored as not clear or not done, and low if more than two criteria were scored as not clear or not done.
We considered audit and feedback in addition to interactive, small group meetings separately from audit and feedback in combination with written educational materials or didactic meetings, which have been found to have little or no effect on professional practice.1,9,10
We defined “multifaceted” interventions as including two or more interventions.
We categorised the intensity of feedback as “high”, “moderate” or “low” on the basis of combinations of the following components: recipient, format, source, frequency, duration and content of the feedback.
The complexity of the targeted behaviour and the seriousness of the outcome were categorised in a subjective manner independently by GJ and JMY, or GJ and ADO as “high”, “moderate” or “low”. Baseline compliance with targeted behaviour for dichotomous outcomes was based on the mean value of pre‐intervention level of compliance in both the audit and feedback group and the control group.
We included only studies of moderate or high quality in the primary analyses, and studies that reported baseline data. Three analyses were conducted across all types of interventions (audit and feedback alone, audit and feedback with educational meetings or audit and feedback as part of a multifaceted intervention compared with no intervention): one using the adjusted risk ratio as the measure of effect, one using the adjusted risk difference and the third using the adjusted percentage change relative to the control mean after the intervention. All outcomes were expressed as compliance with the desired practice. Professional and patients' outcomes were analysed separately.
We considered the following potential sources of heterogeneity to explain variation in the results:
- the type of intervention
- the intensity of the feedback
- the complexity of the targeted behaviour
- the seriousness of the outcome
- baseline compliance with desired practice
- study quality
We visually explored heterogeneity by preparing tables, and bubble and box plots to explore the size of the observed effects in relationship to each of these variables. We also plotted regression lines to aid the visual analysis of the bubble plots.
The visual analyses were supplemented with meta‐regression to examine how the size of the effect was related to the six potential explanatory variables, weighted according to the number of healthcare professionals. The main analysis comprised a multiple linear regression using only main effects; baseline compliance was treated as a continuous explanatory variable and the others as categorical. These analyses were conducted using generalised linear modelling in SAS.
As there were important baseline differences in compliance between the intervention and control groups in many studies, our primary analyses were based on adjusted estimates of effect, where we adjusted for baseline differences in compliance.