This is the second of two reviews of trials of preventive home visits to elderly people published in the BMJ
in the past 18 months. Elkan et al conclude that home visits reduce mortality and admissions to nursing homes, whereas last year's review found no evidence supporting their effectiveness and argued that existing programmes should be reconsidered.1-1
Why did the two reviews reach such contrasting conclusions?
The main reason is the different methodological approaches adopted by the two groups. Van Haastregt et al reported the results from individual trials as “no significant effects” or “significant favourable effects.”1-1
For example, they found a “significant” reduction (P<0.05) in admissions to institutions in only two out of seven trials and that overall effects were “modest and inconsistent.” This “vote counting” approach is clearly unsound as it ignores the direction and size of effects from individual studies and their confidence intervals.1-2,1-3
If the BMJ
and other journals adopt the recent recommendation that “the description of differences as statistically significant is not acceptable,”1-4
then the confusion created by such analyses could be avoided.
In contrast to the paper by Van Haastregt et al the present review used meta-analysis to summarise results. The potential of this approach is illustrated in the figure, which shows the effects on admission to long term care: six out of eight trials show a beneficial effect of preventive home visits. The evidence against the null hypothesis was fairly strong in two trials (Stuck P=0.021 and Hall P=0.025) but weak in the others (P>0.10). The pooled analysis, however, indicates that there is convincing evidence for a clinically important reduction in the risk of admission to long term institutional care (P=0.001). The reduction in the odds of admission is likely to be at least 17% and could be as large as 51%.
Figure Meta-analysis of eight trials of effect of preventive home visits on admission to long term institutional care. Data taken from table 4. Elkan et al's classification of study population (general elderly population or frail elderly) and mortality in control (more ...)
Van Haagstregt et al argued that the data should not be combined statistically, given the heterogeneous nature of the interventions and the populations enrolled in the different trials.1-1
Interestingly, there was little evidence of heterogeneity between trials in the analysis shown in the figure (P=0.46) and those performed by Elkan et al. The power of tests of heterogeneity is notoriously low and combining studies is always questionable if there is important clinical heterogeneity. However, only by graphically and statistically analysing effect estimates from individual trials can we identify factors introducing heterogeneity. Elkan et al attempted this but their analysis was limited to a few crude factors. For example, they explored the importance of the underlying risk by stratifying trials according to whether older people from the general population or frail elderly people had been enrolled. They found no difference between these groups, which may be due to misclassification of the Hall study. This trial was supposedly performed in frail elderly people, but mortality in the control group was low (see figure). When the effects are ordered according to mortality, as shown, they get smaller with increasing mortality in the control group (figure). This important finding was recently confirmed by Stuck et al in a trial designed to examine effects in older people at low and high risk for admission to a nursing home.1-5
The analysis carried out by Elkan et al found no improvement in functional status, which is inconsistent with the rationale for home visits. How could mortality and admissions to a nursing home be reduced without an effect on functional status? Unfortunately, only four studies contributed to this analysis, confidence intervals were wide, and Elkan et al did not contact investigators to obtain additional data. Future reviewers should collaborate with original investigators to define the exact characteristics of interventions, obtain data on implementation and adherence, and standardise outcome measures and quality assessment. Several additional trials which have been published recently will increase the power of their analyses. The results are likely to generate useful hypotheses, which should be addressed in trials that are powered to examine effects across prespecified interventions and subgroups of elderly people. Trials and meta-analyses show that preventive home visits can work. The challenge now is to tease out which components of the intervention are effective and which populations are most likely to benefit.