As expected, there were no significant differences among the three intervention groups at baseline in any of the 16 central outcomes (Rosenheck et al., 2003
). Comparison of results using standard repeated-measures methods and using multiple imputation methods revealed differences in statistically significant results on six of 16 outcome measures, five of which involved measures of substance abuse and one of which involved housing outcomes (). We did not adjust the level of α
for multiple comparisons because days housed was the primary outcome measure, and the analysis of the other outcomes is exploratory. All the tests were two-tailed.
Statistical Evaluation of the Outcome Differences Across the Treatment Groups
While only one marginally significant difference was found between groups on substance abuse outcomes using ordinary repeated-measures analyses (HUD-VASH veterans used alcohol on fewer days than intensive case management controls; p value = 0.046), with multiple imputation analysis the HUD-VASH group had substantially and significantly fewer days of alcohol use than the standard care (i.e., short-term case management) group (p value = 0.0047); fewer days on which they drank to intoxication (p value = 0.0053); and fewer days of drug use (p value = 0.028); as well as lower scores on the ASI composite drug problem index (p value = 0.015; ).
plots the results for days of intoxication, using standard methods, while shows the results with multiple imputation. HUD-VASH clients were also found to have had lower scores than the intensive case management control group (group 2) on the ASI composite alcohol problems index (p value = 0.018).
FIGURE 1 Estimated days intoxicated over 3 years: repeated-measures analysis without multiple imputation. Days intoxicated for past 30 days at each visit time point (baseline = 0, 6, 12, 18, 24, 30, 36 months) were estimated and plotted against time point for (more ...)
FIGURE 2 Days intoxicated over 3 years by treatment group: repeated-measures analysis following multiple imputation. After the multiple imputation for missing values, we plotted the estimated days intoxicated against the seven visiting time points. From this plot, (more ...)
The only nonsubstance abuse measure for which results changed with multiple imputation concerned nights slept in institutions. In the original analysis, the HUD-VASH group had fewer nights in institutions than the standard care (i.e., short-term case management) group (p value = 0.021). With multiple imputation, HUD-VASH veterans were also found to have spent fewer days in institutions than the intensive case management without vouchers group (p value = 0.030).
While there were no significant differences in money spent on alcohol and drugs among any of the three intervention groups in the original analysis, with multiple imputation, the HUD-VASH group was also found to have significantly lower expenditures on alcohol and drugs (p = 0.048) than the standard care group.
We further explore the analysis of the number of days of drinking to intoxication to understand better the difference in results between the original analysis and the analysis with multiple imputations. As noted, the analysis with multiple imputations, but not the standard analysis, showed that the HUD-VASH group had fewer days of intoxication than both the intensive case management only group and the short-term case management group. The explanation for this difference is that subjects with more days of intoxication were significantly more likely to miss follow-up visits. The values restored by the imputation method are thus most likely to be those pertaining to observations that are expected to have reported more days of intoxication. Since data loss was greatest in the intensive case management only and short-term case management groups, they had more imputed observations that reported days of intoxication than the HUD-VASH group. Differences between the results with multiple imputation analysis and the original analysis on the other outcomes have similar explanations.