Home | About | Journals | Submit | Contact Us | Français |

**|**HHS Author Manuscripts**|**PMC2880516

Formats

Article sections

- Abstract
- 1. Introduction
- 2. Models used in MI inference
- 3. Multiple imputation under a multivariate linear mixed-effects model (MLMM)
- 4. A simulation study
- 5. Discussion
- References

Authors

Related links

Comput Stat Data Anal. Author manuscript; available in PMC 2010 June 3.

Published in final edited form as:

Comput Stat Data Anal. 2010 March 1; 54(3): 790–801.

doi: 10.1016/j.csda.2009.01.016PMCID: PMC2880516

NIHMSID: NIHMS199197

See other articles in PMC that cite the published article.

Multivariate extensions of well-known linear mixed-effects models have been increasingly utilized in inference by multiple imputation in the analysis of multilevel incomplete data. The normality assumption for the underlying error terms and random effects plays a crucial role in simulating the posterior predictive distribution from which the multiple imputations are drawn. The plausibility of this normality assumption on the subject-specific random effects is assessed. Specifically, the performance of multiple imputation created under a multivariate linear mixed-effects model is investigated on a diverse set of incomplete data sets simulated under varying distributional characteristics. Under moderate amounts of missing data, the simulation study confirms that the underlying model leads to a well-calibrated procedure with negligible biases and actual coverage rates close to nominal rates in estimates of the regression coefficients. Estimation quality of the random-effect variance and association measures, however, are negatively affected from both the misspecification of the random-effect distribution and number of incompletely-observed variables. Some of the adverse impacts include lower coverage rates and increased biases.

The existence of missing data is inevitable in many settings. Among them are multilevel applications where missing values are arbitrarily seen in the observational units nested within naturally occurring groups, such as patients nested within hospitals. As most statistical methods for analyzing data under such circumstances are not designed to handle arbitrary missingness (some model fitting techniques can handle longitudinal data with drop-out under missing at random), in general, missing values are recognized as a complicating factor to the analyses. Indeed, identifying appropriate analytical tools to draw the desired inferences in the presence of missing data is often a daunting task because such tools need to account for the reasons for missingness (i.e. missingness mechanism) as well as the key features of the multilevel design such as levels of variations. Ignoring these essential characteristics may result in misleading results such as biased parameter estimates (e.g. underestimated variances) and reduced statistical power (Rubin, 1976, 1987; Schafer, 1997).

One of the most popular methods in conducting inference from incomplete data applications is multiple imputation (MI) (Rubin, 1987). MI is a simulation-based inferential tool operating on *M* > 1 “completed” data sets, where the missing values are replaced by random draws from their respective predictive distributions (e.g. posterior predictive distribution of missing data). These *M* versions of completed data are then analyzed by standard complete-data methods and the results are combined into a single inferential statement using rules to yield estimates, standard errors and *p*-values that formally incorporate the missing data uncertainty into the modeling process (Rubin, 1987). The key ideas and the advantages of MI are given by Rubin (1987) and Schafer (1997).

The crucial step in MI is filling in the missing data by drawing from the posterior predictive distribution of the missing data (i.e. the conditional distribution of the missing data, given the observed data and the unknown parameters). This typically involves positing a parametric model for the data and using it to derive this conditional distribution. In multilevel data applications, multivariate extensions of the mixed-effects models based on normality have often been perceived as a natural assumption because (1) it reflects the design features; and (2) the conditional distribution of the missing data given the observed data is easily tractable (Schafer and Yucel, 2002). There are two software products implementing the MI-based inference in multilevel data applications. The first one is freeware and available as a R (R Development Core Team, 2007) package called pan (Schafer and Yucel, 2002), and the second one is a macro called mimacro (Carpenter and Goldstein, 2004) which operates within a commercially available software package MLwiN (Rasbash et al., 2006). Because these are extensively used, it is important to examine how good their performance is when the underlying distributional assumption on the random-effects is not met. To the best of our knowledge, despite their growing use in the analyses of multilevel incomplete data, the plausibility and impact of the normality assumption of the random-effects have not been thoroughly evaluated. An evaluation using simulated examples that represent deviations from the standard normality assumption, such as skewness, non-zero peakedness, heavy tails, and flatness of the density that clearly violate the characteristics of normality, is provided in the current study. Performance of MI under varying distributional assumptions and sample sizes is also examined. Throughout the computations, R package pan (Schafer and Yucel, 2002) was used.

Section 2 gives a brief overview of the model used to impute the missing values for continuous or nearly-continuous variables and provides a discussion of the crucial assumptions of MI inference. Section 3 then summarizes how multiple imputations are created under a multivariate generalization of mixed-effects models. Section 4 outlines and discusses the findings of the simulations. Finally, Section 5 concludes the manuscript with a discussion of the implications of these findings.

Below, a brief review on the imputation model commonly used to jointly impute missing values for a set of continuous random variables under a multivariate linear mixed-effects model is given. This model assumes two levels in the data collection, e.g. repeated measurements from individuals or secondary sampling units (US residents) within primary sampling units (US Census blocks). Allowing higher levels of nesting is possible by incorporating the corresponding random-effects in the imputation model (1) below, which has been implemented in MLwiN (Browne, 2003, ch. 17; Carpenter and Goldstein, 2004). We focus on the two-level model because it simplifies the assessment of the impact of the departures from normality and it is the underlying model of the R package pan.

Let *y _{i}* denote an

$${y}_{i}={X}_{i}\beta +{Z}_{i}{b}_{i}+{\epsilon}_{i},$$

(1)

where *X _{i}* (

Following the standard notation in the missing data literature, the complete data *Y* is comprised of the observed data *Y _{obs}* and

Explicit or implicit assumptions are made about the missing-data mechanisms in all missing-data methods. To set the notation for the discussion of missingness mechanisms, let *R* denote the set of missing-value indicators; note that *R* has the same dimension as *Y*, and it is always observed. Each element of *R* takes the value of 0 or 1 depending on whether the corresponding element in *Y* is missing or observed, respectively. Similar to *Y*, *R* can be seen as a random variable; and the conditional distribution of *R* given *Y* depends on a set of parameters, say γ. The small letters *r* and *y _{obs}* denote the realized values of

Most tools available to the practitioners of MI (via software packages such as SAS PROC MIXED, R packages norm, cat, mix and pan) assume *missing at random* (MAR) as the missingness mechanism. This assumption simply means that the missingness probability may depend on the observed data but not on the missing data over the conditional distribution of *R* given *Y _{obs}*. More formally,

A brief overview of MI and how it is conducted under a multivariate linear mixed-effects model (MLMM) for a two-level incomplete data is given (see Schafer and Yucel (2002) for computational details). MI is a Monte Carlo simulation-based technique where the missing values are replaced by a moderate number (*M* > 1 and usually less than 10) of simulated values. How these simulated values are drawn relate to distributional assumptions and the missingness mechanism (Schafer and Yucel, 2002; Browne, 2003, ch. 17; Carpenter and Goldstein, 2004).

The principle idea of MI is that it treats the missing data as an explicit source of random variability which is incorporated into computation of standard errors. The process of creating imputations, analyzing the imputed data sets and combining the results is a Monte Carlo version of averaging the statistical results over the predictive distribution of the missing data, ∫ *P*(θ|*Y _{obs}*,

The role of the imputation model in multilevel data such as those arising in longitudinal or clustered designs, is to preserve the relationships among variables (subject to missingness or not) and correlations among the observations from the same subject or cluster. Often the meaning or interpretation of its parameters is not crucial. The key importance of the imputation model is in its functionality to predict, simulate the missing observations and to capture, as much as possible, reasons for the missingness (to make the MAR plausible). MLMM given in (1), for these reasons, has found a common use among imputers. While other models of a similar nature are also possible, we focus on model (1) as it is the only one implemented in software products (R package pan and MLwiN).

Multiple imputations under model (1) are *M* independent draws ${Y}_{\mathit{\text{mis}}}^{(1)},\dots ,{Y}_{\mathit{\text{mis}}}^{(M)}$ from a posterior predictive distribution for the missing data, *P*(*Y _{mis}*|

An MCMC procedure called the Gibbs sampler is the underlying iterative simulation algorithm used to draw the missing values from *P*(*Y _{mis}*|

To reflect the uncertainty in the unknown parameters θ, the Bayesian paradigm has often been used in the preceding MCMC to create multiply imputed data sets. Under the model given in (1), independent inverted Wishart priors are used on the variance components: Σ^{−1}, Ψ^{−1}: Σ^{−1} ~ *W*(ν_{1}, Λ_{1}) and Ψ^{−1} ~ *W*(ν_{2}, Λ_{2}), where *W*(ν, Λ) denotes a Wishart variate with ν > 0 degrees of freedom and mean νΛ > 0. It is helpful to regard ${\nu}_{1}^{-1}{\Lambda}_{1}^{-1}\phantom{\rule{thinmathspace}{0ex}}\text{and}\phantom{\rule{thinmathspace}{0ex}}{\nu}_{2}^{-1}{\Lambda}_{2}^{-1}$ as prior guesses for Σ and Ψ with confidence equivalent to ν_{1} and ν_{2} degrees of freedom, respectively. For β, an improper prior uniform ‘density’ over ^{pr} is often employed. In practice, determination of these hyperparameters can be difficult. Here, we used the R package **mlmmm** (Yucel, 2007) which computes maximum likelihood estimates of model (1) parameters to assign as the initial guesses for Ψ and Σ and as the initial values β^{(0)}, Σ^{(0)} and Ψ^{(0)} for the Gibbs sampler.

Computational details of the algorithm derived under these priors, Gaussian assumptions on the *b _{i}*, ε

Studies in the multiple imputation literature indicate that inference by MI is quite robust to departures from model assumptions used to derive the posterior predictive distribution of missing data *P*(*Y _{mis}*|

The next section assesses the viability of the Gaussian assumption in extensive simulation scenarios representing a diverse set of distributional violations of the assumed normality of the random-effects. We devise a simulation study primarily to generate the conditions under which the incomplete data were realized and where the imputation approach relies on the incompatible normality assumption. This strategy provides a feasible way of evaluation of the performance of the MLMM imputation with normal random-effects (Demirtas, 2004, 2005).

The characteristics of the MLMM-based MI when the real data-generating mechanism employs non-normal random effects are evaluated via the simulation study. Simulation design consists of the following steps: data generation, imposing a missingness mechanism, MI inference under a hypothetical analyst’s model, parameter estimation, and evaluation of this estimation.

A bivariate linear mixed-effects model was used to generate the complete data:

$${[{y}_{\mathit{\text{ij}}1}{y}_{\mathit{\text{ij}}2}]}^{\mathrm{T}}={[{\beta}_{1}{\beta}_{2}]}^{\mathrm{T}}+{[{b}_{i1}{b}_{i2}]}^{\mathrm{T}}+{[{\epsilon}_{\mathit{\text{ij}}1}{\epsilon}_{\mathit{\text{ij}}2}]}^{\mathrm{T}},$$

(2)

where *i* = 1, 2, …, *m*, and *j* = 1, 2, …, *n _{i}*. The number of clusters (

The error term ε_{i} are assumed to be distributed as *N*(0, Σ) and our simulations used values of σ_{11} = 0.9, σ_{12} = 0.3 and σ_{22} = 0.4. Subject-specific random-effects *b _{i}* = [

**Multivariate normal distribution:**This is the default assumption under MLMM imputation, and was primarily used for comparative reasons. Random variates, say*b*, were drawn from*N*_{2}(0, Σ_{b}):Elements of Σ$$f(b|{\Sigma}_{b})={(2\pi )}^{-1}{|{\Sigma}_{b}|}^{-1/2}\phantom{\rule{thinmathspace}{0ex}}\text{exp}\phantom{\rule{thinmathspace}{0ex}}\left(\frac{1}{2}{b}^{\mathrm{T}}{\Sigma}_{b}^{-1}b\right)\phantom{\rule{thinmathspace}{0ex}}.$$_{b}were set as: ${\sigma}_{{b}_{1}}^{2}=3\phantom{\rule{thinmathspace}{0ex}}{\sigma}_{{b}_{2}}^{2}=1.5$ and σ_{b1,b2}= 0.25. These values were also used in the following distributions.**Multivariate t distribution:**Multivariate*t*distribution with ν = 4 was used to observe the behavior of MI under MLMM when the underlying distribution of the random-effects has heavier tails than normal (see Fig. 1). Here the random variates follow the density function of the form:$$f(b|{\Sigma}_{b})=\frac{\Gamma ((\nu +d)/2)}{\Gamma (\nu /2){\nu}^{d/2}{\pi}^{d/2}}{|{\Sigma}_{b}|}^{-1/2}\phantom{\rule{thinmathspace}{0ex}}{\left(1+\frac{1}{\nu}{b}^{t}{\Sigma}_{b}^{-1}b\right)}^{-(\nu +d)/2}.$$**Multivariate Laplace distribution:**Multivariate Laplace distribution or double exponential distribution was considered to examine the behavior of MI inferences under distributions with heavier tails than normal (see Fig. 2). The density function is given by,$$f(b|{\Sigma}_{b},\gamma )=\frac{\gamma \Gamma (1)}{2\pi \Gamma (2/\gamma )}{|{\Sigma}_{b}|}^{-1/2}\phantom{\rule{thinmathspace}{0ex}}\text{exp}\phantom{\rule{thinmathspace}{0ex}}(-{({b}^{\mathrm{T}}{\Sigma}_{b}^{-1}b)}^{\gamma /2}).$$**Multivariate uniform distribution:**Random bi-variates from*U*(0, 1) were generated with Σ_{d}_{b}to assess the behavior under a flat density.

Throughout the simulation study, we considered two general cases: (1) missingness is confined to covariates; and (2) arbitrary missingness is present in both response and covariates of the analyst’s model which is described in Section 4.1.3. To study the first case, we imposed missing values on the covariate *Y*_{2} under the missingness mechanism MAR defined by

$$\text{log}\frac{P({r}_{{Y}_{2}}=1|{Y}_{1})}{1-P({r}_{{Y}_{2}}=1|{Y}_{1})}=1.3+1.1{y}_{1},$$

where *r*_{Y2} is the missingness indicator for *Y*_{2}. Conditioning on completely-observed *y*_{1} makes the MAR assumption (Little and Rubin, 2002) as assumed by MLMM imputation, more plausible. The model stated above results in approximately 45% missingness in *Y*_{2} values. Tables 1 and and22 in the results section pertain to case (1).

Summary of the results under random-effects following a multivariate normal and *t*_{4}. Missingness rate is (approximately) 45% on *y*_{2}. For each parameter of interest (PI), MI results and complete-case only analysis are given in the first and second rows, **...**

Missing values were imposed under two independent intercept-only regression models leading to MCAR mechanisms (stronger assumption than MAR) to study the arbitrary missingness on both covariates and responses:

$$\text{log}\frac{P({r}_{{Y}_{1}}=1)}{1-P({r}_{{Y}_{1}}=1)}=-0.2,\text{logP(rY2=1)1\u2212P(rY2=1)=0.2.}$$

Under these specifications, rates of missingness in *Y*_{1} and *Y*_{2} are, on average, 45% and 55%. Tables 3 and and44 in the results section pertain the case (2).

To mimic the practice of MI, the model given in (3) is assumed to underlie the substantive goal of the analysis, i.e. the analyst’s model. Hence, we performed MI inference on the parameters of this model and evaluated the estimation quality using the criteria given in Section 4.1.4.

$${y}_{1,\mathit{\text{ij}}}={\beta}_{0}+{\beta}_{1}{y}_{2,\mathit{\text{ij}}}+{b}_{i}+{\epsilon}_{\mathit{\text{ij}}},$$

(3)

where *b _{i}* and ε

One of the important issues in MI inference is the compatibility of the MI model with the analyst’s model as well as the data-generation mechanism. MI practitioners are typically advised to include auxiliary variables either to account for the causes of missingness or to improve the missing data procedure. Collins et al. (2001) discuss this subject extensively in cross-sectional settings and present a simulation study assessing the benefits and costs of inclusive and restrictive MI strategies. Theoretical aspects of the compatibility of the MI model with the analyst’s model was earlier explored by Meng (1994), who also coined the term “uncongeniality” (i.e. incompatibility of MI and analyst’s model). In general, to prevent inferential biases, the MI models should be kept general enough to imply any particular analyst’s model.

Multiple imputations were created using the R package pan. The simulation procedure (data generation, imposing missing values, creating MIs, finding the estimates of the analyst’s model (3) for each of the imputed datasets, and combining these estimates using Rubin’s rules) was repeated 1000 times to fully describe the behavior in a repetitive sampling environment as well as to reduce the simulation error as much as possible. Several quantities were used to measure the accuracy as well as the efficiency of parameter estimates under MLMM-based MI when the distributional assumptions are violated. Briefly, these quantities are (assuming the unknown parameter is θ):

**Coverage rate (CR):**The percentage of times that the true parameter value is covered in the 95% confidence interval. Here the true parameter value is the average parameter estimate across the simulations before the missing values are imposed. If a procedure is working well, the actual coverage should be close to the nominal rate of 95% in our study. However, it is important to evaluate coverage with the other measures because high variances can lead to higher CRs. The performance of the procedure is regarded to be poor if its coverage drops below 90% (Collins et al., 2001). If the procedure results in CRs that are close to 100% or below 85%, extra caution should be taken when using that procedure.**Root-mean-square error (RMSE):**Because nonresponse or missing values have undesirable effect on the variances, it is important to evaluate this adverse effect. An integrated measure of bias and variance is used, evaluating in terms of combined accuracy and precision.*RMSE*() is defined as $\sqrt{E{(\hat{\theta}-\theta )}^{2}}$.**Average width of confidence interval (AW):**The distance between the average lower and upper confidence interval limits across 1000 confidence intervals. A high CR along with narrow, calibrated confidence intervals translates into greater accuracy and higher power.

As noted in Section 3, it is important to monitor the convergence of the Gibbs sampler in the R package pan, the computational engine for producing the multiple imputations under model MLMM (1). To assess the convergence, the Gibbs sampler was run for 10,000 cycles under mild prior distributions; we set ${\nu}_{1}=2,{\Lambda}_{1}^{-1}=2\hat{\Sigma},{\nu}_{2}=2,\phantom{\rule{thinmathspace}{0ex}}\text{and}\phantom{\rule{thinmathspace}{0ex}}{\Lambda}_{2}^{-1}=2\hat{\Psi}$ where and were computed by the R package mlmmm (Yucel, 2007). We assessed time-series plots and sample autocorrelations for elements of β, Σ and Ψ for several of the simulations. This exercise suggested that, even in the extreme cases of *m* = 100, *n _{i}* = 10 and

The simulation results focus on the performance of the MLMM-based MI on the parameter estimates of the analyst’s model given by (3) (i.e. _{0}, _{1} and _{b}). Results show that the residual variance term is mostly unaffected by the current model misspecifications, and are, therefore, not reported here. Tables 1–4 present the results from scenarios that mimic a missing covariate (*y*_{2} missing on average 45% under MAR) and an arbitrary missingness (both *y*_{1} and *y*_{2} missing on average 45% and 55%, respectively, under MCAR). The first column of Tables 1–4 denotes the respective distributions of the random-effects and error terms used to simulate data and sample size (*m*, *n _{i}*). The second column denotes the parameter of interest (PI): β

Tables 1 and and22 show that, across the board, the MI inferences lead to acceptable parameter estimation and coverage, indicating a good performance of the MLMM-based MI procedure. We specifically observe estimates with minimal biases and CIs with excellent coverage rates, even when the distribution assumption is clearly violated and when the sample sizes are small. The performance of the MI is far superior to the unprincipled method of simple case-deletion, which leads to significant biases as well as dismal coverage rates (in some cases it is as dramatic as 0%). It is also noted that in some conditions the performance of the MI procedure is not as satisfactory. When the underlying distribution of the random-effects has heavier tails than normal, specifically a *t* distribution (see the bottom panel in Table 1), we observe weak performance (e.g. higher RMSEs, lower CRs and larger AWs) in the estimation of σ_{b}. However, in most conditions, regardless of the distributional assumption of the random-effects, the MI procedure results in well-calibrated estimates with minimal bias and coverage rates that are around the nominal rate of 95%. This consistent performance across the distributions and the varying sample sizes indicate that, under the MLMM-based MI inference, large and small sample properties of the maximum likelihood estimates are satisfied (Demidenko, 2004). Examples of these properties include unbiasedness, smaller uncertainty measures, and decreased RMSE for increased sample size. In the mixed-effects framework, we follow the convention of large *m* and *n _{i}* for “large sample” while noting that the large

Overall performance of the MLMM-based MI is weaker when both variables are subject to missingness under the non-normal random-effects. We note bigger biases, relatively lower CRs, higher RMSEs and larger AWs in Tables 3 and and44 compared to Tables 1 and and2.2. This simply means that under arbitrary patterns of missing data, the inferences are more vulnerable to the misspecification of the random-effect distribution as well as to the small sample sizes. The estimates describing the associations and the spread of the random-effect distribution are most impacted. There is little discernible difference in the overall performance in the estimation of β_{0} between the two missing-data scenarios (univariate versus arbitrary). When both response and covariate are subject to missingness, even though the estimation of β_{1} is still impacted by the relatively higher RMSE, AW, the CR and small biases indicate that the inference on the fixed-effects remains insensitive to the non-normality of the random-effects and to the small sample sizes. This essentially implies that the MI inference leads to estimates with good frequentist properties even when the joint aspects of data are not well-estimated due to arbitrary patterns of missing data. However, when the distribution of random-effects is not normal, the validity of the MI inferences on the random-effect variance is highly questionable in terms of bias, CR or RMSE.

Recall that data generation scenario (given by model (2)) assumes a weak correlation between *Y*_{1} and *Y*_{2}. To assess whether this assumption has any discernible impact on the performance measures, a similar simulation experiment to the one presented above under three different conditions for weak, moderate and high correlations among *Y*_{1} and *Y*_{2} was also conducted. All scenarios are in agreement on well-calibrated MI inference under MLMM. Higher correlations between *Y*_{1} and *Y*_{2} translate into the improved accuracy measures of performance, such as reduced AWs, higher rates of coverage, and lower RMSE. However, it should be noted that even when *Y*_{1} and *Y*_{2} are weakly correlated, these performance measures still indicate inferences with desirable properties (e.g. unbiasedness, low average width, low RMSE, close to 95% coverage rates). The effect of the correlations has little impact on the overall performance measures (Table 5).

The overall goal of our work was to assess the impact of the normality assumption of the random effects on the MI-based inferences in multilevel incomplete data applications. In problems where the missing values are confined to a small number of variables, or when there are sources for auxiliary information that can be used as predictors in the imputation model, the assumption of normality does not seem to matter on the final inferences, even when the random-effects are wrongly assumed to be normal. However, when the missingness is arbitrary in many variables and the rates of missingness are high (or when the sample sizes are small), the validity of MI-based estimation is more adversely affected under the similar non-normal random effects. For incorrectly specified distributions, this negative effect is more clearly seen in parameters describing the relationships and/or the distribution of the random-effects. This is somewhat expected, as very little useful data might be available to estimate the “relatedness” aspects of the distribution. Our simulation study also suggests that taking no action about the missing data can lead to inferences that are far more misleading, regardless of correct or incorrect specification of distributional assumptions.

For the purposes of assessing the sensitivity of MLMM-based MI to distributional assumptions on the random-effects, a limited number of distinct distributions was considered. Therefore, the simulation study does not claim to be generalizable, as there are many other scenarios that can be potentially encountered in real data applications. However, we believe that our simulation study is valuable in the sense that it sheds some light on the effects of making wrong assumptions on the distribution of the random effects (or any distribution on the subject-specific effects when there is not much data support for estimation) in the imputation model.

MI-based procedures, when cautiously used, produce reasonably well-calibrated inference as long as the missingness is not widely spread across the data. However, it is important for the practitioners of missing data methods not to use the MLMM-based software blindly. Similar to the general practice of model-fitting and the associated checks for assessing the goodness-of-fit, inference by multiple imputation should always be performed in conjunction with the tools necessary to assess the plausibility of the underlying assumptions. What are these tools? While they are not as widely implemented as the imputation procedures, there are still well-established techniques for this purpose. Posterior predictive checks, as suggested by Gelman et al. (1996, 2003), can be employed to assess the plausibility of the imputation model. As a direct check for the quality of the assumed model in creating imputations, one can devise a calibration strategy to make the imputations better represent the structure of the data and possibly the varying structure of the clusters. Post-imputation checks and calibration-based methods are presented in detail by Gelman et al. (2005) and Yucel et al. (2008).

The results suggest a number of directions for future research. The first direction is the imputation models with less restrictive distributional assumptions such as those based on marginal models. These models can be particularly useful in small sample scenarios, especially when the missingness is seen indiscriminately in the response or covariates. A second potentially fruitful direction is to explore the ad-hoc methods of imputation, e.g. predictive mean matching applied to multilevel settings. These methods usually make minimal assumptions, but may lead to underestimation of the standard errors in the multilevel settings. However, with improvements to adjust the standard errors, the pay-off might be considerable when there is a reasonable hesitation to use a fully parametric approach for multiple imputation. Finally, we believe that the most important direction is the dissemination of any computational development and/or improvements by means of statistical software.

The authors thank the associate reviewer and the reviewer for helpful comments which greatly improved our manuscript. First author’s research was partially supported by the National Science Foundation.

- Bates D, Sarkar D. lme4: Linear mixed-effects models using s4 classes. 2007
- Browne WJ. MCMC estimation in MLwiN (version 2.0) London, UK: Institute of Education, University of London; 2003.
- Carpenter J, Goldstein H. Multiple imputation in MLwiN. Multilevel modelling newsletter 4. 2004
- Collins L, Schafer J, Kam C-M. A comparison of inclusive and restrictive strategies in modern missing data procedures. Psychological Methods. 2001;6:330–351. [PubMed]
- Demidenko E. Mixed Models: Theory and Applications. New York, NY: J. Wiley & Sons; 2004.
- Demirtas H. Simulation-driven inferences for multiply imputed longitudinal datasets. Statistica Neerlandica. 2004;58:466–482.
- Demirtas H. Multiple imputation under Bayesianly smoothed pattern-mixture models for non-ignorable drop-out. Statistics in Medicine. 2005;24:2345–2363. [PubMed]
- Demirtas H, Freels S, Yucel R. Plausibility of multivariate normality assumption when multiply imputing non-Gaussian continuous outcomes: A simulation assessment. Journal of Statistical Computation and Simulation. 2008;78:69–84.
- Gelman A, Carlin JB, Stern HS, Rubin DB. Bayesian Data Analysis. second ed. London: Chapman & Hall Ltd.; 2003.
- Gelman A, Mechelen I, Verbeke G, Heitjan D, Meulders M. Multiple imputation for model checking: Completed-data plots with missing and latent data. Biometrics. 2005;61:74–85. [PubMed]
- Gelman A, Meng X, Stern H. Posterior predictive assessment of model fitness via realized discrepancies. Statistica Sinica. 1996;6:733–807.
- Little R, Rubin D. Statistical Analysis with Missing Data. second ed. New York, NY: J. Wiley & Sons; 2002.
- Meng X. Multiple-imputation inferences with uncongenial sources of input. Statistical Science. 1994;10:538–573.
- R Development Core Team. R: A Language and Environment for Statistical Computing. R foundation for statistical Computing. Vienna, Australia: 2007. ISBN: 3-900051-07-0, URL: http://www.R-project.org.
- Rasbash J, Steel F, Browne W, Prosser B. Mlwin User’s Manual. Centre for Multilevel Modelling. Bristol, UK: 2006.
- Rubin DB. Inference and missing data. Biometrika. 1976;63:581–590.
- Rubin DB. Multiple Imputation for Nonresponse in Surveys. New York: John Wiley and Sons; 1987.
- Schafer J. Analysis of Incomplete Multivariate Data. London: Chapman & Hall; 1997.
- Schafer J, Graham J. Missing data: Our view of the state of the art. Psychological Methods. 2002;7:147–177. [PubMed]
- Schafer J, Yucel R. Computational strategies for multivariate linear mixed-effects models with missing values. Journal of Computational and Graphical Statistics. 2002;11:421–442.
- Yucel R. R mlmmm package: Fitting multivariate linear mixed-effects models with missing values. 2007. URL http://www.R-project.org.
- Yucel R, He Y, Zaslavsky A. Using calibration to improve rounding in imputation. The American Statistician. 2008;62:125–129.

PubMed Central Canada is a service of the Canadian Institutes of Health Research (CIHR) working in partnership with the National Research Council's national science library in cooperation with the National Center for Biotechnology Information at the U.S. National Library of Medicine(NCBI/NLM). It includes content provided to the PubMed Central International archive by participating publishers. |