We estimate a longitudinal Multiple Indicator Multiple Cause (MIMIC) model of performance. The observed performance measures are assumed to be linear functions of an unobserved latent variable, and measurement error. We posit that variation in true performance is determined by time-varying and time-invariant covariates representing measured heterogeneity, and by stochastic error components representing unmeasured heterogeneity.
Specifically, our model is formulated as follows. Let Yjit
denote the j
indicators of childhood immunization for plan i
in period t
, where j
=6 and t
=5 in our application. We assume that the indicators are a function of true performance (q
) but are also affected by random variation that in part may include measurement error denoted by u
. The reported HEDIS scores (Y
's) are derived using either the administrative or hybrid method. Because reporting method (hybrid versus administrative) may affect the values of the indicators (Y
), but should not affect true performance (q
), we can write the model as:
The vector λ0jt
consists of indicator-period-specific constants that represent means of the indicators in each period t
=1998, 1999, 2000, 2001, and 2002. The vector λ1j
comprises HEDIS-specific factor loadings that measure the strength of the relationship between the latent variable and the HEDIS measures. Equation (1)
also shows how our empirical model allows us to aggregate individual measures, thereby allowing us to estimate transition probabilities based on the latent variable q
. We posit that the variation in the observed indicators (Y
) is generated by variation in q
and variation in u
. If u
represents measurement error, then the relevant part of the variation in Y
is captured by variation in q
. Note that q
) is not subscripted by j
We assume that the latent variable is a function of covariates, Xit
Xit includes plan and market characteristics, e.g., age of the plan, profit status, HMO penetration in the market, HMO competition, etc. In addition to the effect of the measured covariates, we posit that several unmeasured factors affect performance. These factors are denoted by vit. For example, several potentially relevant variables such as contracted physicians' performance or the priority the plan places on quality improvement are not available in our data and represent unmeasured heterogeneity across plans that might affect both the level and the growth in performance.
The unmeasured component vit
is specified as
The random component δi represents unmeasured plan-specific and time-invariant factors that affect levels of performance, while the term (ηi×t ) represents unmeasured plan-level heterogeneity in growth rates. We allow for correlation in the components δi and ηi, which we assume are jointly normally distributed.
In addition, it
represents transitory shocks in each period that might have spillover effects in the periods that follow. For example, a new Surgeon General's report in year (t
) that highlights the importance of childhood immunization might improve the HMO scores on childhood immunization in year (t
) and in subsequent years, but the effect of this “shock” might wane over time. We let it
denote the shock to performance induced by the warning and assume that it is a normally distributed AR(1) variable.
Given that we assume all of the stochastic components of the model are normally distributed, it follows that qit
is also distributed normally. In particular,
is the mean of vector of variables X
in period t
is the variance–covariance matrix of vit
. The six measures and five time periods provide sufficient information to identify and estimate the coefficients and parameters of the variance–covariance matrix. As noted earlier, we use data on all 457 plans that reported at least 1 year of data. However, identification of the plan-level heterogeneity components and the autocorrelation parameter use information on plans that were in our sample for two or more years. The model parameters, including the β
's and all the components of Σvv
are estimated via maximum likelihood.
Computing the Transition Probabilities
Once we know the mean and variance–covariance matrix of qit, we completely define the distribution of q. All the required transition probabilities can be computed from this distribution. Without loss of generality, we classify plans into three different tiers of performance (high, middle, and low). We use the distribution of plans in 2000 to define the two thresholds that determine whether a plan is in the high, middle, or low tier. Specifically, the upper threshold is the mean of plan latent performance (q) in 2000 plus the standard deviation of plan latent performance in 2000. Analogously, the lower threshold is the mean of plan latent performance (q) in 2000 minus the standard deviation of plan latent performance in 2000. Once we have estimated the parameters of the model, we can compute the probability of whether a plan will be in the upper, middle, or lower tier of performance conditional on any pattern of past performance.
We note that the transition probabilities depend only on the absolute performance of the plan (i.e., own HEDIS score) rather than being dependent on the scores of other plans. An alternative approach that redefined thresholds each year could be developed, but such an approach may be more difficult to interpret by consumers because the ratings would reflect, in part, the performance of other plans in the sample. This could result in a change in the plans' rating despite no change in the plan performance.
Because some of our transition probabilities involve the evaluation of multivariate normal integrals, we use the Geweke, Hajivassiliou, and Keane simulator (Geweke, Keane, and Runkle 1994
) to compute the probabilities (Appendix SA2, available on the web). Our model includes both time-invariant and time-varying measured and unmeasured variables in the probability statements, and thus the estimated transition probabilities can vary across plans and over time as well. We outline the procedure we use to compute the transition probabilities, but the details are provided in an appendix to this paper (Appendix SA2).