Search tips
Search criteria 


Logo of nihpaAbout Author manuscriptsSubmit a manuscriptHHS Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
Demogr Res. Author manuscript; available in PMC 2010 May 11.
Published in final edited form as:
Demogr Res. 2010 January 26; 22(6): 129–158.
doi:  10.4054/DemRes.2010.22.6
PMCID: PMC2867357

Estimation of multi-state life table functions and their variability from complex survey data using the SPACE Program1


The multistate life table (MSLT) model is an important demographic method to document life cycle processes. In this paper, we present the SPACE (Stochastic Population Analysis for Complex Events) program to estimate MSLT functions and their sampling variability. It has several advantages over other programs, including the use of micro-simulation and the bootstrap method to estimate the variance of MSLT functions. Simulation enables researchers to analyze a broader array of statistics than the deterministic approach, and may be especially advantageous in investigating distributions of MSLT functions. The bootstrap method takes sample design into account to correct the potential bias in variance estimates.

1. Introduction

The multi-state life table (MSLT) model is a “time-inhomogeneous, finite-space, continuous-time” Markov model (Schoen 1988:64). Demographers frequently use it to analyze stochastic processes involving multiple and recurrent events to estimate expected duration in various states: for example functional limitations (Crimmins, Hayward, and Saito 1994, 1996; Land, Guralnik, and Blazer 1994); HIV/AIDS (Palloni 1996)’ labor force participation (Hayward and Grady 1990; Hayward, Grady, and McLaughlin 1988): cohabitation and marriage (Bumpass and Lu 2000; Espenshade and Braun 1982; Hofferth 1985; Schoen and Land 1979): poverty (Duncan and Rodgers 1988): and living in socio-economically disadvantaged neighborhoods (Quillian 2003). It has also been used in studies linking longevity to medical spending for the aging population (Goldman et al. 2005; Lubitz et al. 2003).

When the MSLT model was originally developed, life tables were calculated using population-level rates – hence there was limited attention given to estimation techniques and variability in the life table functions. Increasingly, however, the data source is panel data obtained via survey sampling, making it difficult to obtain reliable estimates of life table inputs (i.e., transition rates or probabilities) directly from raw data (Saito, Crimmins, and Hayward 1991). As a result, regression models are used to produce smoothed estimates of life table input, and the variance of MSLT functions is now needed to test hypotheses about the differences between population subgroups.

Currently there are two programs that are publicly available to perform these tasks – the IMaCh (Interpolated Markov Chain) program and the GSMLT (Gibbs Sampling Multistate Life Table) program.8 The IMaCh program is introduced by Lièvre, Brouard and, Heathcote (2003). It estimates transition probabilities using a discrete-time “embedded” Markov chain (eMC) approach, developed by Laditka and Wolf (1998).9 The eMC approach, in contrast to the traditional event history approach, recognizes that observed sample data often fail to include many events of short duration between infrequent follow-up interviews (Hardy and Gill 2004). To recover these missing events, the eMC approach applies the MSLT model to shorter transition periods (e.g., monthly, quarterly, etc.) that are “embedded” within the longer interval between follow-ups. The variance of MSLT functions is estimated as a linear function of the variance and covariance of transition probabilities using the Delta method. Currently, the IMaCh program can estimate health expectancy (HE) and “equilibrium” prevalence of health states (i.e., implied by transition probability estimates in a stationary population), and their variances. The GSMLT program is introduced in Lynch and Brown (2005) and takes a Bayesian approach to estimating life table quantities. In its Bayesian framework, transition probabilities are estimated using a constrained discrete-time multinomial probit model (Lynch and Brown 2005). Once the iterative algorithm has converged, it draws a large number of samples from the posterior distribution of transition probabilities to generate for each sample a multi-state life table, from which the posterior distribution of life table functions can be obtained. The posterior distribution allows researchers to obtain estimates on a range of distributional statistics, including their means and variances. Currently, the GSMLT program only estimates HE and its variance.

This paper introduces another program – the SPACE (Stochastic Population Analysis for Complex Events) program. While the MSLT model fitted in the SPACE program takes the traditional event history approach (i.e., assuming no missing events of short duration between successive interviews), it has two analytical advantages over the other two programs. First, the SPACE program uses micro-simulation to estimate MSLT functions. Micro-simulation is an increasingly popular computational tool in demography research (e.g., Cai and Lubitz 2007; Cai, Schenker, and Lubitz 2006; Laditka et al. 2007; Laditka and Wolf 1998; Lubitz et al. 2003; Wolf 1986; Wolf, Laditka, and Laditka 2002). It “expresses” the transition probability estimates by generating detailed life paths for each member of the target population. As a result, the SPACE program allows users to estimate a variety of MSLT functions directly from simulated data. This computational flexibility compares favorably to the rigidness of the deterministic approach used in the IMaCh program where only a limited number of summary statistics can be produced.

Second, the SPACE program takes a different approach to estimating the variance of MSLT functions. In the IMaCh program, the variance of HE is estimated as a function of the variance and covariance of transition probabilities via the Delta method. In the GSMLT program, the variance is estimated from the posterior distribution of HE. The SPACE program uses the bootstrap method instead. The bootstrap method is a data resampling method commonly used to derive variance estimates when analytic methods are unavailable (Efron and Tibshirani 1986). Since the SPACE program can estimate various MSLT functions via micro-simulation, the bootstrap method can produce variance estimates for all MSLT functions estimated. In addition, the bootstrap method used in the SPACE program is specifically developed for large-scale, complex surveys, which are frequently the primary data source for MSLT applications (e.g., the Medicare Current Beneficiary Survey, the Health and Retirement Study and the Medical Expenditure Panel Survey, etc.). These surveys have design elements such as stratification and multi-stage clustering, which, if not adequately controlled for, will result in biased variance estimates and invalid statistical inference (Lohr 1999).

In the following sections, we will discuss the details of the SPACE program and present some results from an application of the SPACE program to highlight its differences from the IMaCh program and to demonstrate its usefulness. We chose to compare the estimates, by gender and race/ethnicity, with the IMaCh program because it is relatively easy to implement and widely used. The SPACE program is currently available upon request from the authors, and will be available for download from REVES (

2. The SPACE Program

The SPACE program is a statistical program developed to estimate the MSLT functions from survey data. It consists of multiple sets of PC-based SAS® programs with different modeling capacity. However, they are structured similarly: each set of the program contains two components – the data component and the statistical component. The data component prepares the input data sets – both for the full analysis sample and for the bootstrap samples drawn from the full sample – for the statistical component. The statistical component estimates transition rates or probabilities and the MSLT functions based on the estimated parameters. The output includes estimates of MSLT functions and their variances.

2.1 The MSLT Model

The MSLT model characterizes population movement over time in a finite, discrete and mutually exclusive state space as a Markov process. Two types of MSLT models can be fitted in the SPACE program: one follows a first-order Markov chain where the transition probabilities are conditional only on the current age and status, and another follows a semi-Markov process (SMP) model where the transition probabilities are age, status and duration dependent. Since we compare to the IMaCh estimates in this study, we will only consider the duration-independent MSLT model in detail here.

The SPACE program takes the traditional event history approach to estimating the MSLT model parameters by assuming that the observed events are independent, as in the IMaCh and GSMLT programs, and complete (i.e., no missing events between two successive observations). When a person has the same status in both occasions, then it is assumed that no event has occurred between the two observations; if the observations are different, then it is assumed that only one event has occurred. This assumption is restrictive, as spells of short durations may occur frequently between interviews (Hardy et al. 2005). As interviews become less frequent (e.g., every five years), the number of events that the model fails to capture are likely to rise and additional bias can be introduced into the estimates of MSLT functions (Gill et al. 2005; Wolf and Gill 2009)

The SPACE program uses a class of discrete-time hazard models to estimate the age-specific transition probabilities or rates from sample data. The SPACE program can fit a multinomial logistic regression as in Laditka and Wolf (1998) to estimate the transition probabilities directly. The equation takes the following basic form:

equation M1

where pij (age, t) is the transition probability from the current state i to state j (i, j = 1,2,3,…, n, ij) over the annual interval at aget, aij is the intercept, and bij is the coefficient for age at the beginning of annual interval, and cij is the set of coefficients for other covariates.

Alternatively, the SPACE program can estimate the transition rates as in Hayward and Grady (1990) and Crimmins, Hayward, and Saito (1994). The model takes the following form:

equation M2

where μij is the instantaneous transition rates from current state i to state j (i, j = 1,2,3,…, n, ij) over the annual interval at aget. The transition rates are converted into transition probabilities using the formula in Crimmins, Hayward, and Saito (1994:165).

The SPACE program imposes no limit on the number of non-absorbing states in the MSLT models; the number of states depends only on the data and research objective. Also, multiple categorical covariates measured at study baseline (e.g., gender, race/ethnicity, education, etc.) can be included. These covariates may each have more than two categories and their values remain fixed during the survey period.10 Finally, the SPACE program permits uneven lengths in observation intervals, which are calculated as the number of years between interviews.11 The program converts input data from person-year format to annual interval format in which each line of data represents the movement from one interview to the next over the period of one year. If the length of the interval between interviews is two or more years, then a corresponding number of annual intervals are created to facilitate the computation of annual transition parameters. The events are assumed to occur randomly in one of these annual intervals. This approach results in the events occurring, on average, in the middle of the observation interval.

The outputs of these regressions are age-specific transition probability estimates for all possible transitions, conditional on the other covariates included in the model. The SPACE program provides two options to estimate MSLT functions: the deterministic approach as in IMaCh or GSMLT or micro-simulation. The deterministic approach can only estimate HE, while micro-simulation can be used to estimate HE and a variety of other MSLT functions. The details of micro-simulation are discussed in section 2.4.

2.2 Data

The study population for this analysis is drawn from the 1998–2002 panels of the Medicare Current Beneficiary Survey (MCBS). The MCBS is a nationally representative, multi-stage, longitudinal panel survey of the Medicare population, sponsored by the Centers for Medicare and Medicaid Services (CMS), and conducted continuously since 1991. The survey gathers data on a wide range of topics including health status, socio-demographic information, and the use and cost of medical services. Survey records are linked to administrative data on use of and expenditures on Medicare-covered services (hospital, physician, etc.) and on vital status.

The MCBS follows a rotating panel design. Each year a new panel that is representative of the current Medicare population is selected from the list of eligible beneficiaries. Each person in the panel is scheduled to receive 12 interviews over a four-year period, with information on self-reported health status collected once a year in the Fall. The panel is not renewed during the four-year period, but the rates of attrition are small and decline over time. Given the adjustment for survey non-response in sample weights, the bias in estimates is substantially reduced or eliminated (Kautter et al. 2006).

The MCBS has all the elements of a complex survey. Strata are created based on the characteristics of primary sampling units (PSUs), which are basically large geographical areas (i.e., a Metropolitan Statistical Area (MSA) or group of contiguous counties). The largest MSAs in the country are selected with probability one; each is essentially a “stratum.” Within these certainty strata, the individual zip-code clusters are considered PSUs for variance estimation. For each of the non-certainty strata, two PSUs are selected. The analysis sample used in this study has a total of 112 strata and 1,168 PSUs. The individual Medicare beneficiaries are then selected in the third stage (i.e., within each zip-code cluster) stratified by seven age groups (under 45, 45 to 64, 65 to 69, 70 to 74, 75 to 79, 80 to 84 and 85 and over). The oldest people (85 and over) and disabled people (64 and under) are oversampled to allow detailed analysis of their health status and health care needs.

For this study, we use the 1998–2002 panels that contain 14,892 elderly beneficiaries. We exclude 1,017 persons of Hispanic origin or other racial/ethnic groups to focus on non-Hispanic whites and blacks only, since the IMaCh program does not accept more than two categories at a time for any covariate. The full analysis sample contains 50,830 person-year observations for 13,875 persons of age 65 and older.

For explication purposes, we use a simple dichotomous measure of health based only on the presence of limitations in activities of daily living (ADLs). A person is considered disabled (i = 1) if he or she either responds “yes” to having difficulty with one or more of the six ADLs (bathing, dressing, eating, transferring, walking, and using the toilet), or responds “does not do the activity because of a health or physical problem.” Otherwise, this person is considered non-disabled or ‘active’ (i = 2). For those who report limitations with any of the activities, the survey also asks whether they receive help from another person and/or use special equipment. We do not consider “receiving help” in defining functional disability in this study. Survey respondents can move between the disabled and non-disabled states over time; while ‘dead’ (i = 3) is an absorbing state.

2.3 Estimation algorithms for model parameters

The regressions specified above are carried out by two SAS procedures: PROC LOGISTIC and PROC LIFEREG. These procedures offer powerful modeling capabilities. For example, users can use the current state as one of the covariates, instead of using it to stratify the data. Users can select the best functional form for the whole sample, or choose different forms for subsets of the data.12 They can also relax the linear relationship between age and the logit to test other functional forms (e.g., logarithm or polynomial functions), or even to evaluate different forms of the link function (e.g., cumulative, multinomial or complementary log-log). This degree of flexibility is not available in the IMaCh and GSMLT programs.

2.4 Micro-simulation and bootstrapping

Micro-simulation is the main computation technique used by the SPACE program to estimate MSLT functions. Compared to the deterministic approach, micro-simulation holds substantial advantages in terms of the scope of statistics one can calculate. With the deterministic approach, one essentially moves the entire population through the transition matrices, with little insight into individual dynamics. As a result, only a few summary statistics can be derived. Micro-simulation, however, simulates the life path of all members of the population such that a wide variety of summary statistics of the population dynamics can be derived.

The population simulated in the SPACE program is not arbitrary. It is characterized by the estimated transition parameters, conditional on the covariates included in the regression models. To briefly describe how it works, suppose we want to simulate the life histories of a 100,000-person cohort of 65-year old black men. For a hypothetical member of this cohort, we first randomly assign an initial health status, say, active, at age 65 based on the weighted health distribution for black men at age 65 from the input data. We then evaluate possible health changes between age 65 and 66 by comparing a random number from the uniform distribution with the transition probabilities for the age 65–66 interval, given his current status of active health. If his health status changes to disabled, then we generate a new random number from the uniform distribution to compare with the transition probabilities for the age 66–67 interval, conditional on being disabled at age 66. The result of this comparison determines if his health status changes again between age 66 and 67, and is repeated one year at a time until his eventual death. Once this process is repeated for all members of the cohort, we have a complete record of individual health histories from which MSLT functions can be easily calculated by averaging over the individual records. For example, total life expectancy (TLE) is computed by the average number of years lived for the simulated 100,000-person cohort. HE, including expected length of time spent in both active health (ALE) and disability (DLE), is computed by the average number of years spent in each health state.

It is worth noting that the simulation is based upon parameters of a period life table, where the experience of an eighty-year old today is assumed to hold for the next twenty years when a current sixty-year old reaches his or her eightieth birthday. This is unlikely to be the actual experience of any individual. Therefore, the simulated life paths should not be regarded as the actual experience of a single cohort over time.

In order to test the group differences in MSLT functions, we need to estimate their variances. Evaluating group differences in MSLT functions is different from evaluating the differences in parameter estimates in the event-specific models since they arise from a complex set of transitions that are not immediately obvious. For example, suppose one wants to test the hypothesis that males and females differ in their expected health over the life cycle (i.e. expected years in various health states). Health expectancy is determined by age-specific rates of movement into and out of the health states, including the risk of death from each health state. How sex is associated with health expectancy, however, may be unclear for a number of reasons. Sex may significantly affect some transitions and not others. Or, sex effects, even though statistically significant, may be offsetting. It is also possible that sex effects may be non-significant for the whole set of transitions, yet the consistency of effects for a lengthy period of time may combine in a way in which the sex effects are reinforced and magnified with age. The hypothesis of sex differences in health is therefore global in the sense that it takes into account sex differences in all of the transitions defining the process of life cycle health.

In addition, variance estimation for a complex survey needs to consider sources of variability due to stratification and multi-stage clustering that are not present in a simple random sample (SRS). Treating a stratified sample as a SRS usually overestimates the variance, while treating a clustered sample as SRS usually underestimates the variance. Although the net effect is often not obvious, it is nonetheless clear that ignoring the complex sampling design can lead to incorrect statistical inference (Lohr 1999).

To address these issues, the SPACE program uses a version of the rescaling bootstrap method developed specifically for complex surveys (Korn and Graubard 1999:32–33; Rao and Wu 1988; Sitter 1992a), and which has been implemented in recent demographic studies (e.g., Cai and Lubitz 2007; Cai et al. 2006). This approach samples nh − 1 PSUs with replacement within the stratum h, where nh is the number of PSUs in stratum h. For each PSUi sampled from stratum h, the original sample weight is multiplied by equation M3, where mi is the number of times the PSUi is selected. If a rare event is not represented in a particular bootstrap sample, the sample can be redrawn.

It is worth noting that this particular procedure has two potentially offsetting sources of bias. First, this procedure resamples only at the PSU level and thus will underestimate the variance for a multistage survey. This source of bias is not likely to be significant, however, since the additional variability due to sub-sampling at later stages is usually negligible compared to variability at the PSU level (Lohr 1999). Second, this procedure draws the bootstrap samples with replacement, which may lead to overestimation of the variance for data sampled without replacement. This second source of bias may be negligible if the first-stage sampling fraction is small (Rao 1988). If not, then alternative procedures specifically developed for without-replacement samples (e.g., Bickel and Freedman 1984; Sitter 1992b) can be considered. But these procedures are more difficult to implement, however, and require knowledge of the sampling fraction, which is typically not available to researchers using the public versions of the survey data.

The bootstrap method usually requires more computation, and its theoretical properties in complex surveys are not fully studied (Lohr 1999). There is also evidence from simulation studies that the bootstrap method may not outperform the Jackknife and the Balanced Repeated Replication (BRR) methods for stratified one-stage SRS with replacement (Kovar, Rao, and Wu 1988). But, the bootstrap method also has a number of advantages. It can be used to estimate the variance for a broader class of statistics, including sample quantiles. It can also provide consistent variance estimators for surveys with imputed data, and has a “higher potential to be extended to other complex problems” than the BRR and jackknife approaches (Shao and Tu 1995:280). From a practitioner’s perspective, it is reasonable to conclude that the bootstrap method is a suitable all-purpose variance estimator for MSLT functions.

2.5 Comparison of SPACE with IMaCh and GSMLT

There are several major differences among the three programs. The first is their assumptions about the “completeness” of observed data. The IMaCh program takes the eMC approach assuming that observed data are incomplete and allows multiple events between successive interviews. The SPACE and GSMLT program, on the other hand, takes the traditional event history approach assuming that the observed data are complete and allows only one (or zero) event between successive observations. Although the assumption of eMC is conceptually more realistic, a recent study shows that the estimates of HE and transition probabilities from both approaches are surprisingly similar – both are biased (Wolf and Gill 2009). Since HE estimates may be insensitive to the length of the interval between interviews for up to two years (Gill et al. 2005), it is possible that estimates based on both approaches become more biased as the interview becomes less frequent.

Another difference lies in their treatment of the design factors of the survey data. Currently, the GSMLT program makes no adjustment for survey design and assumes input data are from a SRS, while the IMaCh program makes only limited adjustment by assuming that the survey design affects only the sample weight. Since the sample weight reflects the probability of the selection of individuals, not clusters, it cannot remove the bias in variance estimates in clustered survey data (Lohr 1999).

The three programs also differ in the scope of statistics they produce. The SPACE program permits users to estimate a variety of MSLT functions because of its use of micro-simulation, while the other two programs offer limited choices. The IMaCh and GSMLT programs focus on HE estimates. Although the GSMLT program also estimates MSLT functions that can be expressed as a function of the transition probabilities, many statistics that may be of interest to researchers do not meet this criteria (e.g., the probability of death within two years after two prior episodes of disability, or the median of healthy life years). In addition, the complexity of expressing the mathematical relation of MSLT functions to transition probabilities may be challenging for many users.

2.6 Application

In this section we will present the results from an application of the SPACE program and compare them to the IMaCh program. Table 1 shows selected characteristics of sampled persons in the study population. The majority of sampled persons are female, non-Hispanic, white, and between the ages of 65 and 74 and free of ADL limitations. The educational achievement of the panels shows some improvement between 1998 and 2002. The proportion with less than a high school education dropped from 29 percent to 24 percent, while the proportion of high school graduates and those with more than high school education (including college and vocational training) increased from 71 percent to 76 percent. The prevalence of active health dropped slightly from 62 percent in the 1998 panel to 59 percent in the 2002 panel, while the prevalence of ADL limitations increased slightly.

Table 1
Characteristics of the analysis sample of 1998–2002 panels in MCBS

2.6.1 Comparison of IMaCh and SPACE estimates

The coefficient estimates of the logistic regressions from both programs are shown in Tables 2A. We fit the same logistic regressions of the form of eq. (1) for both the IMaCh and SPACE programs. In Table 2A, the IMaCh coefficients are estimated with both one-month and 12-month transition intervals, and the SPACE coefficients are estimated with an annual interval. For both programs, the gender coefficients indicate that elderly women are more likely than elderly men to become disabled, while less likely to recover and to die. The race/ethnicity coefficients indicate that elderly non-Hispanic blacks are more likely to become disabled and die, while less likely than elderly non-Hispanic whites to recover from disability.

The SPACE coefficient estimates for the “annual” interval are slightly different from the IMaCh estimates with a 12-month interval, reflecting the differences in the measurement of the gap between interviews. The SPACE program does not take into account the variation in the actual interval between interviews. It arranges the longitudinal data into pairs of observations and estimates the transition probabilities between one time point and the next. In the case of MCBS data, although it is designed with 12-month intervals, the actual gap ranges from 8 months to 16 months (Table 1). This variation in time interval is ignored by the SPACE program, but not by the IMaCh program, which estimates month-to-month transitions.

Table 2B shows the equilibrium prevalence of health states that is used by the IMaCh program to estimate HE, as well as the smoothed prevalence estimates for SPACE estimates. The SPACE prevalence estimates are similar to the observed prevalence, while the IMaCh prevalence estimates of disability, whether using the one-month or 12-month interval, are noticeably lower.

Using the IMaCh estimates of coefficients with monthly transition interval in Table 2A and the period prevalence estimates in Table 2B, the IMaCh estimates of HE at age 65 by gender and race/ethnicity are shown in Table 3. We also include in Table 3 two sets of variance estimates – the bootstrap estimates that consider survey design and the IMaCh estimates that do not – in order to evaluate the degree of bias in variance estimates when complex survey design is ignored. The bootstrap variance estimates are obtained by first randomly selecting 250 bootstrap samples from the full analytic sample, and then using them as input data sets to the IMaCh program to derive 250 sets of IMaCh point estimates of life expectancy.13 The standard deviation of these 250 estimates are considered the bootstrap SEs of the original IMaCh point estimates and are compared to the IMaCh estimates that do not reflect the complex sampling design of MCBS.

Table 3
IMaCh estimates of health expectancy at age 65 and their standard errors

A common measure of such bias in survey research is the design effect (DEFF) – the ratio of the variance estimates that consider survey design (i.e., the bootstrap estimates) to the estimates that do not (i.e., the IMaCh estimates). Since stratification and clustering have opposing effects on sampling variability, the value of the ratio may suggest the relative size of these design factors: if the ratio is greater than one then the clustering effect may be stronger; if the ratio is less than one then the stratification effect may be stronger. Table 3 shows that all of the bootstrap estimates are larger than the IMaCh estimates, an indication of the larger clustering effect in MCBS. In some cases the bootstrap estimates are much larger. For example, the bootstrap variance of DLE for non-Hispanic white females is 75% larger than the IMaCh estimate.

2.6.2 Convergence of bootstrap variance estimates

A practical issue in the implementation of bootstrap method is how many bootstrap samples to draw. Efron (1987) suggested that 100 samples are sufficient for variance estimates, while other researchers have argued for a much higher number of replications, especially given the rapidly declining cost of computation (e.g., Booth and Sarkar 1998; Chernick 1999). From a practitioner’s perspective, a straightforward approach is to check the convergence pattern of bootstrap variance estimates as more and more samples are drawn. The analyst can typically decide on the number of samples to draw when the variance estimates begin to stabilize. In Figures 1A–1D, we plot the standard errors (SEs) for SPACE estimates of HE at age 65. While the patterns are different across gender and racial/ethnic groups, all four figures show that the SEs begin to stabilize after the first 500 samples. The fluctuations after 500 samples are very small, except maybe for the TLE estimates for 65-year old black men and black women. The SEs for these two groups are 0.60 and 0.52 with 2000 samples, representing a small difference of six and seven percent, respectively, from their corresponding values with 500 samples. The convergence patterns for the percentile estimates of years lived and spent in active health and disability are similar to Figures 1A–1D and not shown here. Since these small differences are not likely to affect the results of hypothesis tests, we choose to use the SEs with 500 bootstrap samples in Table 4.14

Figure 1Figure 1Figure 1Figure 1
Figure 1A: Convergence of SEs for HE estimates for non-Hispanuc white men at age 65
Table 4
SPACE estimates of the average and percentiles of years expected to live and spend in active health and disability at age 65, by gender and race/ethnicity

2.6.3 The distribution of years lived and spent in different health states

Table 4 shows the SPACE estimates of the average, median, 25th and 75th percentiles of the number of years lived, as well as the number of years spent in active health and disability at age 65, by gender and race/ethnicity, based on transition probabilities and prevalence estimated from the full analysis sample as shown in Table 2A and 2B. We also include the TLE at age 65 based on the National Vital Statistics System in 2000 as a comparison (Arias 2002). The SPACE estimates of TLE in Table 4 indicate some small differences from the 2000 Vital Statistics: the largest is 4.7 percent for white women. The differences are caused mostly by the lack of control for variation in the actual time interval between interviews in the SPACE program. We verified this source of difference by manually calculating the transition probabilities using the SPACE coefficient estimates and the IMaCh coefficient estimates with a 12-month transition interval in Table 2A. The IMaCh estimates of TLE with a 12-month interval are closer to the Vital Statistics in year 2000 than the SPACE estimates.

For both gender and racial/ethnic groups, estimates of the median, 25th and 75th percentile of the number of years expected to live and to spend in active health at age 65 indicate generally symmetric distributions, while the distributions of years spent in disability have a longer tail on the right. Due to space constraints, we only present the distributions of years spent in active health and disability at age 65 for non-Hispanic white and black men in Figures 2 and and3.3. Figure 2 shows that the distribution of years spent in active health for white males is generally shifted to the right of the distribution for black males. In substantive terms, this difference in distributions can be illustrated by the fact that 61% of black men spend fewer years in active health than the median number of years spent in active health for white males. In addition, the graph shows that the distribution of years spent in active health for black men is more tightly clustered around its median of 11.5 years compared to white men; the average deviation from the median is 3.5 years for blacks compared to 4.4 years for whites. Black men thus not only have a lower ALE (i.e., average number of years in active health) than whites but it also appears that they are more homogenous with regard to the distribution of years spent in active health than are whites.

Figure 2
Distribution of years spent in active health for 65-year old men
Figure 3
Distribution of years spent in disability for 65-year old men

Figure 3 indicates that the distributions of years spent in disability for black and white men are highly skewed to the left. About half of all men are expected to have 2 or fewer disabled years; about 25 percent will spend five or more years disabled after age 65. As is evident, the distributions are very similar for the two race groups, consistent with the results in Table 4. Thus, it appears that the distributions of years spent in active health differ substantially for black and white men, yet the distributions of years spent in disability do not.

3. Discussion and conclusion

This paper introduced the SPACE program, which is a collection of SAS® programs to estimate MSLT functions and their variances from simple as well as complex survey data, and demonstrated the usefulness of two of its main features – using micro-simulation to estimate a variety of MSLT functions and also the bootstrap method to obtain consistent variance estimates. We showed that the bootstrap variance estimates can correct the downward bias in the IMaCh estimates, which do not take into account the effect of complex survey design. Since most of applications of the MSLT model use data from large complex surveys, it is important to have design-adjusted variance estimates in order to correctly evaluate group differences in MSLT functions. We also showed that micro-simulation can be used to estimate the entire distribution, rather than a single location indicator, of MSLT functions. By combining the use of bootstrap and micro-simulation, the SPACE program offers researchers a useful and flexible means to model the dynamics of complex events.

The results in Wolf and Gill (2009) pose interesting questions about the event history approach to estimating the MSLT model. Wolf and Gill showed that neither the eMC approach nor the traditional event history approach could accurately reproduce the month-to-month changes in functional limitations among the elderly. Although the biases in HE estimates were small, the biases in the transition probability estimates were large. As a result, the authors conclude that, unless we have observed monthly data, we should not expect the MSLT models to replicate the “true” monthly dynamics.

Given the complexity in the disablement process, the results in Wolf and Gill (2009) are not surprising. Studies using the same Precipitating Events Project (PEP) data as Wolf and Gill (2009) found that the onset of and recovery from short-term disability episodes are strongly associated with the frailty of sampled persons, the severity and duration of disability, and the type of precipitating events (i.e., acute illness requiring hospitalization or not) (Gill, Williams, and Tinetti 1999; Hardy and Gill 2005). Since none of these characteristics of the individuals as well as of the events are controlled for in the simple Markov models, it is not surprising to see a large discrepancy between model predictions and observed values.

On the other hand, it is important to consider what constitutes the appropriate use of a MSLT model in demographic research. Many researchers would readily agree that the MSLT model provides only a crude and discrete approximation of the underlying stochastic and continuous process. Such approximation is useful because our information is always incomplete – the myriad of factors contributing to the underlying continuous process is never fully observed. Even if we can estimate monthly transition probabilities from the PEP data with all the additional factors mentioned in the last paragraph, it is still very likely that the predicted monthly transitions will miss many of the events of even shorter duration such as days or weeks. We believe that using MSLT parameter estimates to reproduce the underlying process on a different time scale is not only unnecessary but also not what the model is intended for. Instead, the model should only be used to estimate statistics on a time scale comparable to the input data. If the gap between interviews is measured in years, then the estimates of MSLT transition probability and functions should be measured on a yearly interval as well.

The MSLT model detailed in this study follows a first-order Markov chain where the transition probabilities are conditional on the current age and status, and other covariates. This assumption excludes the possible effect of duration on health changes, which has been reported in several recent studies (e.g., Crimmins and Saito 1993; Hardy et al. 2005; Hardy and Gill 2004). To address this issue, the SPACE program can also fit a duration-dependent MSLT model, the SMP-EM model. This model was developed in Cai, et al. (2006) to estimate the SMP model parameters using the EM algorithm in the presence of partially-observed duration data. It has been shown that the SMP-EM estimates of transition probabilities are more accurate than duration-independent MSLT estimates (Cai et al. 2008).

Another area that awaits further research is the way sample weights are used in MSLT calculations. The IMaCh program uses only a single weight across all monthly intervals within a person to estimate transition probabilities, effectively equalizing the sample representativeness of these “observations” at different time points. The SPACE program, on the other hand, can use multiple weights, one for each time interval. In the current study, we use the cross-sectional weights that correspond to the year when current health status is observed. Although the MCBS provides longitudinal weights to analyze persons across waves of observations, they are designed only for survivors of each panel (Ferraro and Liu 2005), and are not appropriate for analysis of the event of death. Based on our own calculation, health expectancy estimates do not appear to be affected by approaches to sample weights; but it may still be desirable to study this issue formally.

In conclusion, we believe that the SPACE program will be a useful analytical tool to researchers interested in using the MSLT model. When used properly, it will provide valuable insight into the dynamics of complex events that is unavailable in the other programs.


Partial support for this research was provided by an Interagency Agreement with the National Institute on Aging (NIA) and research grants from the NIA (R01 AG013180 and P30 AG17265), infrastructural support from the National Institute of Child Health and Human Development (5 R24 HD042849) and a grant obtained by the Nihon University Population Research Institute from the “Academic Frontier” Project for Private Universities matching fund subsidy from MEXT (Ministry of Education, Culture, Sports, Science and Technology, Japan), 2006–2010.


1Disclaimer: The findings and conclusions in this paper are those of the author(s) and do not necessarily represent the views of the National Center for Health Statistics, Centers for Disease Control and Prevention.

8There is another publically available program to estimate MSLT health expectancy (Weden 2005, downloadable from Since this program does not estimate the sampling variability of MSLT functions, we will not discuss it here.

9Land, Guralnik, and Blazer (1994) developed a continuous-time Markov process model with piecewise-constant hazard rates within each one-year age interval. It permits theoretically infinite number of events between two successive observations.

10Time-dependent covariates can be incorporated into MSLT models as in Yang and Hall (2008). Future versions of SPACE will address this issue.

11Currently the SPACE program estimates transition parameters over annual transition intervals. If the input data contains more frequent interviews (e.g., every 6 months), it can be easily modified to estimate transition parameters over shorter intervals.

12When sample data are drawn from complex surveys, variable selection should be carried out using specialized estimation procedures (e.g., using PROC SURVEYLOGISTIC in SAS) that take into account survey design. Otherwise estimated p-values are incorrect.

13We also examined the variance estimates from 500 bootstrap samples and found only small differences from those with 250 samples, an indication that the bootstrap variance estimates have stabilized with 250 samples.

14The convergence patterns for the SEs in Table 4 are different from the SEs in Table 3. This is likely the results of different estimation techniques – one using the annual transition interval (Table 4) and the other using monthly interval (Table 3).

This open-access work is published under the terms of the Creative Commons Attribution NonCommercial License 2.0 Germany, which permits use, reproduction & distribution in any medium for non-commercial purposes, provided the original author(s) and source are given credit.

See http://


  • Arias E. National Vital Statistics Reports. 3. Vol. 51. Hyattsville: National Center for Health Statistics; 2002. United States tables, 2000.
  • Bickel PJ, Freedman DA. Asymptotic normality and the bootstrap in stratified sampling. The Annals of Statistics. 1984;12(2):470–482. doi: 10.1214/aos/1176346500. [Cross Ref]
  • Booth JG, Sarkar S. Monte Carlo approximation of bootstrap variances. The American Statistician. 1998;52(4):354–357. doi: 10.2307/2685441. [Cross Ref]
  • Bumpass L, Lu H-H. Trends in cohabitation and implications for children’s family contexts in the United States. Population Studies. 2000;54(1):29–41. doi: 10.1080/713779060. [Cross Ref]
  • Cai L, Lubitz J. Was there compression of disability for older Americans from 1992 to 2003? Demography. 2007;44(3):479–495. doi: 10.1353/dem.2007.0022. [PubMed] [Cross Ref]
  • Cai L, Schenker N, Lubitz J. Analysis of functional status transitions by using a semi-Markov process model in the presence of left-censored spells. Journal of the Royal Statistical Society: Series C (Applied Statistics) 2006;55(4):477–491. doi: 10.1111/j.1467-9876.2006.00548.x. [Cross Ref]
  • Cai L, Schenker N, Lubitz J, Diehr P, Arnold A, Fried LP. Evaluation of a method for fitting a semi-Markov process model in the presence of left-censored spells using the Cardiovascular Health Study. Statistics in Medicine. 2008;27(26):5509–5524. doi: 10.1002/sim.3382. [PMC free article] [PubMed] [Cross Ref]
  • Chernick MR. Bootstrap methods: A practitioner’s guide. New York: Wiley; 1999.
  • Crimmins EM, Saito Y. Getting better and getting worse. Journal of Aging and Health. 1993;5(1):3–36. doi: 10.1177/089826439300500101. [Cross Ref]
  • Crimmins EM, Hayward MD, Saito Y. Changing mortality and morbidity rates and the health status and life expectancy of the older population. Demography. 1994;31(1):159–175. doi: 10.2307/2061913. [PubMed] [Cross Ref]
  • Crimmins EM, Hayward MD, Saito Y. Differentials in active life expectancy in the older population of the United States. The Journals of Gerontology Series B: Psychological Sciences and Social Sciences. 1996;51B(3):S111–S120. [PubMed]
  • Duncan GJ, Rodgers WL. Longitudinal aspects of childhood poverty. Journal of Marriage and the Family. 1988;50(4):1007–1021. doi: 10.2307/352111. [Cross Ref]
  • Efron B. Better bootstrap confidence intervals. Journal of the American Statistical Association. 1987;82(397):171–185. doi: 10.2307/2289144. [Cross Ref]
  • Efron B, Tibshirani R. Bootstrap methods for standard errors, confidence intervals, and other measures of statistical accuracy. Statistical Science. 1986;1(1):54–75. doi: 10.1214/ss/1177013815. [Cross Ref]
  • Espenshade TJ, Braun RE. Life course analysis and multistate demography: An application to marriage, divorce, and remarriage. Journal of Marriage and the Family. 1982;44(4):1025–1036. doi: 10.2307/351461. [Cross Ref]
  • Ferraro D, Liu H. Uses of the Medicare Current Beneficiary Survey for analysis across time. Paper presented at the Joint Statistical Meeting of the American Statistical Association; Minneapolis, MN. 2005.
  • Gill TM, Allore H, Hardy SE, Holford TR, Han L. Estimates of active and disabled life expectancy based on different assessment intervals. The Journals of Gerontology Series A: Biological Sciences and Medical Sciences. 2005. pp. 1013–1016. [PubMed]
  • Gill TM, Williams CS, Tinetti ME. The combined effects of baseline vulnerability and acute hospital events on the development of functional dependence among community-living older persons. The Journals of Gerontology Series A: Biological Sciences and Medical Sciences. 1999;54A(7):M377–M383. [PubMed]
  • Goldman DP, Shang B, Bhattacharya J, Garber AM, Hurd M, Joyce GF, Lakdawalla DN, Panis C, Shekelle PG. Consequences of health trends and medical innovation for the future elderly. Health Affairs. 2005 doi: 10.1377/hlthaff.w5.r5. [PubMed] [Cross Ref]
  • Hardy SE, Dubin JA, Holford TR, Gill TM. Transitions between states of disability and independence among older persons. American Journal of Epidemiology. 2005;161(6):575–584. doi: 10.1093/aje/kwi083. [PubMed] [Cross Ref]
  • Hardy SE, Gill TM. Recovery from disability among community-dwelling older persons. JAMA. 2004;291(13):1596–1602. doi: 10.1001/jama.291.13.1596. [PubMed] [Cross Ref]
  • Hardy SE, Gill TM. Factors associated with recovery of independence among newly disabled older persons. Archives of Internal Medicine. 2005;165(1):106–112. doi: 10.1001/archinte.165.1.106. [PubMed] [Cross Ref]
  • Hayward MD, Grady WR. Work and retirement among a cohort of older men in the United-States, 1966–1983. Demography. 1990;27(3):337–356. doi: 10.2307/2061372. [PubMed] [Cross Ref]
  • Hayward MD, Grady WR, McLaughlin SD. Recent changes in mortality and labor force behavior among older Americans: Consequences for nonworking life expectancy. The Journals of Gerontology Series B: Psychological Sciences and Social Sciences. 1988;43(6):S194–S199. doi: 10.1093/geronj/43.6.S194. [PubMed] [Cross Ref]
  • Hofferth SL. Updating children’s life course. Journal of Marriage and Family. 1985;47(1):93–115. doi: 10.2307/352072. [Cross Ref]
  • Kautter J, Khatutsky G, Pope GC, Chromy JR, Adler GS. Impact of nonresponse on Medicare current beneficiary survey estimates. Health Care Financing Review. 2006;27(4):71–93. [PubMed]
  • Korn EL, Graubard BI. Analysis of health surveys. New York: Wiley; 1999.
  • Kovar JG, Rao JNK, Wu CFJ. Bootstrap and other methods to measure errors in survey estimates. Canadian Journal of Statistics - La Revue Canadienne De Statistique. 1988;16(S1):25–45. doi: 10.2307/3315214. [Cross Ref]
  • Laditka JN, Laditka SB, Olatosi B, Elder KT. The health trade-off of rural residence for impaired older adults: Longer life, more impairment. The Journal of Rural Health. 2007;23(2):124–132. doi: 10.1111/j.1748-0361.2007.00079.x. [PubMed] [Cross Ref]
  • Laditka SB, Wolf DA. New methods for analyzing active life expectancy. Journal of Aging and Health. 1998;10(2):214–241. doi: 10.1177/089826439801000206. [Cross Ref]
  • Land KC, Guralnik JM, Blazer DG. Estimating increment-decrement life tables with multiple covariates from panel data: The case of active life expectancy. Demography. 1994;31(2):297–319. doi: 10.2307/2061887. [PubMed] [Cross Ref]
  • Lièvre A, Brouard N, Heathcote C. The estimation of health expectancies from cross-longitudinal surveys. Mathematical Population Studies. 2003;10(4):211–248. doi: 10.1080/713644739. [Cross Ref]
  • Lohr SL. Sampling: Design and Analysis. Pacific Grove, CA: Duxbury Press; 1999.
  • Lubitz J, Cai L, Kramarow E, Lentzner H. Health, life expectancy, and health care spending among the elderly. The New England Journal of Medicine. 2003;349(11):1048–1055. doi: 10.1056/NEJMsa020614. [PubMed] [Cross Ref]
  • Lynch SM, Brown JS. A new approach to estimating life tables with covariates and constructing interval estimates of life table quantities. Sociological Methodology. 2005;35(1):189–238. doi: 10.1111/j.0081-1750.2006.00167.x. [Cross Ref]
  • Palloni A. Demography of HIV/AIDS. Population Index. 1996;62(4):601–652. doi: 10.2307/3646371. [Cross Ref]
  • Quillian L. How long are exposures to poor neighborhoods? The long-term dynamics of entry and exit from poor neighborhoods. Population Research and Policy Review. 2003;22(3):221–249. doi: 10.1023/A:1026077008571. [Cross Ref]
  • Rao JNK. Variance estimation in sample surveys. In: Balakrishnan N, Rao CR, editors. Handbook of Statistics. Amsterdam: Elsevier Science Publishers; 1988. pp. 427–447.
  • Rao JNK, Wu CFJ. Resampling inference with complex survey data. Journal of the American Statistical Association. 1988;83(401):231–241. doi: 10.2307/2288945. [Cross Ref]
  • Saito Y, Crimmins EM, Hayward MD. Stability of estimates of active life expectancy using two methods of life table construction. Cahiers Québécois de Démographie. 1991;20(2):291–327.
  • Schoen R. Modeling Multigroup Populations. New York: Plenum Press; 1988.
  • Schoen R, Land KC. A general algorithm for estimating a Markov-generated increment-decrement life table with applications to marital-status patterns. Journal of the American Statistical Association. 1979;74(368):761–776. doi: 10.2307/2286398. [PubMed] [Cross Ref]
  • Shao J, Tu D. The jackknife and bootstrap. New York, NY: Springer Verlag; 1995.
  • Sitter RR. Comparing three bootstrap methods for survey data. The Canadian Journal of Statistics - La Revue Canadienne De Statistique. 1992a;20(2):135–154. doi: 10.2307/3315464. [Cross Ref]
  • Sitter RR. A resampling procedure for complex survey data. Journal of the American Statistical Association. 1992b;87(419):755–765. doi: 10.2307/2290213. [Cross Ref]
  • Weden MM. Johns Hopkins University, Bloomberg School of Public Health; 2005. Multistate life table analysis: Promoting interdisciplinary public health research using demographic methods with Stata Statistical Software.
  • Wolf DA. Simulation methods for analyzing continuous-time event-history models. Sociological Methodology. 1986;16(10):283–308. doi: 10.2307/270926. [Cross Ref]
  • Wolf DA, Gill TM. Modeling transition rates using panel current-status data: How serious is the bias? Demography. 2009;46(2):371–386. doi: 10.1353/dem.0.0057. [PMC free article] [PubMed] [Cross Ref]
  • Wolf DA, Laditka SB, Laditka JN. Patterns of active life among older women: Differences within and between groups. Journal of Women and Aging. 2002;14(1–2):9–25. doi: 10.1300/J074v14n01_02. [PubMed] [Cross Ref]
  • Yang Z, Hall AG. The financial burden of overweight and obesity among elderly Americans: The dynamics of weight, longevity, and health care cost. Health Services Research. 2008;43(3):849–868. doi: 10.1111/j.1475-6773.2007.00801.x. [PMC free article] [PubMed] [Cross Ref]