PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of hsresearchLink to Publisher's site
 
Health Serv Res. 2003 August; 38(4): 1157–1176.
PMCID: PMC1360938

Measuring Continuity of Care for Clients of Public Mental Health Systems

Abstract

Objectives

The aims of this research were to generate a set of time-variant measures of continuity of outpatient care using administrative data, and to evaluate the validity of these measures for persons in the community with serious mental illness (SMI) who use public mental health services.

Data Sources

Individuals with SMI were identified using multistage random sampling from shelters, streets, and public mental health clinics in Houston, Texas.

Study Design

The study design was observational, cross-sectional, and retrospective. Based on a review of the literature, five distinct conceptual dimensions of continuity of care were defined: timeliness, intensity, comprehensiveness, stability, and coordination. Repeated measures of continuity were generated for each day of the year. Construct validity was assessed by comparing continuity for housed persons and homeless persons based on the assumption that homelessness is a risk factor for low continuity of outpatient care.

Data Collection

Subjects were interviewed to collect sociodemographic and clinical information. Service use was retrospectively tracked through the administrative records of multiple public sector agencies.

Principal Findings

All five continuity measures demonstrated good construct validity by the fact that homelessness was significantly (p<0.001) and substantially associated with lower continuity of care.

Discussion

The five continuity-of-care measures are relatively easy and inexpensive to generate using administrative data. The five continuity-of-care measures may be useful for identifying individuals at risk for poor outcomes and for evaluating the ability of public service systems to keep clients engaged in care over time.

Keywords: Continuity of care, homeless, serious mental illness

Individuals with chronic illnesses typically need good continuity of outpatient care to achieve and maintain desired clinical outcomes. Serious mental illness (SMI) is one such chronic illness for which continuity of care is likely to be especially critical (Ware et al. 1999; Bachrach 1981; 1993; Lehman et al. 1994). Most persons with SMI use public sector care, and because public sector services are often fragmented and complex, persons with SMI may have a particularly difficult time remaining continuously engaged in care. The value that public systems of care place on continuity is demonstrated by the high level of resources devoted to case management, an intervention intended to coordinate care and enhance treatment continuity (Bachrach 1981; Mueser et al. 1998; Lehman, Steinwachs, and Co-Investigators of the PORT Project 1998).

For the purposes of this paper, we conceptualized continuity of care as a characterization of an individual's outpatient service utilization pattern over time. We examined continuity of outpatient care specifically because discontinuity of outpatient care might place clients at risk for poor clinical outcomes as well as lead to the inappropriate and costly use of hospitals, emergency departments, and in the case of the SMI, jails or prisons. As with the provision of high-quality clinical services, continuity of outpatient care over time is an ideal to be strived for and achieved. For populations with chronic illnesses, time periods with sporadic or haphazard service system contacts provide critical information about the discontinuity of care that is not necessarily captured by measures of utilization and quality which focus solely on the services that are being provided.

Continuity is an aspect of outpatient care that readily lends itself to assessment using administrative data, yet surprisingly little work has been done to develop dimensional measures of continuity and to test the validity and clinical relevance of these measures. Based on a thorough review of the literature, we selected five conceptual dimensions of continuity of outpatient care that we believed are likely (1) to impact clinical outcomes, (2) to represent distinct dimensions of continuity, and (3) to lend themselves to routine assessment using administrative data (Bachrach 1981; Magill and Senf 1987; Tessler and Gamache 1994; Shern et al. 1994; Lehman et al. 1994; Johnson et al. 1997; Greenberg, Rosenheck, and Seibyl 2002). The conceptual dimensions are: (1) the timeliness of service use, (2) the intensity of services that are received, (3) the comprehensiveness of services that are received, (4) the stability of the relationship between the client and the provider team, and (5) the coordination of service provision. Realizing that these conceptual dimensions of continuity cannot be perfectly measured with administrative data, we subsequently identified five proxy measures of continuity: (1) the frequency of encounters, (2) the quantity of encounters, (3) variety of types of services received during encounters, (4) the locational consistency of encounters across facilities and (5) receipt of case management services.

A number of important conceptual dimensions of continuity have been reported in the literature, but were not used in this study. For example, the vast majority of prior continuity-of-care research has focused on the transition from inpatient to outpatient care (Tessler and Gamache 1994; Sytema, Micciolo, and Tansella 1997; Farrell, Koch, and Blank 1996; Tessler 1987; Lehman et al. 1994; Fortney et al. 1995; Johnson et al. 1997). However, because the present study focused on the continuity of outpatient care for community-based clients, we did not consider many of the commonly employed continuity measures such as the presence of a follow-up outpatient visit within 30 days of discharge. Other examples of dimensions of continuity that we did not use included (1) the level of communication of clinical information across services systems and providers (Bachrach 1981), (2) the degree to which a client's service use pattern matches their individualized treatment plan (Tessler, Willis, and Gubman 1986; Tessler 1987), and (3) the degree to which providers pinch hit for one another and troubleshoot clients' problems before they happen (Ware et al. 1999). While these may all be important conceptual dimensions of continuity, these dimensions cannot be measured using administrative data. In contrast, the continuity-of-care measures that we adopted can all be measured with administrative data, which are easily obtainable and provide a rich source of information about service use patterns over time.

The objective of this research was to evaluate these proposed measures of continuity of care. Because our objective was to create measures that captured different aspects of continuity rather than creating items designed to be combined into a single scale, content validity and internal consistency were not assessed. Rather, the analysis establishes the degree of correlation among the five measures to demonstrate that the measures are in fact capturing different dimensions of continuity. Construct validity was established using the method of extreme groups (Streiner and Norman 1989). Specifically, to test construct validity, we compared the continuity measures across two populations. One population (homeless persons with SMI) is expected to have low continuity of care, while the second population (housed persons with SMI) is expected to have high continuity of care. We hypothesized that if our continuity measure were valid, significant differences between these two groups would be found for each of the five proposed measures of continuity of care. Criterion validity, or more specifically predictive validity (Streiner and Norman 1989) will be established in future research efforts by examining the relationship between the proposed continuity-of-care measures and outcomes.

Methods

Rationale for Defining Time-Variant Continuity Measures

Continuity is a temporal concept (Bachrach 1981) and should be measured repeatedly over time. Although continuity of care should be assessed over a relatively long period of time to identify patients at risk for poor outcomes, the temporal units of measurement should be short enough to identify clinically relevant gaps in service use patterns. If the temporal units of measurement are too long, clinically relevant periods of discontinuity may be masked. Therefore, it is preferable to repeatedly measure continuity of care over relatively brief units of time and subsequently generate time-variant indicators. These time-variant measures of continuity can then be appropriately summarized to assess continuity of care over longer periods of time. For example, although the average level of continuity over time could be calculated easily, this measure would not distinguish between those patients with stable service use patterns and those patients oscillating between periods of high service use and low service use. A more relevant summary measure of continuity would be the minimum value of continuity over time, which represents the worst period of continuity. Another relevant summary measure of continuity would be the variation in continuity of care over long periods of time. However, to avoid the loss of information associated with summary measures of continuity, the time-variant measures of continuity can also be examined in disaggregate units of time. For example, with repeated measures of continuity of care over time, random effects models or generalized linear models can be used to identify risk factors for periods of discontinuity.

Proposed Continuity Measures

Using encounter dates, facility identifiers, and clinic codes available in administrative data records, repeated measures of continuity were generated for each day of the year. Frequency was defined as the number of days in the year since the last encounter with the mental health service system. Encounters occurring over a year ago were not included because they are unlikely to impact the mental health of individuals with SMI. Frequency measures the duration of temporal gaps in service use over the past year and can range from 0 to 365. The other daily measures of continuity of care were generated using a 90-day moving window to capture service use in the previous three months. A 90-day window was used to allow the continuity measures to vary over a relatively brief period and because persons with SMI should probably have at least one encounter every 90 days. Quantity was defined as the number of mental health encounters in the previous 90 days. Variety was defined as the number of different types of mental health services received in the previous 90 days. Consistency was defined using the “modified modified continuity index” developed in the primary care literature (Magill and Senf 1987). This index was originally intended to measure the degree to which primary care services are provided by the same physician over time. In the present context, the index measures the degree to which mental health services were provided at the same facility (however, provider, patient care team, or clinic would also be appropriate). Based on the number of facilities visited in the previous 90 days and the total number of visits in the previous 90 days, the formula for locational consistency is:

equation M1

The index ranges from 0 to 1. If there was only 1 encounter, the index equals 1. If there were no encounters, the index is set to 0. Case management is a dichotomous measure defined as receipt of case-management services in the previous 90 days.

Data Source

The proposed method of measuring continuity was applied to data collected from the “Homelessness and Public Sector Service Use” study funded by the National Institutes of Mental Health (NIMH) and conducted by RAND. The study was designed to examine issues of service use across a range of agencies, including physical health, mental health, criminal justice, and social service agencies. The study design was cross-sectional and retrospective, and involved a multistage random sample drawn in 1996 (January to November) from two populations in three sectors: homeless adults (with and without SMI) from the shelters and streets of Houston, Texas; and adults with SMI who were in treatment in Harris County (Houston) public mental health clinics, the great majority of whom were stably housed. Clinic sampling was conducted in two stages from six of the Harris County Mental Health/Mental Retardation Authority clinics. Days within clinics were sampled proportional to size based on a census of visitors to each clinic, and then visits were sampled on those days in such a way as to produce a self-weighting sample.

Shelter sampling was conducted in two stages after measures of shelter size were collected via a census of eligible shelters. In the first stage, nights across the interview period within all eligible shelters were sampled proportional to size. In the second stage, visits by homeless clients on those shelter-nights were sampled so that the product of the first- and second-stage sampling probabilities would be constant for all visits, thereby producing a “self-weighting” sample of shelter visits (i.e., each visit had an equal probability of being sampled).

Street sampling was conducted in three stages. After interviews with Houston police, police beat tracts were first sampled (in proportion to the estimated homeless population), then selected blocks within sampled tract-nights, and then all homeless persons on those blocks on those particular nights were approached. To increase the sample size, the probability street sample was supplemented with a purposive one (i.e., all homeless persons observed on contiguous blocks were approached). Because the probability sample and the purposive sample did not differ significantly in terms of key demographics, they were combined. The resulting sample was not self-weighting.

Analytical weights were constructed to combine the sample across the sectors, and adjusted to represent persons rather than visits and to take into account nonresponse. The analytic weight for each respondent is equivalent to an estimate of the number of persons represented by that respondent in the target population. All statistics and analyses were calculated using these weights. The number of cases reported was unweighted.

The total number of “visits” sampled across all three sectors was 1,453, which corresponded to 1,344 (92 percent) unique individuals because 84 people were sampled twice, 11 were sampled three times, and one was sampled four times. Of the 1,344 individuals, 39 (3 percent) were found to be ineligible when screened. Within each sector, the sample yield from the total of 1,305 eligible individuals sampled was as follows: 622 (86 percent) of 720 eligible individuals completed interviews in the shelter sector; 175 (63 percent) of 279 eligible in the street sector; 217 (71 percent) of 306 eligible in the clinic sector. The 291 individuals (22 percent of the 1,305 eligible) who did not complete interviews did not do so for a variety of reasons including refusal (43 percent of the 291) or too intoxicated (12 percent of the 291). Five of the respondents sampled via the clinic sector met our definition of homelessness, and were thus classified as homeless in the analysis below. The analysis presented here focused on the subsample of 261 subjects with a mental health disorder who reported living in Houston continuously during the previous two years and who had any contact with the public mental health system in the year before the interview. Of these, 59 (22.6 percent) reported being homeless some time during the 30 days prior to the interview, and 202 (77.4 percent) did not.

Primary Data Collection

Subjects were administered a structured face-to-face interview and compensated $25. From both homeless and clinic subjects, we collected sociodemographic and clinic information along with homelessness history and mobility outside of Houston. Mental health diagnoses were assigned for homeless subjects using a modified version of the Composite International Diagnostic Interview (CIDI)(Robins et al. 1988). The version used in this study was developed under sponsorship of the National Institutes of Mental Health (NIMH) in consultation with authors of both the University of Michigan version (UM-CIDI) (Wittchen et al. 1995) and the World Health Organization version (WHO-CIDI) (Andrews and Peters 1998). Recent (in the past year) alcohol and drug dependence were assessed with the UM-CIDI Short Form (Kessler and Mroczek 1995), which consists of screening scales developed from the National Comorbidity Survey (Kessler et al. 1994). Sections of the full CIDI were used to assign diagnoses of major affective disorder (depression and bipolar disorder) and schizophrenia. Among homeless persons, serious mental illness (SMI) was defined as meeting criteria for (1) lifetime schizophrenia, excluding those without any symptoms in the past three years, and (2) lifetime affective disorder, excluding those whose episode fell below CIDI severity criteria (e.g., did not see a doctor, did not take medication, did not experience a deterioration in functioning, and were not hospitalized), those who had a single episode only, those who had no episode in the last three years, those with grief only, and those with dysthymia only. This definition focused our analysis on persons with chronic and serious mental illnesses. For clinic subjects, we obtained the primary diagnosis from clinic administrative data. Health status was measured with several assessments. Subjects rated their general overall health from “excellent” to “poor” using a single item with a five-point Likert scale. Limitations in physical functioning were assessed using a ten-item scale drawn from the SF-36 (Ware Jr. and Sherbourne 1992). Instrumental functioning was measured using the Brief Instrumental Functioning Scale (BIFS) (Sullivan et al. 2001), which asks if subjects can perform a set of six activities on their own or if they require help.

Administrative Data Collection

For all of the subjects interviewed, we attempted to retrospectively track the use of outpatient mental health services across all relevant systems of care to ensure that continuity was measured from the perspective of the individual (who could be accessing multiple systems). Subjects were matched in the administrative databases using iterative procedures to repeatedly assess and recheck for appropriate matches using identifiers such as name and social security number. Utilization data was successfully obtained from the two relevant public systems of care: the Harris County Mental Health/Mental Retardation Authority and the Houston VA Medical Center. Although we captured information about the utilization of private sector mental health services from Texas Medicaid records, encounter dates were not available and, thus, the small amount of private sector service use could not be included in the continuity-of-care measures. Because service categories differ dramatically across the two public service systems, the variety continuity measure was generated using data only from the Mental Health and Mental Retardation Agency, which provided the vast majority of services in the sample. The following types of outpatient services were included in the variety continuity measure: mental health, psychosocial, crisis, brief intervention, decentralized intake services (i.e., evaluation), and other. Repeated measures of continuity were generated for each day of the year preceding the baseline interview (e.g., days −365, −187 and −1). Note that because frequency on day −365 requires information about service use in the year prior to that day, two years of administrated data were needed to calculate the continuity measures.

Analysis

To determine whether the various continuity-of-care measures capture different information about distinct dimensional concepts, Spearman rank correlation coefficients were calculated across the five measures for each of the 365 days. Spearman rank correlation coefficients were used because rank correlations do not require distributional assumptions and because they are relatively insensitive to outliers. The Spearman correlation coefficients were then averaged across the 365 days of the year to generate a summary measure of association between the continuity measures. If the correlation coefficients differ from unity, it indicates that the various measures capture different dimensions of continuity of care. Note that for persons who did not use any services in the previous 90 days, the values of the quantity, consistency, variety, and case management all converge to zero. Thus, for those days when a subject has no service use in the prior 90 days, quantity, variety, consistency, and case management are all perfectly correlated. Days without prior service use were included in the main analysis because not including them would result in an incomplete characterization of the service use pattern. However, as a sensitivity analysis, we calculated the correlation coefficients after deleting all days with no service use in the previous 90 days to provide information about the degree to which the various measures are capturing different dimensions of continuity for days with prior service use. A correlation matrix is presented with the coefficients based on all observations displayed below the diagonal and the coefficients based on the subset of observations with service use in the previous 90 days displayed above the diagonal.

To establish construct validity, we used the method of extreme groups (Streiner and Norman 1989). More specifically, we estimated the difference in continuity of care between housed and homeless persons with SMI who had contact with the public mental health system during the course of a year. These two extreme groups were chosen because they were expected to have very different levels of continuity of outpatient care due to the fact that homelessness is likely to be a critical factor contributing to discontinuity of care. To determine whether the continuity measures differentiated people in the housed and homeless groups, unadjusted comparisons of the continuity of care were first generated by calculating derived summary measures for each person over time. Specifically, for each person, the mean, maximum, minimum, standard deviation, and coefficient of variation of the continuity-of-care measures were calculated across the 365 days. T-tests were then used to compare the temporally averaged summary measures of each person across the housed and homeless groups. The comparison of temporally averaged continuity means across the housed and homeless group provides preliminary information about construct validity of the proposed measures. However, these temporal averages of continuity may mask time periods of poor continuity. Therefore, for each person, the maximum value of frequency and the minimum values of quantity, consistency, variety, and case management were calculated to identify the period of lowest continuity during the course of the year. The maximum and minimum values of continuity were then compared across the housed and homeless groups. Likewise, to examine the degree of temporal variation in the continuity-of-care measures over time, the coefficients of variations for each person were compared across the housed and homeless groups. The coefficient of variation is the percentage the standard deviation is of the mean and provides information about the degree to which continuity of care varies from day to day.

To avoid the problems associated with comparing temporally summarized continuity measures across the housed and homeless groups, the hypothesis that continuity of care was associated with housing status was tested using a repeated measures generalized linear modeling approach with a first-order autoregressive structure. The approach models the within-subject correlation in continuity across the 365 repeated continuity measures (one for each day of the year) over time. Depending on the statistical distribution of a particular continuity measure, the appropriate link function was used in the specification of the generalized linear model. Because the quantity and variety measures are counts, a Poisson distribution was used with a natural log link function. Because case management is a binary measure, a binomial distribution was used with a logit link function. Because the frequency measure represents the time between events, the exponential distribution (a special case of the gamma distribution) was used with an inverse link function. For consistency (an index ranging from 0 to 1), a normal distribution was used with the identity link function.

The variable of interest in the generalized linear models was housing status. Covariates were included in the model to control for differences in predisposing, enabling, and need factors between the housed and homeless samples. These covariates included age, male gender, race (Caucasian, African American, Hispanic, and other), education (years in school), any health insurance coverage (including eligibility for indigent care in Harris County), employed all or most of the time since age 18, overall health status, physical functioning, instrumental functioning, schizophrenia (compared to an affective disorder), substance abuse comorbidity, and time (e.g., day of the year prior to the interview).

The significance of the adjusted differences in continuity between housed and homeless persons is given by the p-value of the parameter estimate for housing status in the estimated generalized linear model. To facilitate the interpretation of the substantiveness of the housing status parameter estimates, adjusted continuity-of-care measures were generated by using the generalized linear model to predict values for a typical person in the sample (e.g., a hypothetical person with mean values for all the covariates). Predicted values were calculated for a hypothetical homeless person at three points in time (days −365, −187 and −1) and for a hypothetical housed person at the same three time points.

Results

For a selected person in the sample, Figure 1 displays the five continuity-of-care measures for each of the 365 days preceding the interview. Table 1 displays the temporally averaged Spearman rank correlation coefficients that measure the degree of association among the various continuity-of-care measures. The correlation coefficients below the diagonal (based on all observations) suggest a high degree of correlation between quantity, frequency, and variety. When days with no prior service use were dropped, the level of correlation among the continuity measures diminished dramatically, as evidenced by the coefficients above the diagonal. Note that the signs of four of the five correlation coefficients for consistency switched directions when no prior service use days were dropped, indicating that more service use is associated with lower locational consistency. These findings suggest that the various continuity-of-care measures are capturing different information about service use patterns for time periods when services are used, but not necessarily for time periods when services are not used. Despite the tendency of the continuity measures to be correlated during periods of no service use, continuity is still measured for all days because measuring continuity only during periods of service use would result in an incomplete characterization of the service use pattern.

Figure 1
Continuity-of-Care Measures for a Selected Person in Sample
Table 1
Spearman Rank Correlation Coefficients1 for the Continuity-of-Care Measures Averaged over Time

Table 2 displays the predisposing, enabling, and need factors for persons in the housed and homeless sample. There were significant differences between the housed and homeless groups. Compared to housed persons, homeless persons were more likely to be male (83 percent versus 49 percent), African American (59 percent versus 42 percent), to report being employed most of the time (63 percent versus 51 percent), had slightly better instrumental functioning (3.5 versus 3.3), were more likely to have a substance abuse comorbidity (34 percent versus 13 percent), and were somewhat less likely to have any health coverage (71 percent versus 77 percent). Housed and homeless persons in the sample did not differ with respect to age, education, physical functioning, schizophrenia diagnosis, and overall health.

Table 2
Characteristics of the Sample of Housed and Homeless Persons

Table 3 displays the derived summary measures for each person over time for the housed and homeless groups. T-tests comparing the temporally averaged continuity-of-care measures indicated that the SMI housed group had substantially and significantly (p<0.01) greater continuity of care than the SMI homeless group. Moreover, compared to the housed SMI group, the homeless SMI group had significantly (p<0.01) greater maximum values for frequency and significantly (p<0.01) smaller minimum values for quantity, variety, consistency, and coordination. In short, the periods of worst continuity were worse for homeless persons than for housed persons. The housed SMI group had significantly (p<0.01) different day-to-day variations than the SMI homeless group, but the direction was not consistent. The coefficients of variation for the homeless group were higher for quantity, variety, and consistency, but lower for frequency and coordination. For the homeless group, the relatively large coefficients of variation for quantity, consistency, and variety indicates that these measures of continuity varied substantially over time, which justifies our time-variant approach to measuring continuity.

Table 3
Unadjusted Summary Measures of Continuity of Care for Housed and Homeless Persons

Results from the generalized linear models also indicated that homelessness was significantly (p<0.001) associated with lower continuity of mental health care across all five measures. Table 4 displays the adjusted measures of continuity for a hypothetical housed person and the same hypothetical homeless person with SMI at three points in time (i.e., days −365, −187, and −1). Substantial differences in frequency, quantity, variety, consistency, and case management of mental health services are present at all three points in time. Other case-mix factors were also significantly related to continuity. Age was a significant predictor of lower quantity (p=0.02), but greater consistency (p=0.02). Comorbid substance abuse was a significant predictor of lower variety (p=0.02). Schizophrenia was a significant predictor of greater quantity of services (p=0.05) and greater variety of care (p=0.02). Time (e.g., days −365, −187 and −1) was a significant predictor of greater frequency (p<0.01), variety (p=0.01), coordination (p=0.02), and consistency (p<0.01). The observed positive relationship between time and continuity of care probably reflects the fact that most of the study participants were sampled from a clinic at baseline (e.g., day 0).

Table 4
Adjusted Time-Variant Measures of Continuity of Care for Housed and Homeless Persons

Discussion

Based on a thorough review of the literature, we identified five conceptually distinct dimensions of continuity of outpatient care that could be readily measured using administrative data. Using administrative records from two public mental health systems in Houston, Texas, we generated time-variant measures of continuity of outpatient mental health care for a sample of persons with a severe mental illness. The continuity measures were not found to be highly correlated with one another, especially when only observations with prior service use in the prior 90 days were examined. These findings suggest that the measures were capturing different dimensions of continuity of care.

To establish the construct validity of the proposed measures, both unadjusted temporally aggregated comparisons and case-mix adjusted temporally disaggregated comparisons of continuity were made across groups of housed persons and homeless persons. Unadjusted comparisons of temporally averaged measures of continuity of care identified substantial differences between housed and homeless persons across all five continuity measures. The differences in continuity among the housed and homeless groups were magnified when the periods of lowest continuity were compared. The results of the generalized linear model analyses, which compared the continuity of housed and homeless persons on each day of the year, also demonstrated that homeless individuals had lower continuity of care across all five measures compared to housed individuals. Compared to housed persons with SMI, the service use trajectories of homeless persons with SMI were characterized by a longer duration between encounters, a lower volume of encounters, fewer types of services received, a lower likelihood of receiving all services from the same facility and a lower likelihood of having a case manager. These results suggest that the measures of continuity have good construct validity as evidenced by the significant and substantial differences in these measures across two extreme groups of persons with SMI.

The continuity measures developed in this research can be conceptualized as performance measures for a health care system or as a characterization of an individual client's service use pattern over time. As such, these continuity measures can be used for two main purposes: (1) evaluating systems of care and (2) identifying individual clients at risk for poor health outcomes. If conceptualized as a performance measure, continuity-of-care measures averaged across clients can be used to evaluate health care systems. However, because continuity of care should be measured from the perspective of the individual and not from a particular service system (Bachrach 1981), it is recommended that information be obtained from all relevant sectors of care. However, before these proposed continuity measures should be implemented as performance measures, it will be necessary to determine their predictive validity (i.e., their ability to predict poor outcomes). It is expected that low continuity of outpatient care over time will place public mental health system clients at risk for encounters with other, less appropriate, elements of the service system such as hospitals and emergency rooms, as well as placing them at risk for encounters with the criminal justice system. As such, these continuity measures should be able to predict adverse events such as incarceration, hospitalization, and emergency room admissions. In theory, the time-varying nature of the proposed continuity measures should allow the sequencing of outpatient encounters and adverse events to be exploited in order to more precisely identify the cause-and-effect relationship between continuity and outcomes. Specifically, with time-variant measures of continuity and dates of adverse events, survival analysis or event history analysis can be used to estimate the impact of continuity on time until the next adverse event.

The proposed continuity measures have both strengths and limitations. A theoretical strength of these continuity measures is the time-variant nature that facilitates the identification of gaps in clients' service use patterns. A practical strength of these continuity measures is that they can be generated using encounter dates, facility/provider identifiers, and clinic codes that are readily available in administrative data records. An empirical strength of the five continuity measures is that they seem to measure conceptually different information about service use patterns and that they differentiate between the service use patterns of people who are housed and people who are homeless. This differentiation provides evidence of construct validity. A limitation of the continuity measures is that they do not capture many of the important conceptual dimensions of continuity that have been proposed in the literature. Moreover, the proposed measures are only proxies for the conceptual dimensions of continuity that were identified and may not accurately reflect the underlying concepts. Finally, these measures have not yet been assessed according to their predictive validity. In addition, future research is needed to identify thresholds for continuity below which clients may be at risk for poor outcomes.

There were also strengths and weaknesses with respect to the determination of construct validity (i.e., the ability of the continuity measures to differentiate between the service use patterns of housed persons and homeless persons). A strength of this investigation is that it combined both face-to-face interview data and administrative data collected from relevant public mental health service systems. The greatest limitation of the validation was that all of the housed persons were sampled from clinics, while nearly all of the homeless persons were sampled from the streets and shelters. However, all the study participants in the subsample analyzed were public mental health system clients (defined as having had at least one visit in the year before the baseline interview). Another limitation was that the two groups were assigned diagnoses using different methodologies (chart review versus CIDI). If the housed sample had greater unmeasured clinical severity than the homeless sample, the association between homelessness and continuity of care may have been overestimated, thus biasing our assessment of construct validity. The fact that clinical data (collected during the face-to-face interview) were used to control for case-mix differences alleviates this concern somewhat, as does the fact that the sampling weights for the clinic sample weighted infrequent users more than frequent users.

Acknowledgments

The authors would like to thank Chloe Bird, Audrey Burnam, Peter Mendel, Susan Ridgely, Suzanne Wenzel, and an anonymous reviewer for helpful advice and comments on earlier drafts of this manuscript.

Footnotes

This research was supported by the National Institute on Alcohol Abuse and Alcoholism (grant no. R01 AA12328 to Koegel and Burnam), the NIMH Center for Mental Healthcare Research (grant no. P50 MH48197 to G. Richard Smith), and the VA HSR&D Center for Mental Health Services and Outcomes Research (grant no. HFP 90-019 to Richard Owen).

References

  • Andrews G, Peters L. “The Psychometric Properties of the Composite International Diagnostic Interview.” Social Psychiatry and Psychiatric Epidemiology. 1998;33(2):80–8. [PubMed]
  • Bachrach LL. “Continuity of Care for Chronic Mental Patients: A Conceptual Analysis.” American Journal of Psychiatry. 1981;138(11):1449–56. [PubMed]
  • Bachrach LL. “Continuity of Care and Approaches to Case Management for Long-term Mentally Ill Patients.” Hospital and Community Psychiatry. 1993;44(5):465–8. [PubMed]
  • Farrell SP, Koch JR, Blank M. “Rural and Urban Differences in Continuity of Care after State Hospital Discharge.” Psychiatric Services. 1996;47(7):652–4. [PubMed]
  • Fortney JC, Booth BM, Blow FC, Bunn JY, Cook CAL. “The Effects of Travel Barriers and Age on the Utilization of Alcoholism Treatment Aftercare.” American Journal of Drug and Alcohol Abuse. 1995;21(3):391–406. [PubMed]
  • Greenberg GA, Rosenheck RA, Seibyl CL. “Continuity of Care and Clinical Effectiveness: Outcomes Following Residential Treatment for Severe Substance Abuse.” Medical Care. 2002;40(3):246–59. [PubMed]
  • Johnson S, Prosser D, Bindman J, Szmukler G. “Continuity of Care for the Severely Mentally Ill: Concepts and Measures.” Social Psychiatry and Psychiatric Epidemiology. 1997;32:137–42. [PubMed]
  • Kessler RC, McGonagle KA, Zhao S, Nelson CB, Hughes M, Eshleman S, Wittchen HU, Kendler KS. “Lifetime and 12-month Prevalence of DSM-III-R Psychiatric Disorders in the United States: Results from the National Comorbidity Survey.” Archives of General Psychiatry. 1994;51(1):8–19. [PubMed]
  • Kessler R, Mroczek D. 1995. “Scoring the UM-CIDI Short Forms.” Manuscript.
  • Lehman AF, Postrado LT, Roth D, McNary SW, Goldman HH. “Continuity of Care and Client Outcomes in the Robert Wood Johnson Foundation Program on Chronic Mental Illness.” Milbank Quarterly. 1994;72(1):105–22. [PubMed]
  • Lehman AF, Steinwachs DM. “Translating Research into Practice: The Schizophrenia Patient Outcomes Research Team (PORT) Treatment Recommendations.” Schizophrenia Bulletin. 10;24(1):1. [PubMed]
  • Magill M, Senf J. “A New Method for Measuring Continuity of Care in Family Practice Residencies.” Journal of Family Practice. 1987;24:165–8. [PubMed]
  • Mueser KT, Bond GR, Drake RE, Resnick SG. “Models of Community Care for Severe Mental Illness: A Review of Research on Case Management.” Schizophrenia Bulletin. 1998;24(1):37–74. [PubMed]
  • Robins LN, Wing J, Wittchen HU, Helzer JE, Babor TF, Burke J, Farmer A, Jablenski A, Pickens R, Regier DA, Sartorius N, Towle LH. “The Composite International Diagnostic Interview: An Epidemiologic Instrument Suitable for Use in Conjunction with Different Diagnostic Systems and in Different Cultures.” Archives of General Psychiatry. 1988;45:1069–77. [PubMed]
  • Shern DL, Wilson NZ, Coen AS, Patrick DC, Foster M, Bartsch DA, Demmler J. “Client Outcomes II: Longitudinal Client Data from the Colorado Treatment Outcome Study.” Milbank Quarterly. 1994;72(1):123–48. [PubMed]
  • Streiner DL, Norman GR. Health Measurement Scales: A Practical Guide to Their Development and Use. New York: Oxford University Press; 1989.
  • Sullivan G, Dumenci L, Burnam A, Koegel P. “Validation of the Brief Instrumental Functioning Scale in a Homeless Population.” Psychiatric Services. 2001;52(8):1097–9. [PubMed]
  • Sytema S, Micciolo R, Tansella M. “Continuity of Care for Patients with Schizophrenia and Related Disorders: A Comparative South-Verona and Groningen Case-register Study.” Psychological Medicine. 1997;27:1355–62. [PubMed]
  • Tessler RC. “Continuity of Care and Client Outcome.” Psychosocial Rehabilitation Journal. 1987;11(1):39–53.
  • Tessler R, Gamache G. “Continuity of Care, Residence, and Family Burden in Ohio.” Milbank Quarterly. 1994;72(1):149–69. [PubMed]
  • Tessler RC, Willis G, Gubman GD. “Defining and Measuring Continuity of Care.” Psychosocial Rehabilitation Journal. 1986;10(1):27–38.
  • Ware JE, Jr, Sherbourne CD. “The MOS 36-item Short-Form Health Survey (SF-36): I. Conceptual Framework and Item Selection.” Medical Care. 1992;30(6):473–83. [PubMed]
  • Ware N, Tugenberg T, Dickey B, McHorney CA. “An Ethnographic Study of the Meaning of Continuity of Care in Mental Health Services.” Psychiatric Services. 1999;50(3):395–400. [PubMed]
  • Wittchen H, Kessler RC, Zhao S, Abelson J. “Reliability and Clinical Validity of UM-CIDI DSM-III-R Generalized Anxiety Disorder.” Journal of Psychiatric Research. 1995;29(2):95–110. [PubMed]

Articles from Health Services Research are provided here courtesy of Health Research & Educational Trust