|Home | About | Journals | Submit | Contact Us | Français|
Improving patient compliance with physicians’ treatment or prescription recommendations is an important goal in medical practice. We examine the relationship between treatment progress and patient compliance. We hypothesize that patients balance expected benefits and costs during a treatment episode when deciding on compliance; a patient is more likely to comply if doing so results in an expected gain in health benefit. We use a unique data set of outpatient alcohol abuse treatment to identify a relationship between treatment progress and compliance. Treatment progress is measured by the clinician’s comments after each attended visit. Compliance is measured by a client attending a scheduled appointment, and continuing with treatment. We find that a patient who is making progress is less likely to drop out of treatment. We find no evidence that treatment progress raises the likelihood of a patient attending the next scheduled visit. Our results are robust to unobserved patient heterogeneity.
There has been an increase in the interest in patient compliance in the past two decades (Trostle 1997; Bloom 2001; Wosinska 2005).1 In general, compliance is defined as following or adhering to medical advice.2 Clinicians generally agree that patient compliance is an integral part of effective medical care, but the degree of compliance is low. Patients frequently do not take prescribed medicines, do not keep office appointments, do not follow through with treatment programs, and do not adjust lifestyles according to medical conditions. Begg (1984) reports 6–20% of patients do not even redeem their prescriptions. Smith and Yawn (1994) document that 19–28% of appointments are cancelled or missed, while Sellers et al. (1979) laments that 70% of clients in behavioral programs (such as substance abuse or diet control) fail to complete the programs. Noncompliance has been reported across many diseases.3
Failure to comply implies the absence of key inputs in health production (Keller et. al. 1982; Ellickson et al. 1999). In addition, noncompliance may necessitate more expensive treatment later.4 Noncompliance also may lead to medical errors because physicians may be misinformed about patients’ behaviors (see Melnikow and Kiefe 1994). The evidence suggests that lack of compliance leads to negative health outcomes and higher healthcare costs. In one study, noncompliance is claimed to lead to 125,000 premature deaths each year in the United States (Loden and Schooler 2000). The cost of noncompliance in the U.S. due to hospital re-admissions and lost productivity has been estimated at around $100 billion a year (National Pharmaceutical Council 1992; Johnson and Bootman 1995).
Clearly, understanding why patients do not comply is important. Many have viewed noncompliance as resulting from patients’ irrational behavior (Haynes 1979b; Trostle 1997). Increasingly, however, studies have turned attention to more objective factors, such as treatment complexity, side effects, and physician-patient interactions (Haynes 1979b; Conrad 1985). A more balanced approach regards patient compliance as the client’s decision in light of the benefits and costs of continued treatment.5
Which factors influence a patient’s compliance? Perhaps surprisingly, common demographic variables (e.g. gender income, age, etc) have not been linked to compliance (Royal Pharmaceutical Society of Great Britain 1998; Agar et al. 2005; Vik et al. 2004). On the other hand, treatment complexity, number of medications or duration of therapeutic regimens, and treatment cost-sharing have been found to be associated with compliance (Conrad 1985; Cramer et al. 1989; Miura et al. 2000; Dor et al. 2004). Other factors such as perceived side effects, perceived treatment benefits and effectiveness, as well as quality of patient-physician relationship have also been identified (Chan 1984; Adams and Howe 1993; Rietveld and Koomen 2002; Spire et al. 2002; Cherubini et al. 2003; Horne et al. 2004; Kennedy et al. 2004; Sloan et al. 2004; Vik et al. 2004; Aikens et al. 2005; Day et al. 2005; Garcia et al. 2005).
We hypothesize that if a patient perceives good progress and expects benefits, he is more likely to comply. This is a natural hypothesis from the standpoint of a patient’s costs and benefits. If a patient has been making good progress during a treatment episode, it seems reasonable to expect him to continue. To test this hypothesis, we study office visits for alcohol problems. Compliance is measured by keeping scheduled visits and continuing with treatment. Our progress variables are whether a client’s drinking problem has improved or whether there has been a relapse since the previous visit, as reported by clinicians and patients.
We use the intertemporal structure in our data to identify the causal effect of treatment progress on compliance. We use treatment progress in an on-going treatment episode to explain compliance in a future visit.6 This allows us to test whether good progress in the past predicts compliance in the future. As far as we know, this is the first attempt to draw a causal relationship between treatment progress and compliance in alcohol outpatient treatments.
We control for a number of patient covariates in our study. Substantial research, starting with Haynes (1979a, 1979b), demonstrates the importance of patient’s knowledge of therapeutic regimes, interactions between patients and doctors, as well as motivation. Other papers have stressed the importance of patients’ medical knowledge by comparing compliance between clients with and without educational training about therapeutic regimes (Wintraub et al. 1973; Brown et al. 1987; Seltzer 1980; Ley and Llewellyn 1995). Patient characteristics and previous experiences of alcohol abuse treatment will capture these effects. Finally, we control for unobserved heterogeneity of patients using random-effect, fixed-effect, and finite-mixture models (Heckman and Singer 1984; Cutler 1995).
Our results show that treatment progress affects patient compliance: a relapse in the previous visit increases the chance of dropping out of treatment, while making progress reduces it. On average, a relapse into drinking increases the chance of dropping out of a treatment program by about 9 percent, while making progress reduces it by 2.7 percent. These magnitudes are small but statistically significant. The results are robust when unobserved client heterogeneity is controlled for. Nevertheless, we do not find evidence that lack of progress or relapse in an earlier visit reduces the chance of missing the next scheduled visit for clients who stay in the program. Perhaps the decision regarding an upcoming visit is more likely subject to factors we do not observe, but the decision to remain in treatment is subject to systematic influence of progress in therapy.
The rest of the paper is organized as follows. Section 2 describes the data, defines measures of patient compliance and treatment progress, and presents summary statistics. Section 3 outlines the estimation strategy. Section 4 presents our main findings and robustness checks. We draw some conclusions and discuss future research in Section 5.
Our data come from alcohol abuse outpatient treatment programs in the state of Maine. There are two sources. The first is the administrative records from the Maine Addiction Treatment System (MATS). MATS was maintained by the Office of Substance Abuse (OSA), an executive agency of Maine.7 MATS collected information on clients enrolled in substance abuse programs that received funding from the federal government or the state of Maine between October 1, 1989 and June 30, 1995. Each client in the program was interviewed by a clinician or an assistant, and a standardized admission and discharge form was filled after the interview. If a client had not come for treatment for a long time, information from the clinical records of the client’s last visit would be used for filling in the discharge form. The admission form recorded a client’s demographics (age, race, sex, and education), living arrangements, household income, employment status, criminal involvement, history of substance abuse and treatment, as well as the frequency of alcohol use at admission. The discharge form recorded the provider and type of enrolled program (e.g.: inpatient or outpatient), the expected source of payment, the frequency of alcohol use at discharge, and the client’s termination status.8
The second data source is a set of medical record abstracts of one thousand MATS episodes. In the summer of 1996, researchers at Boston University collected the data under the supervision of OSA representatives. We selected MATS records of alcohol abuse episodes. Furthermore, we selected clients with medium to high alcohol usage (more than once per month), and being treated on an outpatient basis, without prior inpatient treatment within a year, and from ten largest agencies. We then randomly sampled one hundred episodes from each agency. Their clinical records were obtained directly from these agencies. Finally, these records were linked to the administrative records in MATS through a parallel scrambling algorithm to maintain confidentiality (more details can be found in Lu and Ma 2002). The analysis in the paper is based on the merged sample of about 1000 clients.
The medical record abstract data provide detailed information about each scheduled appointment in a treatment episode. Each client’s treatment record contains the dates of each scheduled visit, the title of the responsible clinician, whether the appointment has been kept, and the reason why a client fails to attend an appointment. In addition, the clinical records include the clinician’s judgment at each visit on a client’s treatment progress; this gives information on the client’s improvement since the previous visit. Reports of client’s progress are constructed from the medical abstracts, and play a key role in our analysis.
In our study, outpatient treatments for alcohol abuse are individual or group counseling visits. Patient compliance is defined as a patient keeping a scheduled appointment. A patient fails to comply if a scheduled appointment is not kept. Some patients may return to the program after having failed to keep an appointment, but some simply drop out of the program altogether. In fact, about 75% of all MATS episodes ended with a patient prematurely exiting the program without the clinician’s approval. We therefore define two kinds of noncompliance: missing a visit, and dropping out of the treatment program, with the latter being a more serious form of noncompliance.
We use two measures of treatment progress, constructed from the clinicians’ assessment notes in the clinical chart. The first is a relapse indicator. For each attended visit, we check from the medical records whether the clinician has indicated if a client has had alcoholic drinks since the previous (attended) visit. The second is an abstinence progress indicator. For each attended visit, we check from the medical records whether the clinician has made any positive comment on the client’s goal of achieving abstinence. Here abstinence is defined as abstaining from alcohol.9
Relapse is widely used and a validated measure of alcohol treatment progress while abstinence is often regarded as the treatment goal (Friedmann et al. 1998; Messina et al. 2000). We have found that assessment reports in our data are consistent; there were very few records where a clinician reported the client making progress and having a relapse simultaneously. Furthermore, we have checked and verified that, in fact, the progress and relapse indicators are highly correlated with health outcomes at discharge, measured by abstinence at time of discharge.
Our definitions of relapse and abstinence progress are different from those in other studies, where they are defined in terms of a client maintaining abstinence for a certain period after treatment (Institute of Medicine 1990; Lu 1999). Our study uses a visit-by-visit perspective in a treatment episode, so we need to keep track of relapse and abstinence progress as treatment proceeds. Our intertemporal perspective focuses on the sequence of decisions affecting compliance.
Our data consist of administrative and clinical records of 988 treatment episodes (12 episodes being excluded due to sampling errors). For our analysis, we further delete some treatment episodes from the sample for various reasons. First, some clients may not have participated in treatment voluntarily, and hence we cannot be sure that their compliance decisions have been voluntary.10 Second, we exclude those visits that are cancelled by clinicians; these outcomes do not reflect patient noncompliance.11 Third, we focus on individual and group treatments, and exclude episodes of family and Alcohol Anonymous counseling.12 Finally, we eliminate episodes where clinicians have made no assessment for treatment progress in the entire episode. After these adjustments, we have a sample with 473 treatment episodes and 5749 scheduled counseling visits.
We have checked that our sample is representative of the full MATS data set. Table 1 lists the number of observations and the characteristics of MATS episodes after applying each selection criterion we have described. There are 23,644 outpatient alcohol abuse episodes in MATS. The number of episodes drops to 50% of the total when we consider only those with medium to high drinking frequency (more than once per month), less than 25% after we consider episodes from the ten largest agencies. Then one hundred episodes were randomly sampled from each of these ten agencies. Despite these reductions in sample sizes, neither patient demographics nor patient prior treatment experiences vary much in Table 1. The largest change is in the distribution of drinking frequency at admission because we have dropped the least severe patients.
Table 2 presents summary statistics of our sample. The majority of clients in our sample are male (68.7%) and single (76.5%). The average client is around 33 years old, with a high school diploma (11.6 years of schooling), and a monthly household income about $700. Of all 473 clients in our sample, only 28.8% work full time while most of the remaining do not work. About 33% of the clients have some legal involvement at admission, either on probation, on parole or waiting for trials. More than half have sought outpatient substance abuse care before, of which about 17% had treatment within a year. About 18% of the clients have recognized psychiatric comorbidity. Slightly less than a quarter of the clients’ treatment are paid for by the state agency (OSA); about a quarter have some third party insurance (“Third Party”); more than 30% cover their expenses through Medicare or Medicaid programs (“Medicaid”); the rest pay out of their own resources (“Self-Paid”).
Table 2 also lists the frequency of drinking at admission and discharge reported in MATS. It is categorized into three groups: less than once per month, less than once per day but more than once per month, and more than once per day. In the substance abuse literature (Heather et al. 2001), these drinking frequencies may reflect a client’s health or treatment outcome of a treatment episode. About 60% of clients are admitted with serious drinking problems, i.e., more than once per day. At discharge, roughly half of the clients are reported to be drinking less than once in a month; according to the definition in MATS, they have achieved abstinence. Still, only half of them improve after treatment (drinking less at discharge) while the other half remain unchanged.
The lower half of Table 2 presents appointment information. The average number of appointments is larger than the median; this is consistent with healthcare use being highly skewed to the right. About three quarters of appointments are for individual counseling while the remaining for group counseling. On average, a client has 12.1 appointments, of which 9.5 are attended and 2.6 missed. Moreover, 338 clients, or roughly 75% of clients, exit the programs prematurely.
Table 2 also displays the percentage of visits in which clinicians report the client making progress towards abstinence (a “progress report”) or relapse (“relapse report”). Among them, 30% of visits are progress reports and 6% are relapses. Because a progress and a relapse report are mutually exclusive (a client cannot make progress while experiencing a relapse), clinicians report neither in about 64% of visits.
Are these reports valid measures of clients’ treatment progress? Figures 1 and and22 present the proportion of progress and relapse reports for the first 25 visits. Clients who achieve abstinence at time of discharge on average are likely to have received more progress reports and fewer relapse reports than those who do not. Such patterns persist throughout scheduled visits. These figures indicate that clients who have more good reports and fewer bad reports have better health status at discharge, and lend support for the validity of these reports for measuring treatment progress. More empirical evidence on the validity of these reports is presented below.
Consider a client j in a course of alcohol abuse treatment, and has been scheduled for an outpatient visit. There are three outcomes: the patient decides to keep the appointment, the patient decides to miss the appointment but will continue with the treatment program, and the patient decides to drop out of the program altogether. We decide to model these three outcomes as arising from two decisions: first the patient decides whether to stay in the treatment program, and second, if he has chosen to stay in the program, he decides whether to keep the appointment. The following diagram illustrates these decisions.
Let the variable c=1,2,3 denote the events “drop out of program,” “miss appointment,” and “keep appointment,” respectively. Let denote the probability of event c for client j in visit t.
By a slight abuse of notation, we let denote the probability of client j dropping out of the program in visit t. The probability of the event is simply . Using a logit specification, we model the probability as the following:
where Xj is a vector of observed client characteristics at admission, including demographics (age, gender, and education), initial health status (drinking frequency), as well as primary payer status, and Δhjt is the health benefit for j if she attends the scheduled outpatient visit t. Since a client makes the compliance decision before knowing the exact treatment progress, it is the expected benefit (E(Δhjt)) that influences her compliance decision. The γD coefficient measures how much expected treatment progress affects compliance. Finally, are random errors.
Again, by a slight abuse of notation, we let be the probability that the patient decides to miss an appointment given that she has chosen to stay in the program. The probability of the event that a client misses an appointment is . Since factors may affect differently a client’s choice of attending a visit and continuing in the program, we let and be independent, and use a separate logit function:
where βM and γM are the coefficients for Xj and E(Δhjt), respectively; is the random logit error term (independent of ). If γM is negative, a client with greater expected treatment progress is more likely to comply and attend a scheduled visit.
A client may choose to attend a scheduled visit. The probability of this event is simply the product of the probabilities of staying in the program and not missing the visit: . Finally, there are J clients in the data, and client j has Tj appointments. The likelihood function is:
where equals 1 if client j drops out of the program in visit t, 0 otherwise; equals 1 if the visit is missed. The above function is estimated by maximum likelihood estimation.
Estimates of the basic model may be biased if there are unmeasured heterogeneities, which may be due to: (i) unmeasured clients’ severity; (ii) unobserved clients’ medical knowledge or treatment motivation; or (iii) unobserved interactions between clients and clinicians. For instance, individuals who suffer form more serious drinking problems are less likely to improve; they are also more likely to drop out of the treatment program or miss a scheduled visit. Similarly, clients who are more knowledgeable or better motivated may have higher chances to benefit from treatment, and comply with the clinicians’ recommended treatment.
The terms and are client specific errors, while and are logistic errors independent of individuals and visits.
We first account for client specific error using the conventional random-effect and fixed-effect models. Each model has its limitations, however. The random-effect model yields inconsistent estimates if the unobserved errors are correlated with the regressors. The fixed-effect model overcomes this problem. Nonetheless, in our sample a significant number of episodes have very few scheduled visits. Controlling for fixed effects in nonlinear estimations in this setting may introduce a small-sample bias (Hsiao 1996, Greene 2002).
We consider a third candidate -- the finite-mixture model -- to control for unobserved heterogeneity (Titterington et al. 1985; Heckman and Singer 1984). The finite-mixture approach approximates nonparametrically the distribution of the unobservables as a set of mass points and their corresponding probabilities. This approach has been applied in health economics models in recent years (Cutler 1995; Deb and Trivedi 1997, 2002; Deb and Holmes 2000; Atella et al. 2004; Conway and Deb 2005; Crawford and Shum 2005). For equation (1), for two mass points, we have:
where and denote two mass points, reflecting different types of clients. The differences between the two mass points are client heterogeneities which affect their probabilities of dropping out. Clients with strong motivation, for instance, have a smaller value of , implying a lower chance of dropping out. The type probabilities π1 and π2 indicate the proportion of the clients in the sample with mass points and respectively.
If there are k mass points for each equation, the likelihood function (3) becomes:
Since there are k mass points for each equation, the likelihood function is factored into the probabilities conditional on the combinations of and and the type probabilities associated with all combinations.
We use clinicians’ progress and relapse reports to proxy a patient’s expected health gain E(Δhit). Consider two consecutive visits, say, visit t and visit t+1. The patient’s experiences between these two visits allow him to form an expectation about his future benefit from treatment. During visit t+1, the clinician may ask the patient about alcohol consumption since visit t, which yields the relapse report in visit t+1. At that time, the clinician may ask patients his experiences and make a judgment on the client’s progress since visit t, and this yields the progress report in visit t+1.
Given that the relapse and progress reports obtained in visit t+1 are about the information between the effect of visit t, the relapse indicator at visit t, rt, is defined as follows. The value of rt is set to 1 if a relapse is reported in the medical record at visit t+1; it is set to 0 otherwise. Likewise, we let pt be the abstinence progress indicator at visit t. The value of pt equals 1 if at visit t+1 the clinician reports that the client has made progress towards abstinence; it is 0 otherwise. We have dated these indicators rt and pt so that they refer to the effect of treatment of visit t. The following diagram illustrates our scheme.
For the empirical analysis, we use the relapse and progress reports rt−2 and pt−2 as explanatory variables for compliance in visit t. We use these lagged measures to avoid an endogeneity bias. The progress and relapse reports rt−1 and pt−1 are available only if the client attends the scheduled visit t; otherwise, these two indicators are missing. Such a pattern of missing report variables is a source of bias since clients may choose noncompliance in anticipation of low treatment benefit. Using relapse and progress reports rt−2 and pt−2 avoids the bias. In the robustness checks, we consider different combinations of lagged progress and relapse indicators.
This section presents our estimation results. We begin by validating the progress and relapse reports for measuring treatment progress. We then present the estimation results with and without accounting for unobserved heterogeneity. Finally, we check whether our results are sensitive to sample selections or alternative measures of treatment progress.
To validate progress and relapse reports as explanatory variables, we find their correlation with health outcomes by estimating the following health production function:
where hjT is client j’s health outcome at discharge, and hjT equals one if a client abstains from drinking at discharge and zero otherwise. In substance abuse treatment, abstinence is considered the ultimate treatment goal (Hoffmann and Miller 1993), and we use it as a measure of the health outcome due to treatment. The vector Xj consists of clients’ characteristics: demographics, insurance sources, initial health status, and the number of attended visits. The vector ΔH′jT is the health gains since admission, measured by numbers of progress and relapse reports in the treatment episode. The random error is εjT, independent among clients. If relapse and progress reports are reliable measures of treatment success, clients receiving more progress or fewer relapse reports should more likely reach abstinence at discharge.
Table 3 presents the logit results on estimating the probability of achieving abstinence at discharge, with and without including progress and relapse indicators in the production function. When a client’s demographics, number of attended visits, drinking frequency at admission, and the interaction of year and agency dummies have been controlled for, estimates of progress and relapse reports are highly significant. After including the progress and relapse indicators in the production function, the likelihood value increases from −264.8 to −225.2. In addition, a client’s chance of abstinence increases if she has received more progress reports or fewer relapse reports during the treatment. These results suggest that progress and relapse reports are good predictors of treatment outcome, measured by abstinence at discharge.
Results in Table 3 also indicate that a client’s chance of achieving abstinence increases if she has lower drinking frequency at admission or attends more visits. Finally, none of the demographic variables, such as age, gender, martial status or legal status at admission, or previous treatment experience has coefficient significantly different from zero.
Table 4 presents estimates of our basic model of compliance, namely equations (1) and (2) of dropping out of the program and missing a scheduled visit conditional on staying in program, respectively; unobserved heterogeneity controls are omitted here. The estimation includes the interactions of year and agency dummies to control for practice variations over time and agencies. In addition, the standard errors in the model are corrected for the patient clustering and heteroskedasticity. More than a quarter of the sample has been dropped since lagged relapse and progress reports were unavailable due to missed visits, reducing the sample to 4198 observations.
According to estimates of Table 4, clients with a higher drinking frequency at admission or those without past legal involvements are more likely to drop out. Likewise, Medicaid or older clients are less likely to drop out. Having a prior treatment experience does not reduce the chance of dropping out, except when the prior treatment is within a year of the current episode. By contrast, only the coefficient of psychological problem is statistically different from zero in the conditional probability of missing the visit; all other coefficients are insignificant.
Our parameters of interest are treatment progress measures. The estimated coefficients of progress and relapse reports are both statistically significant in equation (1). Moreover, a client with a higher number of relapse reports has a higher chance of dropping out of the program; more progress reports decrease that chance. Due to the nonlinearity of the logit function, the estimated coefficients need to be further transformed for calculating their marginal effects. After the transformations, a relapse report in the previous visit on average increases the chance of dropping out of the program in the current visit by 9.0 percent; a progress report decreases that chance by 2.7 percent;13 both are statistically significant at 5%.
Calculating the marginal effects of treatment progress measures on missing a visit is more complicated. The probability of missing the appointment ( ) is the product of the probability of missing the visit conditional on staying ( ) and the probability of not dropping out ( ). The marginal effect of a relapse or progress report on is thus a weighted average of two partial effects: and .14 The overall effect of treatment progress measures depends on both partial effects. For instance, a relapse report on average increases the conditional probability of missing the visit by 2%, but raises the chance of dropping out by 9%. On net, receiving a relapse report increases the unconditional probability of missing the visit by 0.6%, though the effect is not statistically significant. Likewise, a progress report increases the chance of missing the visit by 1.1%, though the estimate is also insignificant.15
In Table 5 we display the results of three models adjusting for unobserved heterogeneity: random-effect, fixed-effect, and finite-mixture models. In the random-effect and fixed-effect models, equations (1) and (2) are separately estimated (the errors in these equations being assumed independent). Thus, their corresponding numbers of observation and likelihood values are reported separately. In the random-effect model, the estimated variances of client specific errors are significantly larger than zero, especially for equation (1). Accounting for unobserved heterogeneity is therefore important. Coefficients of progress and relapse reports of the random effects model are similar to the basic model: a client obtaining a relapse report increases his chance of dropping out, while a progress report reduces the chance. In addition, the coefficients of report variables on the conditional probability equation remain insignificant.
The number of observations in the fixed-effect model drops substantially relative to the random-effect model. The conditional logit model can only use episodes whose compliance responses have changed during the treatment program; episodes without changes in compliance behaviors have been dropped. Results of the fixed-effect model are largely consistent with other models, except that the coefficient of progress report for the drop out equation is no longer statistically significant. This may well be because the fixed-effect model employs only observations of clients who exit the program prematurely; clients who have completed the program are dropped in the conditional logit estimation because they stay in the program throughout the episodes
Table 5 presents estimated probabilities for finite mixture model. Although we have started with four patient types (two mass points for each of the two non-compliance behavior equations), the estimated probability of one combination is less than three percent, resulting in a very small decrease in the likelihood value when moving from four to three combinations. Therefore, we drop one combination and assume only three combinations of patient type.16 Coefficients of report indicators in the finite mixture model are similar to those in the random-effect model; that is, reports influence the chance of dropping out, but not the conditional probability of missing a visit. In addition, both the magnitude and level of significance of the progress report effect are weaker than the relapse measure. This could be because compared with relapse report progress report is relatively more subjective and therefore a less reliable measure of the treatment progress. However, coefficients of previous treatment are no longer significant in the finite mixture model. This provides some evidence that the finite-mixture model is able to capture some degree of client unobserved heterogeneity.
Table 6 illustrates patient compliance by listing predicted behavior for each type in the finite-mixture model. The first type (with higher value of both mass points), accounting for roughly half of the population, corresponds to clients with high probabilities to drop out of the program and miss a scheduled visit. Due to a high rate of drop out, about 5 percent of the Type 1 clients remain in the sample after ten visits. The second and third types of clients are those with a lower drop out rate. From Table 6, half of them still remain in the program after 25 visits. Compared with Type 3 clients, Type 2 clients are less likely to attend visits regularly. Finally, consistent with our expectations, we cannot identify the type that drops out of the program sooner but regularly attends the scheduled visit when they are still in treatment.
To examine the importance of unobserved heterogeneity, we can compare the fit of the basic and finite-mixture models. Figures 3 and and44 display the predicted probability of dropping out and missing a visit for the first twenty five scheduled visits. The finite-mixture model fits the actual dropping out probability better. The sharp decline in the chance of dropping out is largely due to the dropping out behavior of Type 1 clients. The chance of dropping out starts to stabilize after 10th visit; this is because the remaining clients are mostly Type 2 and 3 clients; By comparison, the basic model under-predicts the chance of dropping out in early visits but over-predicts it in later ones. Neither the basic model nor the finite-mixture model fits the chance of missing a visit well.
We conduct five sensitivity checks. We start by estimating the basic and finite-mixture models by reweighting each client. Then we perform four checks on unobserved heterogeneity using the finite-mixture model; they differ in terms of variables capturing treatment progress and sample size.
First, our data are sampled from MATS episodes systematically. Although we have demonstrated in Table 1 that the basic characteristics of our data are representative of MATS episodes, our sampling rules may lead to bias because our observations are not proportionally sampled from ten largest agencies. To examine the effect of disproportional sampling rule on the results, we reestimate the basic model and the finite-mixture model by reweighting each client according to the relative size of the admitted agency in MATS sample after sample selection.17 The estimated results are in Table 7 under “Model 1A” and “Model 1B.”
Second, we consider a nonlinear relationship between patient compliance and treatment progress.18 We categorize relapse and progress report variables according to whether they are obtained in the first five visits or later. Their results are presented in Table 7 under “Model 2.”
Third, we check our results using different proxies for expected treatment progress. Our basic model uses pt − 2 and rt − 2, reports obtained in visit t−1 as proxies for E(Δhit). For this check, we include pt − 3 and rt −3, in addition to pt − 2 and rt − 2. The estimated results are in Table 7 under “Model 3.”
In our basic and previous models, many observations have been dropped due to missing reports. To check whether the results are sensitive to these exclusions, in the fourth model we use the most recently available progress and relapse reports as proxies for E(Δhit); hence we do not have to drop observation due to missing visits and missing reports. The results are presented in Table 7 under “Model 4.”
Fifth, all our previous estimations treat each scheduled visit as one observation in the sample. Although we have accounted for observed client heterogeneity, we may have overemphasized longer episodes since they consist of more visits. To test whether our results are driven by longer episodes in the sample, we include only observations of the first thirty scheduled visits in every episode. The results are in Table 7 under “Model 5.”
Model 1A in Table 7 presents the estimates of progress and relapse using MATS weights on clients for the basic model. For the Dropping Out equation, the progress estimate changes from −0.405 to −0.391, while relapse changes from 0.905 to 0.992. These changes are from 0.063 to 0.076 and from 0.159 to 0.284, correspondingly, for the Missing equation. We think that these changes are modest. Model 1B in table 7 presents the estimates for the finite-mixture model using MATS sampling weights. Except for the relapse estimate in the Missing equation (which is insignificant at the 5% level for both sampling methods), the changes in the estimates between the two sampling methods are negligible. We conclude that our sampling rule does not affect the estimated results.
For Model 2, the coefficients of the progress and relapse indicators obtained in the first five visits are insignificant. Treatment progress in the first five visits does not generate any impact on patient compliance. Clients may need some time to learn about treatment progress. In Model 3, the coefficients of the reports obtained in the visit prior to the last are also insignificant. It is likely that pt − 3 and rt − 3 contain similar information as pt − 2 and rt − 2; the impact of expected treatment progress on compliance has already been picked up by pt − 2 and rt − 2. Results in Models 4 and 5 are largely consistent with our main findings from the basic specification. In summary, our sensitivity analysis results support the robustness of our main findings.
This paper examines the relationship between treatment progress and patient compliance. Treatment progress is an important factor that influences patients’ compliance decisions. We find support for the view that patient decisions about compliance are to some degree, “rational,” reflecting patients’ anticipated benefits from treatment. Our results indicate that experiencing a relapse since the previous visit increases the chance of dropping out of the treatment program; progress decreases that chance. These findings hold after adjusting for client unobserved heterogeneity. For clients who stay in the program, we find no significant evidence that relapse or progress affects the chance of missing a scheduled visit. It is not surprising that missing a scheduled visit is more idiosyncratic than dropping out of treatment altogether. Attendance at a particular visit is probably affected by the immediate circumstances, which are unobserved. This partly explains why some treatment programs adopt a more strong form of intervention (e.g. clients cannot leave the program unless a certain number of visits taken) to ensure clients regularly attend scheduled visits.
Our analyses have some limitations. First, we use the clinician’s reports on client relapse and progress as a proxy for treatment progress. As we have discussed earlier, these comments may be subjective. Our health production function estimation results show that these reports are significantly correlated with client’s health outcome at discharge, validating these reports as treatment progress measures. Nevertheless, the accuracy of our results still may be affected by the quality of these reports.
Next, there are limits in our sampling method. Even though we have shown that our results continue to hold even after accounting for the size of different agencies, we caution the readers that our results may not reflect the compliance behaviors of the average alcohol abuse clients, but clients with certain attributes, particularly those with higher alcohol usage.
Finally, our estimation uses three different methods to control for unobserved patient heterogeneity. Those methods mitigate the estimated bias arising from time-invariant unobserved client characteristics, but not the unobserved heterogeneity evolves over time. Time-variant heterogeneity may arise if a client’s “type” or the standard of treatment progress changes during the treatment episode. For example, for a long episode, a client may become more compliant because she develops a better relationship with her clinician or receives more supports from family members. Also, the standard of treatment progress on a patient may change during a treatment episode. This may be because the clinician changes his expectation on progress, or the patient is treated by different clinicians with different standards. In both cases, our estimates of report variables on patient compliance may be biased because a client’s unobserved characteristics do not remain unchanged over time.. To avoid this kind of bias, one might need to gather more detailed data, for example, obtaining a clinician’s comments on all clients to figure out if standards have changed over time the clinician’s standard over time. One may also consider a more complex estimation method that allows a client’s “type” to change according to the length of a treatment episode.
Our major contribution is to provide a framework to assess the casual relationship between treatment progress and patient compliance. In addition, the finite-mixture model allows us to examine the importance of unobserved heterogeneity in determining a client’s compliance decision. While our findings may be specific to alcohol abuse treatment, our methods can be generalized to other health services, such as chronic illness treatments and psychotherapy. Furthermore, our study has demonstrated the potential value of collecting information about treatment programs and about services accepted and declined by patient.
1More than six thousand citations since 1980 were found in Medline related to compliance (Bloom 2001).
2The terms compliance and adherence are often used interchangeably in the literature. For detailed discussions on the historical use of these two terms, see Hughes (1997).
3Noncompliance has been reported in about 36% of individuals with hypertension (Dunbar-Jacob et al. 2000), 18% to 70% in depression treatment (Engstrom 1991), 40%–50% for clients with schizophrenia (Curson et al. 1985; Buchanan 1992), and 15% to 43% among patients with organ transplants (Didlake et al. 1988; Schweizer et al. 1990).
4Collins (1990) indicates that “compliance….might reduce stroke risks by about one half and coronary heart disease by about one fifth within a few years”. Ghali et al. (1988) found that more than one third of hospital re-admissions for heart failures are due to noncompliance with dietary and medication regimens.
5For instance, the health belief model stresses that a patient’s compliance is determined by beliefs about treatment costs (both monetary and psychological), severity of illness, and health benefit in the future (Jank and Becker 1984; Hughes et al. 1997).
6Our identification strategy is similar to “Granger-causality” in that we use past treatment progress to predict future patient compliance.
7The Department of Human Service was the responsible agency prior to the creation of OSA. OSA was created in July 1990 as a branch of the State’s Executive Department. After July 1, 1996, OSA was transferred to the Department of Mental Health, Mental Retardation, and Substance Abuse Service. OSA was responsible for allocating state and federal funds for substance abuse, and for contracting with agencies that provided substance abuse services.
9A client is regarded as making progress, for instance, if the clinician wrote “the client began to control his alcohol problem” or “the client showed some improvements.”
10We exclude 295 clients who were either (1) enrolled in Driver’s Education and Evaluation Program (DEEP), (2) currently in jail, (3) referred to other agencies or programs (e.g. inpatient service), or (4) deceased or moved away. DEEP are programs to prevent future offenses caused by drivers with problems of substance abuse; enrollees in these programs may be required to attend a certain number of visits in order to reinstate their licenses. We drop 89 clients from two agencies in which clients cannot leave the program unless certain visits are attended. Moreover, 86 clients are dropped because of ambiguity in their completion status.
11There were only 79 scheduled visits cancelled by clinicians.
12We exclude sessions of Alcohol Anonymous counseling because these were often scheduled before individual counseling. We exclude family counseling sessions because all of these appointments were kept. We suspect a clinician will not arrange the family counseling unless the client and his family members will definitely attend the session. A total of 557 sessions were deleted.
13Because we control for the interactions between year and agency dummy, the predicted probability of dropping out due to relapse or progress reports varies with respect to admission agency and year. The reported probability is the average of marginal effects calculated at each admission agency and year.
14The unconditional marginal effect of missing the visit due to factor x is: PM (1 − PD)/x = (1 − PD)(PM/x) −PM (PD/x)
15The standard deviation for the relapse report and the progress report for missing the visit is 0.025 and 0.013 respectively. They are calculated using the Delta method.
16The likelihood value of the finite mixture model drops significantly from 2 combinations (two types for drop out equation) to 3 combinations (from 2548.2 to 2534.4), but reduces very slowly from 3 to 4 combinations (the likelihood value difference is only 0.2). Therefore, we cannot reject the null hypothesis that there is no statistical difference by moving from 3 to 4 combinations (Chi (1, 0.2)=0.35) and our choice of number of mass points is justified. For details on the test, see Mroz (1999).
17In addition to the sampling criteria listed in Table 1, we have also restricted the sample in several different ways. Therefore, each agency’s weight is calculated based on the remaining episodes of every agency after these selection criteria have been applied.
18For instance, the health belief model suggests that progress at different stages of treatment produce different impacts on patient compliance.
*This research received support from R21-AA12886 from the National Institute on Alcohol Abuse and Alcoholism, AHRQ-P01-HS10803, and NIMH-R01- MH068260. Financial support for Lien (National Science Council: NSC-94-2415-004 and National Health Research Institute: NHRI-EX94-9204PP) and Lu (Alberta Heritage Foundation for Medical Research and the Institute of Health Economics) is also greatly appreciated. We are grateful to the Maine Office of Substance Abuse for cooperation in providing the data for the study. We thank Ivan Fernandez-Val, Kevin Lang, Richard Frank, seminar participants at Boston University, the University of New South Wales, and the Centre for Health Economics Research and Evaluation (CHERE) at University of Technology, Sydney for their valuable comments and suggestions. We thank editor Will Manning and two referees for their advice. The authors alone are responsible for the analysis and conclusions.
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
Hsien-Ming Lien, Department of Public Finance, National Cheng-Chi University.
Mingshan Lu, Department of Economics, University of Calgary.
Ching-To Albert Ma, Department of Economics, Boston University and Universidad Carlos III de Madrid.
Thomas G. McGuire, Department of Health Care Policy, Harvard Medical School.