PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of hsresearchLink to Publisher's site
 
Health Serv Res. 2013 February; 48(1): 333–347.
Published online 2012 June 20. doi:  10.1111/j.1475-6773.2012.01436.x
PMCID: PMC3449006
NIHMSID: NIHMS379468

A High-Resolution Analysis of Process Improvement: Use of Quantile Regression for Wait Time

Dongseok Choi, Ph.D., Kim A Hoffman, Ph.D., Mi-Ok Kim, Ph.D., and Dennis McCarty, Ph.D.

Abstract

Objective

Apply quantile regression for a high-resolution analysis of changes in wait time to treatment and assess its applicability to quality improvement data compared with least-squares regression.

Data Source

Addiction treatment programs participating in the Network for the Improvement of Addiction Treatment.

Methods

We used quantile regression to estimate wait time changes at 5, 50, and 95 percent and compared the results with mean trends by least-squares regression.

Principal Findings

Quantile regression analysis found statistically significant changes in the 5 and 95 percent quantiles of wait time that were not identified using least-squares regression.

Conclusions

Quantile regression enabled estimating changes specific to different percentiles of the wait time distribution. It provided a high-resolution analysis that was more sensitive to changes in quantiles of the wait time distributions.

Keywords: Wait time, process improvement, quantile regression, Network for the Improvement of Addiction Treatment (NIATx)

Organizational process improvement often emphasizes reductions in variation; mean performance may remain unchanged if the effect is primarily a reduction in extreme values (high quantiles). Such data may be best analyzed by modeling the temporal regression relationship at the 0.95th quantile or 95th percentile (cases that exceed 95 percent of the distribution) controlling for the independent variables. A conventional linear regression analysis assumes the relationship observed for the mean holds at the different quantiles of the response variable. The estimated temporal mean trend basically determines the shape of temporal trends at individual quantiles. The resulting trend estimates cannot be necessarily true or close to the true temporal trend at a particular quantile of interest. Least-squares analysis is not valid when addressing a performance improvement goal stated as “95 percent of patients have less than 10 days between first contact and first treatment.” Quantile regression, on the other hand, estimates the 95th percentile directly and allows us to investigate the question without constraining the trend at 95 percent to be of a certain shape.

System-level studies of the performance measure “wait time to treatment” are rare in the U.S. health care system despite the importance of this measure. The need for systematically monitored and analyzed process measures is particularly pressing within the substance abuse treatment field (Garnick et al. 2002). Patients seeking treatment for alcohol and drug use disorders often encounter waiting lists and experience delays between the day they request care and the day they enter care. The delays reflect a fixed capacity for new patients and limited financing for expanding access to care (Hoffman and McCarty 2012). The organization and delivery of care, however, can also inhibit access to care and discourage patients from attending addiction treatment appointments (Stark, Campbell, and Brinkerhoff 1990; Hser et al. 1998; Ford et al. 2007).

NIATx was the first widespread application of process improvement techniques to substance abuse treatment (http://www.niatx.net) (Capoccia et al. 2007). Participating agencies were trained to use a simplified version of the Institute for Healthcare Improvement's hospital improvement support system (http://www.ihi.org). A cross-site evaluation of NIATx participants examined: (1) change in days between first contact and first treatment and (2) the percentage of clients that began treatment and completed the first four units of care. Thirteen agencies that began participation in NIATx in August 2003 reduced days to treatment 37 percent—from 19.6 to 12.4 days across all levels of care. Retention in care also improved; the proportion of clients who completed a first session of care and returned for a second and third session of care increased 18 percent between the first and second session (72–85 percent) and 17 percent between the first and third session of care (62–73 percent) (McCarty et al. 2007). A subsequent analysis of 14 outpatient and intensive outpatient treatment programs within the second NIATx cohort replicated the reduction in wait time and the improvement in retention and noted that the first cohort sustained the gains during a 20-month follow-up (Hoffman et al. 2008).

Previous analyses of wait time to treatment within the Network for the Improvement of Addiction Treatment (NIATx) employed a conventional least-squares regression analysis and estimated rates of change in the mean of the monthly averaged outcome variables (McCarty et al. 2007; Hoffman et al. 2008). Analysis of mean function is useful, but other aspects of the response distribution can also be of interest.

Patient wait times may have unequal variation due to complex interactions between variables or unobserved exogenous noises that are not accounted for in the regression model. Figure 1 shows distributions of patient wait time in a NIATx program. The distribution of all wait times across the 15-month study period appears in Figure 1a. Figure 1b presents wait time distributions for month 1 (first), month 8 (mid), and month 15 (last) and illustrates change in wait time distributions over time. This variation in the distributions implies that more than one single slope describes the temporal trends in wait time from the first contact to the first treatment. In the presence of such heterogeneity, conventional least-squares regression models may underestimate, overestimate, or fail to detect important changes occurring locally at a certain quantile of data, because it focuses on changes in the means (Terrell et al. 1996; Cade, Terrell, and Schroeder 1999). Figure 2 provides expository guidelines where quantile regression is desirable over the conventional least-squares regression analysis.

Figure 1
The Distributions of Wait Time to Treatment in Program 8. (A) Density Plot of All Wait Time. (B) Density Plot of Wait Time in the First (n = 39), Mid (n = 49) and Last Months (n = 33)
Figure 2
Guidelines How to Choose between Least-Squares Regression and Quantile Regression

Quantile regression (Koenker and Bassett 1978) extends the concept of percentile or quantile in the univariate analysis to regression and estimates regression relationships specific to a certain percentile of the response variable controlling for the independent variables. In its simplest application, quantile regression examines median regression and describes changes in the center of the distribution. As the median is robust to outliers in the univariate analysis, median regression is also robust to outliers. Median, however, is just one of the many quantiles that can be examined. A series of quantile regression for different percentiles estimates multiple rates of change, providing a more robust and detailed description of the regression relationships that can be overlooked by other regression methods. Quantile regression is well established and available in common statistical software (e.g., proc quantreg procedure in SAS; quantreg package in R). Applications include economics (Hendricks and Koenker 1992; Buchinsky 1994; Manning, Blunmberg, and Moutlton 1995; Poterba and Rueben 1995); biology, especially for growth curve and duration models (Cade, Terrell, and Schroeder 1999; Koenker and Geling 2001); medicine, especially for reference charts (Cole 1988; Cole and Green 1992; Wei et al. 2006); environmental modeling (Pandey and Nguyen 1999); and infrastructure studies (He, Simpson, and Wang 2000). More examples are available in Koenker (2005).

Methods

This analysis uses data from outpatient and intensive outpatient treatment programs from the two cohorts of data collection—cohort 1 (n = 9; October 2003 to December 2004) and cohort 2 (n = 14; January 2005 to March 2006). The analysis was limited to a total of 23 outpatient (OP) or intensive outpatient (IOP) treatment programs; four agencies provided both outpatient and intensive outpatient treatment services. Exclusions included sites that had less than 10 months of data or did not complete substantive process changes.

We used a cubic polynomial function to model temporal changes in the NIATx data. Specifically in the quantile regression, a regression quantile of interest assumes a form of

equation image
(1)

where y is the response variable (patient wait time from contact to the first treatment), t is the independent variable (the first contact time of a patient), and τ is the quantile of interest with 0 < τ < 1. β0(τ), β1(τ), β2(τ), β3(τ) are the regression coefficients at the specified quantile τ, respectively, corresponding to the intercept, linear, quadratic, and cubic terms of the patient contact time t. As in the least-squares analysis, nonsignificant β2(τ), β3(τ) estimates suggest a linear function is sufficient to model the regression quantile trend. We note that the coefficients are given as functions of τ, that is, not necessarily βj(τ1) = βj(τ2) for certain two quantiles 0 < τ1τ2 < 1. Hence, the regression relationship at one quantile, for example, median (τ1 = 0.5) is allowed to differ from the regression relationship at a different quantile, for example, 95th percentile (τ2 = 0.95).

The least-squares counterpart for equation (1) is

equation image
(2)

where the regression coefficients, β0, β1, β2, β3, are given as constants. In addition, the least-squares analysis assumes the regression relationship (2) in the conditional mean holds same across the distribution of y. That is, the τth regression quantile is assumed to take the form of

equation image
(3)

We note that only the intercept β0(τ) is supposed to change, whereas the coefficients β1, β2, β3 indicating the regression relationship between the patient wait time and patient contact time are held constant regardless of the quantile of interest. This means that the least-squares constrains all regression quantile curves to be parallel to the regression curve corresponding to the conditional mean with only the intercept allowed to change. The regression quantile in (3) with τ = 0.05 or τ = 0.95 corresponds to the lower or upper limit of a two-sided 90 percent prediction band. As both limits are constrained to be parallel to the conditional mean, the width of 95 percent prediction band is held same across time t, which is the constant variability assumption of the least squares. On the contrary, the quantile regression model (1) allows all the regression coefficients to change as functions of τ, and the width of 95 percent prediction band will not be held same across time t. Heterogeneity in the regression relationship is often exhibited as nonconstant variability across independent variables due to complex interactions between variables or unobserved exogenous noises that are not accounted for in the regression model. The least-squares method does not take into account such heterogeneity, whereas the quantile regression does.

All computations were done in R (R Development Core Team 2010). We used the lm function for the least-squares regression and the rq function (quantreg package) for quantile regression in R. The rq function estimates coefficients by minimizing ∑ρτ(YiXiβ), where ρτ (r) = τ max {r,0}+(1−τ) max {−r,0}, for τ quantile. Appendix 1 displays a sample set codes for analyzing data from a program.

Results

NIATx patient-level data on days waiting between first contact and first treatment session were analyzed using least-squares regression and quantile regression models. Figure 3 presents the distributions of wait time from three of the NIATx participants and illustrates the differences in four regression analyses: (1) least-squares regression of the mean with the 90 percent prediction bands, (2) quantile regression on the median, (3) quantile regression at 5 percent, and (4) quantile regression at 95 percent. In all three cases, the fitted conditional mean curve from least-squares regression model (LS) and the conditional median (50 percent) quantile curve (QR) were close to each other. The 5 and 95 percent quantile regression curves, however, show quite different temporal patterns from the corresponding prediction band curves provided by the least-squares regression and also from the median quantile regression. The lower and upper limits of the 90 percent prediction band by the least-squares method are parallel to the conditional mean curve, which is valid only if the variability in the data had remained constant across patient contact time. On the other hand, the 5 and 95 percent regression quantiles are not parallel, correctly reflecting the greater variability in the patient wait times shortly after the respective programs started participating in the NIATx as compared with 6 months post. See Table 1A for the relative percent changes. The majority of the fitted models for three intensive outpatient programs in Table 1A utilized a cubic polynomial function while a few utilized a simple linear function to model temporal trend patterns.

Figure 3
Quantile Regression Analysis of Wait Time of Three Intensive Outpatient Programs. Solid Lines Are Estimated 5, 50, and 95 percent Quantile Curves Using Quantile Regression (QR), and Dashed Lines Are the Estimated Mean Curve and 90 percent Prediction Bands ...
Table 1
Comparing Least-Squares Regression (LS) and Quantile Regression (QR) Analyses of Three Intensive Outpatient Programs in the Networks for the Improvement of Addiction Treatment. (A) Estimated Relative Changes in Wait Time (days) at Start and End of Data ...

It is notable that the 5 percent quantile fits are all above or at 0, whereas the lower limit of the prediction band by the least squares extends to the negative value. Wait time cannot be negative, and this artificial effect due to the least-squares method can be fixed by lifting the lower limit to zero as a post hoc fix in practice. In contrast, quantile regression naturally respects the range of the response data. We also note that the 5 percent quantile fits for (b) and (c) in Table 1A were horizontal lines at 0 without standard error estimates. This may be due to the substantial amount of cases with same-day admissions in both programs.

While there were relatively small or no changes at 5 percent quantile, the relative changes at 95 percent quantile were similar to those of corresponding quantile regression fit for median. In Figure 3b, the estimated 95 percent conditional quantile was 0.75 days at the end of the intervention. This can be interpreted as 95 percent of cases received their first treatment within 1 day, a remarkable improvement when compared with only about 50 percent admitted within 7 days or 95 percent within 44 days at the beginning of the intervention. The other two cases also showed substantial reductions in days to treatment. The reduction in wait time for 95 percent of patients for program (a) was 79 to 16 days, while wait time for program (c) declines from 16 to 13 days. Table 1B presents the corresponding coefficient estimates and their p-values for regression model and quantile regression model in Figure 3.

Table 2 presents the comparisons of the estimated changes using the least-squares regression and quantile regression models for all outpatient and intensive outpatient programs with sufficient cases in the NIATx data. Compared with the least-squares regression, quantile regression found more statistically significant changes over time in more addiction treatment centers. In particular, the statistically significant changes at 95 percent quantile were often detected when there were no significant changes detected at the mean or median level.

Table 2
The Estimated Wait Times at Start and End of Data Spans and Percent Changes by the Least-Squares Regression Model and Quantile Regression (QR) for All Outpatients and Intensive Outpatients Programs in Networks for the Improvement of Addiction Treatment. ...

Discussions

Quantile regression can provide high-resolution analyses in which the changes in the low or high quantiles can have different functional forms than the one in the mean values. Quantile regression directly models a series of response quantiles as functions of independent variables and can pinpoint quantiles where improvements occur. Our analysis found that evaluating changes in the mean values can miss important changes that occurred in the high quantiles, when interventional changes in fact focus on reduction of variation. Improvements in the high-quantile cases were detected with or without significant changes at the mean level. In addition, the trend pattern in the high-quantile cases tended to be more complex than and different from that of the mean level. These results illustrate the value of using quantile regression with the wait time to treatment measure and could potentially be of use with other process improvement and health service delivery measures.

Quantile regression techniques can be applied widely in the health care service field; for instance, estimating the effect of a health care reform on the frequency of individual counseling or doctor visits when the reform effect is potentially different in different quantiles of the outcome distribution. Quantile regression is a powerful method for studying such heterogeneous treatment effects. Within the health delivery service literature, there is discussion over approaches used to estimate the quality of health systems at various levels, from the level of the individual patient, organization or hospital, region, etc. Quantile regression offers a powerful tool to evaluate whether the effect of explanatory variables on the outcome of interest differs depending on where in the distribution the unit of analysis is located. Greater application of quantile regression may help health care systems identify subtle but meaningful improvements and increase their confidence in the effectiveness of change efforts.

Acknowledgments

Joint Acknowledgments/Disclosure Statement: Awards from the National Institute on Drug Abuse (R01 DA018282, R01 DA020832, R03 DA027946) and the National Cancer Institute (R03 CA133944) supported the analysis. Special thanks to our colleagues at the University of Wisconsin—David H. Gustafson, Jay Ford, Todd Molfenter, and Andy Quanbeck.

Disclosures: None.

Disclaimers: None

SUPPORTING INFORMATION

Additional supporting information may be found in the online version of this article:

Appendix SA1: Author Matrix.

Appendix S1: R Codes for Least-Squares and Quantile Regressions.

Please note: Wiley-Blackwell is not responsible for the content or functionality of any supporting materials supplied by the authors. Any queries (other than missing material) should be directed to the corresponding author for the article.

References

  • Buchinsky M. “Changes in the U.S. Wage Structure 1963–1984: Application of Quantile Regression” Econometrica. 1994;62:405–58.
  • Cade B, Terrell J, Schroeder R. “Estimating Effects of Limiting Factors with Regression Quantiles” Ecology. 1999;80:311–23.
  • Capoccia VA, Cotter F, Gustafson DH, Cassidy EF, Ford JH, Madden L, Owens BH, Farnum SO, McCarty D, Molfenter T. “Making “Stone Soup”: Improvements in Clinic Access and Retention in Addiction Treatment” Journal on Quality and Patient Safety. 2007;33(2):95–103. (9) [PubMed]
  • Cole T. “Fitting Smoothed Centile Curves to Reference Data” Journal of Royal Statistical Society A. 1988;151:385–418.
  • Cole TJ, Green PJ. “Smoothing Reference Centile Curves: The LMS Method and Penalized Likelihood” Statistics in Medicine. 1992;11(10):1305–19. [PubMed]
  • Ford JH, 2nd, Green CA, Hoffman KA, Wisdom JP, Riley KJ, Bergmann L, Molfenter T. “Process Improvement Needs in Substance Abuse Treatment: Admissions Walk-through Results” Journal of Substance Abuse Treatment. 2007;33(4):379–89. [PMC free article] [PubMed]
  • Garnick DW, Lee MT, Chalk M, Gastfriend D, Horgan CM, McCorry F, McLellan AT, Merrick EL. “Establishing the Feasibility of Performance Measures for Alcohol and Other Drugs” Journal of Substance Abuse Treatment. 2002;23(4):375–85. [PubMed]
  • He X, Simpson D, Wang G. “Breakdown Points of t-Type Regression Estimators” Biometrika. 2000;87(3):675–87.
  • Hendricks W, Koenker R. “Hierarchical Spline Models for Conditional Quantiles and the Demand for Electricity” Journal of the American Statistical Association. 1992;87:58–68.
  • Hoffman KA, McCarty D. “Improving the Quality of Addiction Treatment” In: Miller P, editor. Encyclopedia of Addictive Behaviors. Amsterdam: Elsevier; in press.
  • Hoffman KA, Choi JH, 2nd, Ford D, Gustafson DH, McCarty D. “Replication and Sustainability of Improved Access and Retention within the Network for the Improvement of Addiction Treatment” Drug and Alcohol Dependence. 2008;98(1–2):63–9. [PMC free article] [PubMed]
  • Hser YI, Maglione M, Polinsky ML, Anglin MD. “Predicting Drug Treatment Entry among Treatment-Seeking Individuals” Journal of Substance Abuse Treatment. 1998;15(3):213–20. [PubMed]
  • Koenker R. Quantile Regression. New York: Cambridge University Press; 2005.
  • Koenker R, Bassett G. “Regression Quantiles” Econometrica. 1978;46(1):33–50.
  • Koenker R, Geling O. “Reappraising Medfly Longevity” Journal of the American Statistical Association. 2001;96:458–68.
  • Manning W, Blunmberg L, Moutlton L. “The Demand for Alcohol: The Differential Response to Price” Journal of Health Economics. 1995;14(2):123–48. [PubMed]
  • McCarty D, Gustafson DH, Wisdom JP, Ford J, Choi D, Molfenter T, Capoccia V, Cotter F. “The Network for the Improvement of Addiction Treatment (NIATx): Enhancing Access and Retention” Drug and Alcohol Dependence. 2007;88(2–3):138–45. [PMC free article] [PubMed]
  • Pandey G, Nguyen V. “A Comparative Study of Regression Based Methods in Regional Flood Frequency Analysis” Journal of Hydrology. 1999;225:91–101.
  • Poterba J, Rueben K. “The Effect of Property Tax Limits on Wages and Employment in the Local Public Sector” American Economic Review. 1995;85:384–9.
  • R Development Core Team. 2010. “R: A language and Environment for Statistical Computing” [accessed on September 30, 2010]. Available at: http://www.R-project.org.
  • Stark MJ, Campbell BK, Brinkerhoff CV. ““Hello, May We Help You?” A Study of Attrition Prevention at the Time of the First Phone Contact with Substance-Abusing Clients” American Journal of Drug and Alcohol Abuse. 1990;16(1–2):67–76. [PubMed]
  • Terrell J, Cade B, Carpenter J, Thompson J. “Modeling Stream Fish Habitat Limitations from Wedged-Shaped Patterns of Variation in Standing Stock” Transactions of the American Fisheries Society. 1996;125(1):104–17.
  • Wei Y, Pere A, Koenker R, He X. “Quantile Regression Methods for Reference Growth Charts” Statistics in Medicine. 2006;25(8):1369–82. [PubMed]

Articles from Health Services Research are provided here courtesy of Health Research & Educational Trust