|Home | About | Journals | Submit | Contact Us | Français|
Researchers have questioned whether the addictions treatment infrastructure will be able to deliver high quality care to the large numbers of people in need. In this context, the Robert Wood Johnson Foundation (RWJF) and Center for Substance Abuse Treatment (CSAT) created a nationwide network to improve access and retention in treatment. Applicant agencies described results of an admissions process “walk-through.” This qualitative study used narrative text from 327 applications to RWJF, focusing on admissions-related problems. We developed and applied a coding scheme, then extracted themes from code-derived text. Primary themes described problems reported during treatment admissions: poor staff engagement with clients, burdensome procedures and processes, difficulties addressing the clients' complex lives and needs, and infrastructure problems. Sub-themes elucidated specific process-related problems. Though findings from our analyses are descriptive and exploratory, they suggest the value of walk-through exercises for program assessment and program-level factors that may affect treatment access and retention.
Substance abuse problems are a major source of dysfunction and reduced quality of life, and have high economic, medical, family, and social costs (Donnermeyer, 1997; Mark, Woody, Juday, & Kleber, 2001; Office of National Drug Control Policy, 2004; Rosenheck & Kosten, 2001; Substance Abuse and Mental Health Services Administration [SAMHSA], 2004). Despite these negative consequences, in 2005 only about 2.3 million of an estimated 23.2 million Americans with substance-related problems received some form of treatment, and many who sought help were unable to obtain treatment (SAMHSA, 2006). In addition, only about 51% of those who enter care complete treatment (SAMHSA, 2002). These findings show that substance-related problems remain under-addressed in our health care system, and that significant barriers exist for those who need care.
Such systemic problems suggest that major changes are needed to improve access and delivery of substance abuse treatment, yet poor treatment results have often been blamed on patients with little attention paid to the way care is organized or delivered, or the extent to which available care meets patient needs (Broome, Simpson, & Joe, 1999). Recently, more attention has been focused on treatment facilities, their ability to engage and retain clients in treatment for prescribed lengths of time, and their ability to provide appropriate therapeutic services (Simpson, Joe, Rowan-Szal, & Greener, 1997). Moreover, McLellan, Carise, and Kleber (2003) have questioned whether the addictions service infrastructure, in its current form, can support adequate provision of high quality care.
In addition, a host of treatment access and other difficulties are compounded by problematic “business processes” that negatively affect individuals seeking care (Ebener & Kilmer, 2001). Further, these problems are exacerbated by regulatory requirements that add additional barriers through increased paperwork, assessment requirements, and financial screening (Martin, 2005; Soman, Brindis, & Dunn-Malhotra, 1996). Together, these barriers can hinder initial intake assessments, delay entry into treatment, and lead to missed treatment opportunities (Farabee, Leukefeld, & Hays, 1998; Hser, Maglione, Polinsky, & Anglin, 1998).
To address these problems, the Robert Wood Johnson Foundation (RWJF) and the Center for Substance Abuse Treatment (CSAT) created a nationwide effort to identify and address barriers to access and retention in addiction treatment. The RWJF Paths to Recovery initiative and CSAT Strengthening Treatment Access and Retention (STAR) program formed a nationwide learning collaborative called the Network for the Improvement of Addiction Treatment (NIATx) (Capoccia, Cotter, Gustafson, Cassidy, Ford, Madden, Owens, Farnum, McCarty, & Molfenter, 2007). NIATx provides collaborative learning opportunities and technical support to agencies so they can improve treatment access and retention. NIATx's central tenets are that patient-level outcomes are directly and indirectly affected by agency practices and policies, and by organizational influences (Heinrich & Fournier, 2005), and that process improvement can make organizational systems more consumer friendly, improving outcomes.
Substance abuse treatment agencies seeking participation in the RWJF Paths to Recovery program submitted six-page letters of intent that are the focus of the work presented here. As part of the application process, and as an introduction to process improvement, Paths to Recovery applicants were given instructions for completing an admissions “walk-through” exercise and asked to describe their agency's strengths and weaknesses in their letter of intent, basing their answers on the “walk-through” findings.
Walk-throughs are typically conducted by an employee of an organization, assuming the role of a prospective client and interacting with the organization as would a client. Such walk-throughs enable organizations to better understand their clients' points of view; can uncover assumptions, inconsistencies, and limitations of systems; and can generate ideas for improving organizational processes (Gustafson, 2004). Such patient-centered approaches have increasingly been called for to improve the quality of medical care (Institute of Medicine, 2001).
This paper examines agency and process information submitted as part of the first round of Paths to Recovery letters of intent, focusing on use of patient-centered walk-through exercises and describing potential barriers to treatment identified in the context of these exercises. Based on this work, we describe the organizational processes agencies identified as having the potential to impede access to care or affect treatment continuation.
The overarching goal of walk-through exercises is to identify problem practices and processes in order to improve service delivery and address customer needs by enabling providers to understand the experience of receiving care from the perspective of patients and their families (Gustafson, 2004). That is, a walk-through answers the question “What is it like to be our customer?” The specific objectives of the walk-through exercises reported here were to 1) identify potential service barriers and process problems experienced by clients attempting to obtain substance abuse treatment, and 2) to uncover methods for reducing or eliminating the problems identified.
Applicants were instructed to select two detail-oriented people, committed to enhancing customer service, to play the roles of “client” and “family member.” These individuals were to try to make the walk-through as realistic and informative as possible, and were instructed to choose a particular addiction problem to explore. For example, walk-through clients might present with the profile of a “typical” client or a client of a type the agency identifies as having special needs. Alternatively they might choose to represent a type of client that the agency is concerned about serving well. Applicants were told to let admissions staff know about the walk-through prior to conducting the exercise in order to ensure that staff did not feel undermined, examined surreptitiously, or tricked. Applicants were also reminded that admissions staff would likely be on their best behavior because they knew about the walk-through, and for this reason, that it was important for them to ask staff to treat the walk-through clients as they would anyone else.
Applicants were told to walk-through the admissions process just as a typical client and family member would experience it, starting with the customer's first contact and extending through the third outpatient visit, third day in inpatient care, or through a transfer between levels of care. During this time, walk-through clients were to try to think and feel as a client or family member would think and feel, to be sure to look around, and ask themselves “What might a customer be thinking? How might s/he feel?,” and to document those observations and feelings. Clients were also instructed to ask the staff, at each step in the process, about any changes they could think of that would improve the process for the client or for the staff working with the client, again documenting all ideas and feelings.
Following the exercise, both the walk-through client and family member were instructed to record a list of the needs and problems identified, and to write a brief report outlining the experience. In this report, they were to answer two questions: (1) What most surprised you during the walkthrough?, and (2) What two things would you most want to change?. In addition to these instructions, agencies were also provided with some additional suggestions, including a series of questions to help direct their focus toward key aspects of the admission process and to guide collection of appropriate and useful information (see Figure 1).
Paths to Recovery applicants were 327 addiction treatment agencies who submitted a letter of intent to RWJF in the spring of 2003. All applicants were required to have nonprofit status and to serve a client base that was at least 50% publicly funded. Letters of intent also included descriptive information about the agency and its setting (see Table 1), a report of the walk-through findings, and a description of areas for improvement the agencies identified as a primary focus for possible intervention.
We analyzed text from applicants' letters of intent, focusing on the walk-through exercise and agencies' self-identified strengths and weaknesses. All related text was coded and analyzed using Atlas.ti (Muhr, 1997). Atlas.ti is a software program that aids in the coding and retrieval of qualitative data. Electronic text files are attached to Atlas projects (Hermeneutic Units) and a coding scheme (developed by the researcher or research team) is either uploaded or entered directly into the Atlas.ti project. Trained coders then code by reading text on-line, selecting text that is consistent with a particular code or codes (based on code definitions created by the team), and then applying the appropriate code to the text (electronically, typically via dragging and dropping the code onto the selected text). When text has been coded, the Atlas query tool can be used to retrieve text associated with a particular code or a combination of codes (the latter using Boolean logic). Each query produces an electronic text file that includes all text meeting the criteria specified in the query. Queries identify each separately coded section with the filename of the document from which it was drawn and the line numbers of that document. This allows for easy return to the full document if additional information or clarification of the text is needed.
We used two primary sources of information for developing our coding scheme. First, we began with a list of preliminary descriptive codes derived from eight “promising practices & strategies” for process improvement in addiction treatment settings (Network for the Improvement of Addiction Treatment, 2006) that were developed based on a systematic literature review of relevant process improvement literature commissioned by the national program office for Pathways to Recovery (Network for the Improvement of Addiction Treatment, 2007). When the coding scheme was complete, we had included six descriptive categories derived (at least in part) from this review and the eight promising practices. These descriptive codes included: “initial contact,” “assessment/intake appointment,” “levels of care,” “engagement,” “paperwork,” and “family involvement.”
Because this study was exploratory in nature, we also used inductive methods for generating codes. Over a period of several weeks, six of the authors (JF, CG, JW, KR and LB) reviewed the same text from a subset of the letters of intent. For each application, each author independently created a set of descriptive categories or “codes” that reflected what s/he believed to reflect the information found in each application reviewed. The authors then met to discuss the text, line by line, creating a preliminary list of codes that identified common descriptive, pattern, or interpretive components (based on content, relevance, and prevalence) that occurred across the applications reviewed (Lofland & Lofland, 1995; Luborsky, 1994; Miles & Huberman, 1994). During this process we resolved disagreements though discussion, and created and refined the definitions for each code (Leininger, 1985). We then began to independently apply the preliminary coding scheme to text from several additional applications, jointly reviewing the text and associated codes line by line, discussing and resolving cases of disagreement, and finalizing codes and definitions (final n of codes = 39). When agreement across coders was consistently good, the coding scheme was finalized and all documents coded. To maintain consistency, coders were asked to code a new section of text each week for review and discussion during a weekly project meeting.
Once all text was coded, we used the Atlas.ti query tool to produce reports of text related to the admissions process. Themes and subthemes were identified by having two authors independently review all text for each of these queries and create a report of the themes they identified. The first three authors then reviewed, discussed, and consolidated the themes identified. To ensure validity, we also resolved discrepancies and looked for contradictory cases or examples in which the themes did not apply (Bernard, 1994; Miller & Crabtree, 1998; Spradley, 1980). The themes reported in the sections that follow were those that applied to the admissions process and appeared repeatedly in the data. Examples of contradictions are presented with the primary themes.
Despite heterogeneity among applicant agencies in treatment processes, philosophies, staffing, and other characteristics, we found four primary themes associated with agency barriers to admissions: poor staff engagement with clients, burdensome procedures and processes, difficulties addressing the complex lives and needs of clients, and problems with infrastructure. We describe these themes (as well as some sub-themes) and provide examples of themes from agency applications. When quotations or our interpretations refer to the experiences of clients, these “clients” are agency staff playing the role of clients as part of the walk-through exercise.
Applicants reported that they found staff members playing a vital role in framing the experience of the client, fostering open communication, and engendering trust, and that information provided by staff, and the ways in which it was provided, could affect client engagement. For example, some applicants reported that information was improperly disseminated or that clients received outdated or conflicting information from staff. Indeed, walk-through clients sometimes received different information and directions for the same treatment agency from different staff members. This misinformation was seen as affecting the client's ability to successfully navigate treatment admission processes, and possibly influence clients' ability to make informed decisions about treatment. In the sections that follow, we describe themes derived from text that related how staff communicated and interacted with clients.
Typically, the first contact of a prospective client or family member with the treatment agency was by telephone. One applicant reported that initial contacts have the potential to produce apprehension and uncertainty on the part of the caller, and some agencies were concerned that the success of the interaction during that first call might affect treatment entry. Various walk-through clients (and walk-through “family members”) described frustrations when their initial telephone calls were improperly handled:
Some walk-through clients mentioned that information provided to them about the treatment program would have been easier to understand and remember if the staff person had been following a well-written script or had clear guidance about information that needed to be reviewed.
As a result of these kinds of inconsistencies, some agencies reported that initial contact experiences could be quite different from client to client, even within the same agency. The absence of clear guidelines for initial contacts also was seen as resulting in incomplete information being provided to clients about treatment, particularly regarding paperwork requirements. Failures by staff to relay important information were identified in some agencies as having the potential to produce unexpected financial or other difficulties, and caused some clients to distrust the reliability of future information. For example, one client described not being instructed to bring required financial documents which would have necessitated an additional trip to the clinic had he been an actual client. Worse, some applicants discovered that staff disseminated incorrect information that could affect access to treatment. As an example, one agency found that walk-through clients were erroneously told that a school or family court would have to make a referral when that was not the case, and reported that the time associated with obtaining an external agency referral could increase the time between first contact and treatment admission.
Some applicants also identified improved telephone protocols as having the potential to address another problem they encountered—the assumption that the consumer knew about substance abuse treatment procedures and practices.
Other applicant agencies reported that their staff had provided complete and accurate information at the initial telephone call and beyond. These interactions included well-presented information about the agency, including funding and treatment options.
Although sharing information with clients is often considered a “patient-centered” practice, in some agencies the information provided at the assessment visit was noted as overwhelming and confusing, making comprehension difficult and potentially impairing the client's ability to make decisions about treatment options. Agencies discovered that simply being “informed” was not sufficient to support successful entry into treatment, particularly for those whose cognitive capacity was diminished as a result of their substance use or comorbid mental health problems. For example:
Another applicant noted that the walk-through clients “were given so much information they could not remember it all, and they ended up not being sure of when their appointment for orientation was scheduled.” As a result, clients and staff may end up with contradictory expectations about important aspects of care, such as length and intensity of treatment.
In addition to providing patients with information, the admissions process affords an opportunity for engaging the client and forming a therapeutic alliance. Some walk-through clients reported that their agency did not perform well with regard to encouraging client participation or making clients feel comfortable about entering treatment.
At the same time, attributes or qualities in staff members that enhance the admissions process can be elusive. One participant noted that she missed “the kind of personal attention that would be reassuring to a client.”
On the other hand, applicants reported many examples of staff performing well at recruiting, engaging, and retaining clients. In these instances, agency staff were often thorough, sympathetic, and caring in their interactions with clients. Such examples can provide useful information for agencies that want to improve these processes.
Walk-through clients identified various procedural problems that could create barriers to treatment and negatively affect client engagement. A common concern was the length of time required to successfully complete intake procedures. Other observations included frustration with the overwhelming amount of paperwork and forms, redundancies in paperwork, and the sometimes mechanical and routine feeling of the intake. Examples follow:
Among problems noted by applicants was the need for improving the speed—and therefore the method—by which they moved clients through the intake process. Most commonly, the length of time from the first point of contact to completion of the assessment was deemed unacceptable. One walk-through client, who described the process as “stressful and frustrating,” noted that a two-week wait to be admitted into the program made her feel “scared and devalued.” Long delays and burdensome processes between first contact and scheduling an assessment appointment were also identified as having potential to increase client no-show rates. The following provides one example of the kinds of burdensome processes applicants identified:
Walk-through clients also found themselves disappointed by the length of time it took to get an initial appointment—in some cases more than two months. Long waits were seen as fueling anxiety in already uncertain clients and family members, and delays were expected to affect whether or not a patient continued seeking treatment. The following examples were provided by agencies that treated adolescents:
Some applicants noted unacceptable time lags between the collection of client information, gaining an evaluation appointment, and progressing to the next level of care. One applicant stated that time delays in the intake process would “irritate and discourage the client” and increase the likelihood of treatment drop out. In some instances, the length of the intake process and access to services varied by level of care.
Walk-through clients who presented with substance abuse problems with comorbid mental illnesses voiced additional concerns. Some encountered long waits that were identified as particularly problematic for people with dual diagnoses, or for those who had needs for psychiatric medications that were not addressed. In the following example, the client's need for medication was addressed, but other delays and process problems were uncovered:
Applicants also expressed concerns over the repetition that occurred in the intake process and the intake paperwork. Duplication, such as multiple and repeat administration of assessments for placement, led to frustration among clients. Irritation also resulted when redundant information was gathering within a single visit, when information provided in prior visits was not used in subsequent visits, and when information collection was duplicated during transitions from one level of service to another. One applicant noted that redundant data collection during intake processes appeared designed to meet the needs of the agency rather than the needs of the client.
Several clients were also surprised and dissatisfied with the number of staff they were required to interact with during intakes, often answering duplicative questions.
In addition to inducing client fatigue, some applicants were concerned that redundancy and lengthy paperwork could interfere with the creation of a successful relationship with providers:
Redundancies in financial assessments and a focus on finances to the exclusion of client needs were also perceived negatively by some walk-through clients. In addition, treatment could be delayed while the agency worked to confirm insurance, and costs were seen as having the potential to impede treatment entry.
By the time people seek treatment, they are often in considerable distress and may be experiencing multiple emotional, medical, financial, housing, family, and social problems. A recurrent theme in the Paths walk-throughs concerned missed opportunities for addressing these issues, with several applicants noting that there was little if any support available to assist patients struggling with various complexities, including language barriers. One applicant noted that such problems could be compounded when walk-through family members (who often provide support to individuals with difficult lives), were excluded from planning and treatment. Language barriers between clients and staff were also reported as lengthening an already long intake process.
A number of walk-through clients who presented with comorbid mental health problems were turned away from treatment because dual-diagnosis programs were full, or more often, because substance abuse treatment programs did not provide mental health services while mental health programs did not provide addiction services. An adolescent treatment facility applicant expressed concern for the “youth who fall through the cracks each year,” unable to access the services they need for co-occurring disorders, indicating:
In the few agencies that provided integrated treatment, Paths to Recovery walk-through clients found that they had to “jump through several hoops” to receive care for both mental health and substance abuse. For example:
These barriers triggered additional steps in the intake process that could serve to create longer delays for clients with co-occurring disorders.
Mandated or court-referred walk-through clients who presented seeking treatment to resolve or avoid legal consequences were faced with additional barriers to treatment entry. These applicants reported that a lack of coordination between courts and agencies resulted in significant frustration, and that without appropriate linkages or mechanisms to share information, the client from the court system had to repeat paperwork at the treatment agency, adding time and tedium to the process of seeking treatment. Some mandated clients also reported concerns that they would face incarceration if the agency was not able to admit them quickly.
Individuals with substance abuse problems often have mobile, turbulent lives and may not have access to a consistent telephone for making or receiving calls. This applicant noted that access to a telephone could be been very important during the three-day delay to intake:
Walk-through clients also presented with transportation challenges that could interfere with their ability to reach the treatment agency and compromise their ability to seek treatment in a timely manner. Agency responses were sometimes found to be inadequate and other transportation options limited.
The lack of adequate transportation or parking, combined with limited appointment times and the need for childcare, was also perceived as affecting the timing of treatment entry.
For walk-through clients whose role included having small children, some found that inadequate access to childcare was a complete barrier to treatment participation.
Intake process barriers, such as repetitive paperwork, did not exist in isolation. Some applicants noted that poorly designed intake processes combined with other problems such as antiquated phone systems, crowded waiting rooms, “uncomfortable chairs,” and “unpleasant bathroom facilities” to create an environment where engagement between the client and the treatment agency could be disrupted. Applicants identified several types of infrastructure problems:
Applicants reported various problems with telephone systems. Walk-through clients often reached automated systems that dropped their calls or routed them to an individual who was not able to resolve their request. In other instances, the phone system was circuitous or made it impossible to reach a “live” person to request services or answer questions.
The process was especially frustrating when the system required the prospective client to know names or numbers of program personnel in order to be connected, as some applicants discovered. Applicants were also concerned that those without telephones would give up when faced with needing to leave a message for a return call, or about delays related to “playing telephone tag” when a “live” person was not available.
Not all treatment agencies had inefficient telephone systems, however, some walk-through clients reported encountering robust systems that routed them quickly and efficiently. A few discovered they were able to reach a “live” person “24/7.” Likewise, some agencies operated state-of-the-art call centers capable of quickly linking clients with the appropriate treatment within their agency. These instances provide positive examples for process improvements that might be implemented elsewhere.
Walk-through clients also noted a lack of privacy and other problems in their waiting rooms and intake facilities. Concerns included crowded waiting areas, lack of confidentiality in waiting areas (“all conversation at the front desk could be overheard”), and intake rooms that did not provide adequate privacy.
Because substance abuse treatment can be stigmatizing (Link, Struening, Rahav, Phelan, & Nuttbrock, 1997; Centers for Disease Control and Prevention, 2005), treatment entry can depend upon the patient perceiving an atmosphere of confidence and trust in which he or she is willing to divulge uncomfortable facts and issues that may be germane to treatment. In the absence of this perception of confidentiality, agencies were concerned that some individuals with substance abuse disorders might refuse or fail to seek treatment, going without needed services.
These qualitative analyses provide important information about admissions processes at substance abuse treatment agencies from the perspective of would-be consumers, and about the value of using walk-through exercises as a method for learning about consumer perspectives. Our findings highlight a series of problems that, if common across other agencies, could hamper efforts to successfully engage individuals in substance abuse treatment. These results also suggest that walk-through exercises provide valuable information that may assist agencies and providers in targeting and framing process improvements for the greatest impact.
The details of the exercises reported here also suggest processes that other agencies or clinicians may want to explore when they conduct walk-through exercises. For agencies that are already aware of problems in their admissions process, our findings provide information about possible opportunities and methods for improvement. For example, client-level complexities offer opportunities for improving the system broadly, perhaps by adding ancillary or wrap-around services or case management functions. Addressing telephone systems and procedures may provide a relatively easy avenue for improving client contacts throughout the treatment process. Other possible responses to the kinds of problems identified in our analyses include:
While such changes may seem difficult to implement, organizations now participating in the NIATx program have found that seemingly minor changes have the potential to improve consumer experiences in important ways. As one example, NIATx agencies have begun to adopt more “mindful,” client-centered approaches—those involving enhancing trust, caring, expertise, and an appropriate amount of autonomy. Such methods can help clients (who may otherwise feel devalued) experience a sense of receiving individualized and attentive care (Epstein, 1999). They can also help make intake processes easier to complete, more timely, confidential, rapport-building, and oriented toward the goal of engaging individuals in treatment, thus affording clients opportunities to develop relationships with program staff.
Finally, the richness of these data, which come from a large number of agency applicants adopting a consumer lens, suggests that clients themselves should also be considered a relevant resource for developing innovative solutions to addressing barriers to substance abuse treatment. In the absence of client assessments, or in combination with them, walk-throughs can help providers understand clients in ways that can produce the “deep knowledge of the underlying needs of the individual patients served…” that is needed to provide high-quality care (Berwick, 2005, p. 325).
The data presented here are from letters of intent submitted by agencies seeking funding. As such, the accounts of their walk-through exercises are necessarily streamlined and were likely presented in ways the agencies expected would appeal to the funding agency. The data are useful, however, in that they are, to our knowledge, the first presentation of a systematic exercise by a large number of substance abuse treatment agencies to understand consumer perspectives and identify problematic agency processes. However, even with the large number of agencies represented, the characteristics of these agencies may not be representative of the field in general. The extent and depth of the problems identified, however, provide indications that agencies were reasonably candid in their reports.
Our findings suggest that when admissions processes are burdensome, much can to be done so that such processes more closely reflect client, rather than agency, needs. Our findings also imply methods treatment agencies might adopt to create more accessible, efficient, and welcoming settings for clients. Although it is beyond the scope of this paper to discuss solutions undertaken by treatment agencies to address the barriers discovered from the walk-throughs, other research suggests that treatment agencies who offer walk-in appointments; use reminder phone calls; or help facilitate client linkages between levels of care (e.g., residential to outpatient) can reduce barriers to treatment, time required to enter treatment, and assessment “no-shows,” and also engage clients more successfully in the next level of care (Capoccia, Cotter, Gustafson, Cassidy, Ford, Madden, Owens, Farnum, McCarty, & Molfenter, 2007; Molfenter, Zetts, Dodd, Owens, Ford, & McCarty, 2005). Future research should consider the bi-directionality of the intake process and how the complex lives of clients affect this process. Clients themselves may also be an important source for identifying areas for future system improvements.
The Robert Wood Johnson Foundation and the Center for Substance Abuse Treatment support the Network for the Improvement of Addiction Treatment through grants and cooperative agreements. Awards to the University of Wisconsin (RWJF 48364; CSAT-SC-04-035) and Oregon Health & Science University (RWJF 46876 and 50165; PIC-STAR-SC-03-044) supported the preparation of this manuscript. The authors would like to thank Beth Hribar and Eldon Edmundson for their help with coding, Martha Swain for her help with editing, and Dennis McCarty for commenting on an earlier draft.
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.