Welfare workers at all welfare center offices in Bronx County screened welfare applicants for SUD. If applicants screened positive, workers assigned clients to receive further assessment at one of two assessment centers. One site provided coordinated care management (CCM) services and the other provided usual care (UC). The evaluation used a three-tiered sampling design. The Administrative Sample
contained 8986 applicants assigned to study sites during a two-year period and evaluated via welfare administration records documenting welfare activity, employment, and treatment utilization. The Screened Sample
contained 1519 representative participants from the Administrative Sample who were briefly assessed by research staff at baseline for current substance use, drug treatment history and motivation, and barriers to employment (see Treatment Assignment and Study Procedures
). Characteristics of the Screened Sample are presented elsewhere (Morgenstern, Hogue, Dasaro, Kuerbis, & Dauber, 2008
). The Interview Sample
contained a targeted subgroup of 421 participants from the Screened Sample who were selected for comprehensive follow-up interviews based on reporting a substance use problem and need for SUD treatment. The current study examines between-condition differences for the Interview Sample in treatment services received and substance use outcomes at one year after baseline (cf ).
CONSORT Diagram: Flow of participants through the study
Demographic, substance use, and other characteristics of the study sample (N = 421) are presented in . Participants were primarily men (66%) and either African American (49%) or Hispanic (43%). The sample averaged 39.6 years of age (SD = 8.5), and most were not married (91%). Fifty-five percent graduated high school or received an equivalency diploma. Severity and chronicity of substance use were high: at baseline the sample reported using alcohol or drugs on more than half the days of each month, and averaged 9.7 years of regular heavy alcohol use and 10.4 years of regular heroin or cocaine use. About 1 in 5 had unstable living conditions and almost half (46%) were involved in the criminal justice system. The vast majority (81%) had received public assistance prior to their application for benefits at baseline. We examined differences in demographics and substance use variables for the non-methadone versus methadone groups. Overall, clients in methadone treatment were significantly older, more likely to be Hispanic, and had greater chronicity and severity of drug problems, but lower chronicity and severity of alcohol problems. However, there were no significant differences at baseline in percent days substance use or prior treatment episodes. Thus, although clients differed in their use of alcohol versus drugs they appeared to have similar levels of severity and chronicity of substance use problems.
Demographics, substance use, and other issues by treatment: Coordinated Care Management (CCM), Usual Care (UC)
Due to interview time constraints at baseline and 1-month follow-up, data on current (within the last 12 months) DSM-IV psychiatric diagnoses (American Psychiatric Association, 1994
) were collected at 3-month follow-up for 315 participants (75% of the sample) using the Mini International Neuropsychiatric Interview
(Sheehan et al., 1998
). A total of 78% met criteria for any SUD: 59% met criteria for substance dependence and 8% for substance abuse (primary drug was cocaine for 55%, heroin 33%, and marijuana 11%); 36% met criteria for alcohol dependence and 10% for alcohol abuse. Among the 22% who did not meet diagnostic criteria for SUD, participants reported an average of 6.7 days (SD = 9.9) of drug use in the previous 30, 6.3 years (9.3) of lifetime regular heavy alcohol use, 9.0 years (9.2) of lifetime heroin or cocaine use, and 2.7 (2.6) prior episodes of alcohol and/or drug treatment. Across the entire interview sample, 23% met criteria for a depressive disorder, 22% for an anxiety disorder, and 25% for antisocial personality disorder.
A total of 119 clients were in drug-free AOD treatment only, 108 were in methadone treatment only or in combination with drug-free treatment, and 194 were not in treatment for SUD. It should be noted that welfare applicants who screened positive on the SUD screen, regardless of whether they were or were not already in treatment, were sent for an assessment. For applicants already in SUD treatment, the assessment focused on whether the current level of SUD treatment was appropriate and whether the applicant was either (1) exempt from welfare work activities because of treatment placement or (2) needed to attend work activities in addition to treatment.
Participant Flow, Selection Criteria, and Study Refusal Rates
The CONSORT diagram () depicts the flow of participants into the study. A total of 1685 applicants (19% of the Administrative Sample) were approached by research assistants at the two study sites in a manner designed to recruit a representative sample and asked to complete a brief study eligibility screen. Of these, 1519 consented to participate in the Screened Sample, whereas 166 (10%) refused for the following reasons: not interested (60%), too personal (14%), no time to spare (11%), and various other reasons (15%). Screen refusal rates did not significantly differ between conditions. Analyses of New York City welfare administration data found no significant differences between the Administrative Sample and the Screened Sample on any of over 20 demographic and welfare status variables.
Selection criteria for the Interview Sample (N = 421) were designed to include a subset of participants from the Screened Sample most likely to benefit from drug treatment services provided in CCM and UC. We sought to include persons with problematic substance use at baseline (only half of the Screened Sample reported alcohol or drug problems; see below) and to exclude persons with severe mental illness or homelessness. Selection criteria were: At least one day of illicit drug use or heavy drinking in past month, or, one day of illicit drug use or heavy drinking in past 6 months and currently motivated to attend treatment; Not hospitalized for mental health problems more than once in past year; Not currently experiencing psychotic symptoms or prescribed antipsychotic medication; Not residing on the streets, in shelters, or in imminent danger of being homeless; Not planning to move from the area for six months.
Of the 1519 persons in the Screened Sample, 976 (64%) were ineligible for the Interview Sample for the following reasons: no problem substance use day in past 6 months and/or not motivated to attend treatment (50%), currently homeless or in imminent danger of being homeless (16%), currently on antipsychotic medications (13%) or psychiatrically hospitalized two more times in past year (6%), or planning to move (3%). Also some were excluded due to assessor concerns about reliability of the respondent’s self-report (7%) or because the respondent was assigned to UC but reported recent involvement in CCM (5%). Note that among the 50% who were ineligible due to lack of current substance use and/or treatment motivation, 34% had been incarcerated in the prior year. Finally, 122 (8%) of the Screened Sample were eligible for the Interview Sample but refused to participate in the study, citing no time to spare (57%), not interested (23%), and other reasons (20%). Refusal rates did not differ between conditions.
Treatment Assignment and Study Assessment Procedures
All persons applying for public assistance in all Bronx county welfare intake centers during the two-year study enrollment period were administered a modified version of the CAGE screening questionnaire for substance involvement (Ewing, 1984
). Applicants who screened positive based on endorsing at least one item were assigned for further evaluation to one of two centralized assessment centers: one site housed the CCM program and the other housed the UC program. Assignment to condition occurred as follows: Welfare workers accessed an automated welfare management system via computer, which assigned the client to the next available assessment slot, either at CCM or UC. Slots turned over several times per week and clients were assigned regardless of geographical proximity to a center, client preference, or any other client characteristic. Welfare workers could not override the computer assignment. A check of administrative data during the three months prior to the start of data collection found no significant differences between the two sites for any of more than 20 demographic, welfare, or treatment-related variables, and no differences in show rates for assessment appointments. Thus, assignment to condition did not appear to be biased.
At each study site, research assistants approached welfare applicants and offered the opportunity to participate in a brief research interview. Applicants were assured that research data were strictly confidential and would not be shared with welfare officials of any kind. Those who signed informed consents were administered an eligibility screen in a private space; those eligible for the follow-up study were then asked to consent to a series of confidential interviews to be completed during the next year, and those who consented were administered a baseline interview. Participants were informed that a federal Certificate of Confidentiality from the National Institutes of Health was obtained to protect their confidentiality. Follow-up interviews were completed 1, 3, 6, and 12 months after baseline in research offices (81%), via mail (9%), by phone (6%), or in home (4%); there were no between-condition differences in type of interview. Participants received product vouchers after each interview. The study was conducted under approval by the governing Internal Review Board.
At both study sites the clinical assessment goals were to determine the need for substance abuse treatment and assign an appropriate level of care. For clients needing any level of treatment, site assessors immediately selected a pre-approved treatment program based on client preference and proximity, phoned the program to arrange an intake appointment, and provided referral information to the client. Intervention characteristics of the two treatment conditions are described below in .
Coordinated Care Management (CCM)
The CCM condition featured an innovative coordinated care management approach that focused on (1) coordinating services among multiple providers as a means to promote outcomes for individual clients and (2) assessment, referral, linkage, support, and monitoring client activity as in standard case management. CCM administrators and care managers (CMs) communicated directly with service providers about program features and service quality, the suitability of program activities for welfare clients, and program emphasis on sobriety and employability. CMs identified programs that matched specific client needs, withdrew clients from programs not serving their needs, and reduced referrals to underperforming programs. Each CM monitored program activities at 4-6 drug treatment sites via bimonthly site visits.
Assessment services were provided by a multidisciplinary team of psychiatrists, psychologists, licensed social workers, nurses, vocational rehabilitation specialists, and credentialed addiction counselors. Assessments lasted 2-3 hours and included structured diagnostic interviews, standardized health and mental health measures, employability questionnaires (e.g., work history, criminal history, family support, housing), substance use treatment motivation and stage of change measures, and a urine toxicology screen. Acute psychiatric cases were assessed by a psychiatrist for referral to a mental health clinic or emergency room. The multidisciplinary team made initial referrals to all indicated services, including drug treatment, work activity, medical and mental health care, domestic violence programs, housing, and childcare. Clients were then assigned to CMs for ongoing monitoring and service linkage.
CMs were stationed in the assessment site or a nearby field office and worked in specific catchment areas to foster cross-agency integration of available services. They maintained caseloads of 30-35 clients whom they contacted at least biweekly in office or by phone; they also visited each client at his/her drug treatment program every two weeks. Clients returned for re-evaluation with their CM every 90 days. During these visits clients were assessed for progress towards meeting treatment plan goals. SUD treatment type and intensity as well as other service linkages were modified accordingly. CMs attempted to match clients to suitable treatment programs and ancillary services. They consulted directly with drug program staff to monitor client progress and link clients to needed medical or mental health services, with welfare-sponsored job programs to help clients transition from treatment to work settings, and with other social systems (e.g., child welfare, criminal justice) to coordinate service plans.
Usual Care (UC)
The UC site was located in a city government building. Assessment services were provided by credentialed addiction counselors. Assessments lasted about 1 hour and focused on the nature and severity of substance use issues affecting employability. Mental health and physical disability issues were briefly evaluated; applicants with potential service needs in these areas completed a second assessment and referral session with a social worker on site the same day. Following assessment, clients were assigned to welfare eligibility workers (WEWs) stationed in field offices. WEWs maintained caseloads of 75-250 clients and made service referrals only during face-to-face meetings. They met with clients on an annual basis for recertification or whenever an emergency or non-compliance issue arose. WEW caseloads were not restricted to substance-using clients, and they did not receive training in managing substance-using populations. Clients were called back to the UC site every 90 days for a re-evaluation. The primary goal of this visit was to reassess whether clients who were exempt from work requirements because of the need for intensive SUD treatment should continue in this status or instead were ready for referral to a work activity.
To promote treatment integrity, an embargo was activated within the computerized appointment system at all Bronx welfare intake centers to ensure that applicants originally assigned to UC during the study period would again be assigned to UC (and not to CCM) if they re-applied for benefits up to three years after baseline. In addition, CCM administrators met monthly with welfare officials to review performance indicators related to coordinated care management activities and adherence to welfare regulations. Treatment fidelity was then assessed in two ways. First, a series of structured field interviews conducted with CCM administration and staff confirmed that behavioral assessment data were being incorporated into treatment planning activities at the assessment site and that CMs were receiving supervision from licensed staff, maintaining small caseloads, attempting regular contact with clients, and making monthly visits to drug programs in which their clients were enrolled. Second, in follow-up client interviews CCM clients reported more in-person contact with CMs during the prior 30 days than UC clients reported with WEWs at 1 month (CCM: M(SD) = 1.8(2.4); UC: M(SD) = 1.1(1.6); t(298) = 3.19, p < .01, Cohen’s d = .37), 3 months (CCM = 1.5(1.9); UC = .58(1.3); t(290) = 5.05, p < .001, d = .59) 6 months (CCM = 1.1(1.9); UC = .82(4.4); t(310) = .82, ns), and 12 months (CCM = .68(1.3); UC = .32(.80); t(324) = 3.12, p < .01, d = .35).
Demographics, substance use history, and other problems
Sex, ethnicity, age, marital status, education, welfare experience, and housing status were obtained via structured interview procedures. Information on substance use and criminal justice involvement was obtained using the Addiction Severity Index
(ASI, 5th edition: McLellan, Kushner, et al., 1992
Primary Outcome: Abstinence Rates
The primary outcome variable was abstinence from alcohol and drug use during each month over the one-year follow-up period. Abstinence rates were determined using self-report and biological measures. The Timeline Follow Back method (TLFB; Sobell & Sobell, 1996
) is a structured interview technique that evaluates quantity and frequency of substance use. The interview involves constructing a retrospective, daily record of the respondent’s substance use over a fixed interval, using calendars and critical marker events as memory aids. The technique has demonstrated good reliability and validity (Sobell, Brown, Leo, & Sobell, 1996
). The TLFB was also used to document periods of time during which respondents were in controlled environments (e.g., treatment centers, jails); abstinence rates were not calculated for these periods. Urine screens (Varian Inc., 2008
) were conducted for the presence of five drug metabolites: marijuana, cocaine, morphine, phencyclidine (PCP), and amphetamines. In addition, hair samples were analyzed for the same five metabolites using radioimmunoassay (RIA) tests followed by mass spectrometry confirmation tests (Psychemedics Corporation, 1991
). Hair analysis provides a much longer time record of drug use than other biological specimens and has low potential for evasion or manipulation of test results (Cone, 1997
). It has been used to verify self-report data for a variety of substances, including heroin (Romano, Barbera, Spadaro, & Valenti, 2003
) and cocaine (Tassiopoulos et al., 2004
Intermediate Outcome: Services Received
We used a modified version of the Treatment Services Review
(TSR; McLellan, Alterman, Cacciola, Metzger, & O’Brien, 1992
), a companion instrument to the ASI that yields data on the number of services received in various domains in the prior 30 days. The TSR has high test-retest reliability and correspondence with independent measures of services provided (McLellan, Alterman, et al, 1992
; McLellan et al., 1994
; McLellan et al., 1998
). The total number of various services received in each ASI domain were tallied: Medical (e.g., physical exam, prescription refill, meeting with Medicaid specialist), Legal (e.g., meeting with legal specialist, court appearance), Alcohol and Other Drug (AOD; e.g., treatment attendance, medication appointment, urine or breathalyzer test), Employment (e.g., job training, meeting with employment specialist), Housing (meeting with city housing agent or shelter caseworker), Mental Health (e.g., psychological testing, individual or group psychotherapy), and Basic Needs (meeting with specialist about childcare, food, clothing, etc.).
Follow-up Completion Rates and Collection of Timeline Abstinence Data
Of the 421 participants who completed baseline interviews, 27 (6%) did not provide data at any follow-up timepoint, leaving 394 participants (221 CCM, 173 UC) included in outcome analyses. Three persons died during the follow-up period; their missed interviews were not included in calculation of completion rates. Across treatment conditions, 87% of participants completed 1-month interviews, 71% 3-month, 75% 6-month, and 79% 12-month. The analyzed sample (n = 394) was compared to those lost to follow-up (n = 27) on demographic and baseline characteristics; no significant differences were found. Overall follow-up rates were 95% in CCM and 92% in UC and were not significantly different between conditions (χ² (1) = 2.41, ns). Chi-square tests were conducted to compare the drop-out subgroups from each condition to one another on baseline characteristics; because of the small size of the subgroups (n = 11 in CCM and 16 in UC), these analyses were considered exploratory. Results indicated that attrition characteristics were not differentially distributed across conditions.
Outcome analyses utilized abstinence data collected with timeline follow back procedures. Whenever a participant missed a 1-, 3-, or 6-month interview appointment, assessors extended the recall period at the next completed appointment to capture data from all prior months going back to the date of the last completed interview. Across treatment conditions, at 1 month, abstinence data were collected on 91% of the 421 participants, 84% of which derived from interviews completed at that timepoint (standard recall) and 16% from subsequent interview timepoints (extended recall). At 3 months, data were collected on 90% of participants, 80% of which were standard recall and 20% extended recalled. At 6 months, data were collected on 87% of participants, of which 82% were standard recall and 18% extended recall. At 12 months data were collected on 79% of participants, 88% of which were standard recall and 12% extended recalled. Note that monthly abstinence rates were not calculated for any month during which the participant either did not provide information on at least 15 days or was in a controlled environment for 15 days or more.
Verification of Drug Use Self-Reports
Urine samples were collected from 350 participants (83% of the sample) during at least one timepoint: n = 266 (73% of completed interviews) at 1 month, 219 (73%) at 3 months, 232 (74%) at 6 months, and 266 (76%) at 12 months. Agreement between urine screens and self-reported drug use at the corresponding month (defined as any outcome other than a positive urine screen combined with negative self-report) was high, ranging from 89% to 98% across all assessment points (Cohen’s kappa
= .74—.94 (Cohen, 1960
)). Also, hair samples were collected from 201 participants (48% of the sample) at 6 months only or, if missed, at 12 months. Agreement between hair analysis and self-report was also high: 91% (k
= .58). Because agreement rates were high, self-report data were considered valid and were not recoded on the basis of biological results.
Data were analyzed using Generalized Estimating Equations (GEE), an extension of the General Linear Model that permits a within-subject repeated measures examination of change over time as well as correction of variance estimates for correlated data within subject (Zeger & Liang, 1986
; Zeger, Liang, & Albert, 1988
). GEEs were used to examine condition differences in receipt of services in various domains as well as abstinence from substance use. Receipt of services within each domain was modeled as a binary outcome (0 = no services received, 1 = some services received) at each of the four study timepoints. Using the same procedures as in our previous work (Morgenstern et al., 2006
), we modeled complete abstinence from all substances for each month from baseline through the 12-month follow-up. Abstinence was treated as a binary outcome (0 = not abstinent, 1 = abstinent) assuming a binomial distribution, logit link function, and exchangeable correlation matrix. Preliminary models were run using the autoregressive correlation structure; however, the exchangeable correlation structure provided a better fit to our data and was used in the final models. The independent variable of primary interest was condition assignment, coded as 0 = UC and 1 = CCM.
To account for factors that may confound an association between condition assignment and abstinence outcomes, we undertook a process of model building in which we included a variety of baseline sample characteristics as covariates in the initial model. These included gender (0 = female, 1 = male), age, race (1 = African American, 2 = Hispanic, 3 = Other), education (0 = less than 12 years, 1 = 12 years or more), employment history (0 = no work in past 30 days, 1 = some work in past 30 days), housing status (0 = unstable housing, 1 = stable housing), legal involvement (0 = criminal justice history, 1 = no criminal justice history), mental health problems (0 = mental health problems, 1 = no mental health problems), methadone treatment status (0 = not receiving methadone treatment, 1 = receiving methadone treatment), number of prior AOD treatment episodes, and abstinence during the 6 months pre-baseline (number of months completely abstinent, ranging from 0 to 6). In the initial full GEE model, the following covariates had a marginal statistical association (p
< .10) with abstinence and were retained in the final model: pre-baseline abstinence and methadone treatment status. Condition by time interactions were also examined, but these were not significant and dropped from the final model. In the final reduced model, we tested the main effect of condition (CCM vs. UC) and the moderating effect of methadone treatment status (methadone vs. no methadone) to determine whether condition effects differed for clients receiving methadone. Given the lower statistical power for detecting interactions compared to main effects, we considered interaction effects significant at p
< .10 (Aiken & West, 1991
; Cohen, Cohen, West, & Aiken, 2003