Our meta-analysis indicates that IPS produces better competitive employment outcomes for persons with SMI than alternative vocational programs regardless of a range of background demographic, clinical, and employment characteristics. These findings may come as a surprise to program providers and investigators who assume that supported employment is suitable for only a segment of the target population.18,50
Instead, the data indicate that IPS outcomes generalize broadly to people with SMI, with no clearly contraindicated subgroups. These findings are consonant with the zero exclusion principle. While we may yet identify subgroups that benefit from sheltered workshops, skills training prior to employment, or some other stepwise approach, the burden of proof rests with proponents of these views.
Our meta-analysis provides practical assistance to researchers planning future RCTs of IPS for specific target populations. Specifically, this article includes a table of effect sizes critical for conducting power analyses. For example, the meta-analysis provides guidance on designing a study evaluating a program for middle-aged or older adults.
Rather than additional RCTs comparing IPS to other vocational models, future research should emphasize enhancing the IPS approach for clients who do not benefit from a trial of supported employment.13
For example, clients who are unable to maintain employment or work as many hours as they would like may benefit from individualized clinical interventions tailored to specific deficits and barriers to employment. To date, promising augmentations of IPS have focused on cognitive remediation,26
but there are a host of other barriers warranting targeted interventions, such as co-occurring physical illnesses, poor symptom control, the deficit syndrome,51
criminal justice involvement, and family interference.
This meta-analysis has several distinctive strengths. All 4 RCTs ensured high fidelity to the IPS model. All provided active, highly regarded comparison interventions, substantially better than those most clients receive in routine mental health care.52
All used a common protocol, a standardized follow-up period, and the identical set of predictor measures, thereby avoiding the challenges of calibrating different scales. All used intent-to-treat protocols, with modest sample losses. Finally, because of the common protocol, this meta-analysis required much less imputation than previous meta-analyses. For these reasons, the overall results shown in and the associated effect sizes significantly advance the literature, including the Cochrane Review53
review, which are now badly outdated. Another recent review4
encompasses more IPS studies but provides limited effect size information and lacks the standardization included in the current analysis.
One criticism of the IPS model has been that it is limited to individuals who want to work and therefore excludes a sizeable proportion of the target population who could benefit from employment services.50
This view reflects a misunderstanding about IPS. We strongly concur with the recommendations of the Schizophrenia Patient Outcomes Research Team that vocational goals should be part of every client's treatment plan.1
The IPS model strongly advocates (and employs) assertive outreach, motivational interview techniques, and active encouragement of employment through a variety of venues.54
Ultimately, however, the decision to enter an employment program should be the client's; the purpose of the informational group is to enable clients to make informed choices. Supported employment studies that have enrolled clients without regard to the client's interest in working have predictably obtained substantially lower employment rates.55
The proportion of clients enrolled in mental health services who have vocational goals has been estimated in numerous surveys to be in the 50%–60% range.56
Clients not expressing vocational aspirations often seek other meaningful roles, such as student, homemaker, retiree with hobbies and recreational activities, self-help group member, and/or volunteer worker, among many other possibilities. Among SSDI beneficiaries with psychiatric disabilities not receiving mental health services, the percentage of those who may agree to participate in IPS is probably closer to 20%–30%.14
Reaching out to those who are currently unengaged in the world of work is crucially important, but currently the field lacks both adequate information about specific predictors of outcome for this subgroup and evidence-based interventions for engaging those who might benefit. Clearly, this is a direction for future research.
A related issue in the current study regards the external validity of the meta-analytic sample to the population of interest. Specifically, are there substantial selection biases that preclude generalizing these results to the broader population of individuals with SMI who could benefit from supported employment? Several points are pertinent. First, these studies were conducted in public mental health settings so that the issue of translating an efficacy trial into an effectiveness trial does not apply here. Second, the exclusion criteria for the current studies are far less stringent than in medication studies or most studies of psychosocial interventions. Third, in one study included in this meta-analysis, the study sample was compared on demographic and clinical characteristics with the total population of clients with SMI served in the state's public mental health system.32
In general, the statistical differences were modest. Fourth, the 66% participation rate among those attending information groups in the 4 studies was relatively high, suggesting good penetration within the target population. Finally, in contrast to the current group of studies, other studies have examined predictors of employment in supported employment programs that recruited clients without regard to their expressed interest in employment. In these other studies, the interpretation of the results is confounded by motivational issues. So, eg, Gold et al57
obtained paradoxical findings in one such study: Cognitive variables did not predict whether or not clients obtained employment, but among those who obtained work, cognitive variables were a predictor. One interpretation of these results is that the heterogeneity of the study sample, by including participants who had no genuine interest in employment, masked the findings for the target sample for whom employment was a goal. In summary, the decision to include clients without a vocational goal in analyses of predictors of employment outcomes is a methodological one; our view is that partitioning the population and treating these as 2 distinct groups and 2 separate research questions will lead ultimately to more precise results and better guide the design and improvement of evidence-based practices.
In any meta-analysis, the question could be raised about the clinical significance of the findings. That is, effect sizes are a statistical measure, but how do the findings translate into practical outcomes? First, we note that working has social validity; little explanation is required for a layperson to understand the meaning of employment, in contrast to, eg, interpreting the meaning of changes on a symptom measure or a cognitive assessment. Second, several prospective studies have suggested that job tenure is associated with a variety of positive outcomes outside the realm of employment, including higher self-esteem, better control of psychiatric symptoms, and less social disability.58–60
Third, economic modeling suggests that over the long term, small increases in employment across a large population may lead to greatly reduced societal costs.14
Our meta-analysis has several limitations. (a
) Findings are limited to individuals with SMI who are enrolled in mental health treatment. Many other individuals are not in the service system.41
) The 4 RCTs included in the analyses were all conducted by research teams that included the developers of the IPS model. Moreover, the findings apply only to high-fidelity IPS programs and may not generalize to other supported employment approaches. (c
) Our findings are also limited to those who want to work. (d
) We examined only 3 employment outcome indicators. Future studies should include a broader range of employment variables, including earnings, time to first job, and rates of full-time employment. (e
) The 18-month follow-up period was relatively brief. A longer follow-up period might reveal sharper differences because a number of long-term follow-up studies suggest increased employment over time. (f
) The measure of job tenure used in this meta-analysis was censored. That is, some participants working at the 18-month follow-up period continued to work beyond that time period, leading to an underestimate of the mean job tenure for the sample as a whole. The best solution to this problem of underestimation is to conduct studies with longer follow-ups. (g
) Not all variables of interest were examined in these analyses (eg, neurocognitive variables and criminal justice involvement were not measured). (h
) While the sample size was large enough to examine subgroups on individual predictor measures, we had insufficient power to examine combinations of client characteristics (eg, African American males). (i
) Finally, the analyses included a large number of statistical tests, leading to alpha inflation. However, this criticism is only a minor concern because the focus was on effect sizes and not statistical significance.