Search tips
Search criteria 


Logo of nihpaAbout Author manuscriptsSubmit a manuscriptHHS Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
Behav Res Ther. Author manuscript; available in PMC 2010 November 1.
Published in final edited form as:
PMCID: PMC2784096

Treatment for Anxiety Disorders: Efficacy to Effectiveness to Implementation


Anxiety disorders are common, costly and debilitating, and yet often unrecognized or inadequately treated in real world, primary care settings. Our group has been researching ways of delivering evidence-based treatment for anxiety in primary care settings, with special interest to preserving the fidelity of the treatment while at the same time promoting its sustainability once the research is over. In this paper, we describe the programs we have developed and our directions for future research. Our first study evaluated the efficacy of CBT and expert pharmacotherapy recommendations for panic disorder in primary care, using a collaborative care model of service delivery (CCAP). Symptom, disability and mental health functioning measures were superior for the intervention group compared to treatment as usual both in the short term and the long term, although also more costly. In our ongoing CALM study, we have extended our population to include panic disorder, social anxiety disorder, generalized anxiety disorder and posttraumatic disorder, while at the same time utilizing clinicians with limited mental health care experience. In addition to pharmacotherapy management, we developed a computer-assisted CBT that guides both novice clinician and patient, thereby contributing to sustainability once the research is over. We have also incorporated a measurement based approach to treatment planning, using a web-based tracking system of patient status. To date, the computer-assisted CBT program has been shown to be acceptable to clinicians and patients. Clinicians rated the program highly, and patients engaged in the program. Future directions for our research include dissemination and implementation of the CALM program, testing potential alternations to the CALM program, and distance delivery of CALM.

Treatment for Anxiety Disorders: Efficacy to Effectiveness to Implementation


The gap between research and practice is well recognized (NIH, 2006a, 2006b), and has prompted increasing efforts to disseminate (targeted distribution of information on evidence-based health interventions) and implement (adaptation and putting in to practice such interventions over time) evidence-based practices in community based settings. To accomplish this, researchers have to deal with a number of important issues, including “how to translate interventions to encourage uptake and implementation in ways that preserve scientifically validated components of evidence-based practices, how to obtain buy-in of various stakeholders in settings over which researchers have little control, and how to sustain interventions beyond initial demonstrations and funding, particularly in settings with highly constrained resources” (Mendel, Meredith, Schoenbaum, Sherbourne, & Wells, 2008). Our group has been focusing upon ways of preserving the integrity of evidence-based treatment for anxiety disorders in real world settings and promoting their sustainability once the research is complete. In this paper, we describe the needs for effectiveness and implementation research for anxiety disorders, our programs and methodologies, and directions for future implementation research.

Anxiety Disorders and the Need for Movement from Efficacy to Implementation

Anxiety disorders are common, costly and debilitating (e.g., Kessler, Chiu, Demler, & Walters, 2005; Saarni, Suvisaari, Sintonen et al., 2005; Sareen, Jacobi, Cox, Belik, Clara, & Stein, 2006; Stein, Roy-Byrne, Craske et al., 2005). Most individuals with anxiety disorders receive their mental health care in the primary care setting, where unfortunately many patients are unrecognized or inadequately treated (e.g., Kroenke, Spitzer, Williams, Monahan, & Lowe, 2007; Fernandez, Haro, Martinez-Alonso et al., 2007). While there is some evidence that the proportion of individuals with anxiety disorders who seek help for their anxiety (Wang, Lane, Olfson, Pincus, Wells, & Kessler, 2005) is increasing, the care received remains less than optimal, especially when provided by non-specialists (Weisberg, Dyck, Culpepper, & Keller, 2007; Young, Klap, Sherbourne, & Wells, 2001). We (Stein, Roy-Byrne, Craske et al., in submission) found, in sample of 1004 primary care patients with panic disorder, social anxiety disorder, posttraumatic stress disorder and/or generalized anxiety disorder, that only 29.1% had received an appropriate medication for anxiety at an adequate dose and duration in the last six months and only 21.2% received counseling with at least a minimal CBT focus (41.4% received either or both).

There is good evidence for the efficacy of medications (e.g., Roy-Byrne, & Cowley, 2007) and cognitive behavioral therapy for anxiety disorders (e.g., Butler, Chapman, Forman et al., 2006; Norton & Price, 2007). However, the evidence is based mostly on efficacy studies in clinical research settings. Traditionally, efficacy studies seek to determine the clinical effect of treatments under controlled or “ideal” conditions that maximize study internal validity. This is commonly done through the use of placebos, standardized treatments, homogenous patient populations, rigorous follow-up with subjects, and blinding procedures. In their purest form, they involve specialized treatment settings, in which care is provided free, compliance with treatment is induced by care management techniques not typically available in practice settings, cases with medical and psychiatric comorbidity are excluded, and patient treatment preferences are not taken into account. By contrast, effectiveness studies traditionally seek to maximize generalizability of findings by enhancing external validity. They emphasize evaluation of treatments for delivery in usual care/community settings and focus on providers and patients who are representative of such settings. Effectiveness studies seek to answer the question “does the treatment work with real world patients in naturalistic clinical settings?” Finally, these two approaches differ in the outcomes on which they focus: efficacy studies focus primarily on clinical end points, whereas effectiveness studies broaden this focus to include social and work function, quality of life, and cost, thereby attaining a societal as opposed to an exclusively clinical perspective. (To an increasing extent, however, studies are being designed to be “hybrids” with both efficacy and effectiveness elements.) Our first aim was to establish the effectiveness of evidence-based care for anxiety disorders in primary care settings, deemed a necessary step before extending to implementation.

Effectiveness of Evidence-Based Treatment for Panic Disorder Treatment in Primary Care

Effectiveness research requires not only drawing from generalizable samples and settings, but also increasing the flexibility of the process of care so it more closely resembles that found in real-world settings. In other words, it is important to draw patients from a broad and representative group of clinics, and to limit the type of patient characteristics that serve as exclusions to ensure broader generalizability. It is equally important to develop a system or process of care more flexible than the strictly protocolized care delivered in efficacy studies, although organized enough that evidence-based treatments can be delivered in such a manner that their fidelity is reasonably maintained. To this end, we chose a collaborative care strategy that aims to overcome patient, provider, and process of care barriers to mental health treatment. It should be noted that different disciplines and investigators may vary in how much fidelity is sufficient and how much a treatment can be simplified before it ceases to be a substantive treatment. Even within our own group, we have continued to debate these issues.

The collaborative care model is closely patterned on the chronic disease model (Wagner, Austin, Davis et al., 2001). In collaborative care models in primary care, patients typically remain under the care of a primary care physician, while mental health specialists, usually master's level clinicians (e.g., nurses, social workers), coordinate and deliver specialized care in consultation with psychologists and/or psychiatrists. Other components to collaborative care include techniques to help patients manage their condition (such as by patient education, motivational enhancement, and identification and reduction of treatment barriers), as well as ongoing clinical monitoring of outcomes. The collaborative care approach has been tested extensively in primary care settings for depression with good results (e.g., Katon, Russo, Sherbourne et al., 2006; Unutzer, Rubenstein, Katon et al., 2001). The “user-friendliness” of these interventions is believed to be central to successful implementation (Pincus, Pechura, Keyser et al., 2006).

Using these effectiveness principles, we (Roy-Byrne, Craske, Stein et al., 2005) conducted a randomized controlled trial comparing an intervention to treatment as usual for panic disorder (n=231) in primary care (Collaborative Care for Anxiety and Panic; CCAP). The goal of CCAP was to establish the degree to which evidence-based, specialist delivered treatments for panic disorder generalized to primary care settings with non-specialised therapists and more diverse patient populations than is characteristic of efficacy trials. Six primary care clinics associated with university medical schools, with an ethnically and socially diverse patient population, participated. The intervention was comprised of a preparatory video about panic disorder and its treatment, and two components to treatment – pharmacotherapy and cognitive behavioral therapy (CBT). CBT involved up to six, 45 to 60 minute sessions (of which the latter three could be conducted by telephone) over three months, followed by up to six, 15 to 20 minute follow-up phone contacts over the next nine months. The medication component involved a recommendation by a study psychiatrist to the primary care physician regarding pharmacotherapy, according to an algorithm developed for the study. Psychiatrists made their recommendations based on review of weekly progress reports on patient clinical status and medication use, and their recommendations were relayed to the primary care physician by a behavioral health specialist. Hence, the latter were trained to not only deliver CBT but also to coordinate overall care, including pharmacotherapy. The behavioral health specialists became the linchpin tying the patient, primary care provider, and study psychiatrist together, thus providing collaborative care (see Figure 1). They were mostly BA and MA level therapists with limited prior experience in CBT.

Figure 1
Collaborative care in primary care treatment for panic disorder

The intervention resulted in sustained and gradually increasing improvement relative to treatment as usual. The percentage of subjects who responded well, defined as scoring less than 20 on the Anxiety Sensitivity Index, was 46% vs 27% at 3 months, and 63% versus 38% at 12 months. Furthermore, the effects extended to measures of disability and mental health functioning. These results were obtained even though the intervention did not lead to superior provision of guideline concordant pharmacotherapy relative to the treatment as usual condition, suggesting that the results were attributable largely to the CBT component of the intervention. This possibility was supported by evidence for better outcomes to derive from more CBT visits (Craske, Roy-Byrne, Stein, Sullivan, Hazlett-Stevens, Bystritksy, & Sherbourne, 2006). That is, after controlling for a number of baseline variables, number of CBT sessions predicted lower anxiety sensitivity at 3 months and 12 months, and number of follow-up booster telephone sessions predicted lower anxiety sensitivity, less phobic avoidance, and less depression at 12 months. The effectiveness of the brief monthly follow-up phone calls may indicate one way in which evidence-based treatments can be implemented in a less costly manner.

Effectiveness trials typically yield lower effect sizes even with the very same treatment protocol because they are designed with “noise” in attempting to recreate the “real world.” Thus, it is not surprising that our symptom outcomes were not as substantial as is typically seen in efficacy studies with a full course of CBT. Patients with more psychiatric and medical comorbidity than in a typical efficacy study may be one reason. Another likely factor was our decision to reduce participant burden and increase acceptance into the primary care setting by not only condensing CBT from its usual 12 sessions to a six-session format, but also including patients even if they failed to attend a minimum number of CBT session (only 40% of our participants completed all six CBT sessions; mean number of CBT sessions was 4.2 (sd=2.3) and mean number of follow-up telephone calls was 2.6 (2.4). Given our finding of improved outcomes with more CBT contacts (Craske et al., 2006), it was important to explore different ways to increase compliance with CBT. At the same time, it may be unrealistic to expect participation and adherence rates in the real world that approximate those seen in efficacy studies, where patients are already self-selected in ways that may maximize their participation.

We also evaluated the cost effectiveness of our intervention (Katon, Russo, Sherbourne, Stein, Craske, Fan, & Roy-Byrne, 2006). Relative to treatment as usual, intervention patients experienced 60.4 (95% confidence interval 42.9-77.9) more anxiety free days (derived from scores on the Anxiety Sensitivity Index) over a 12 month period. Cost was calculated for individual psychotherapy visits, primary care, emergency room and hospitalization visits, medication and medical testing, and psychiatrist supervision. When considering ambulatory costs (excluding inpatient costs because they decreased precision in determining the incremental cost-effectiveness ratio), the intervention produced an incremental cost effectiveness ratio of $8.40 per anxiety free day; in other words, the intervention was more effective but also more costly than treatment as usual. This result speaks to the value of developing more cost effective but evidence-based interventions, which was the goal of our next and ongoing study.

In sum, our initial evaluation of the effectiveness of evidence-based care for panic disorder showed that the results were substantial and long lasting, extending beyond symptom outcomes to functioning and quality of life, relative to treatment as usual in primary care clinics with relatively inexperienced ‘behavioral health specialists’. However, as an effectiveness trial, the CCAP study was limited in terms of generalizability by restriction to a single anxiety disorder, and by reliance on university affiliated primary care clinics. Also, by recruiting therapists who were external to the primary care system and hence unlikely to continue working there after the study was over (in contrast to selecting as therapists clinicians already employed in this setting) we had not addressed the important issue of sustainability once the research was complete. We next attempted to evaluate ways of improving generalizability of our findings, in a cost effective manner, while maintaining integrity of evidence-based care interventions, and potentially improving sustainability.

Effectiveness and Sustainability of Evidence-Based Anxiety Disorder Treatment in Primary Care

The primary goals of our ongoing study, Coordinated Anxiety Learning and Management (CALM), are to evaluate the clinical effectiveness of evidence-based treatment for panic disorder, generalized anxiety disorder, social anxiety disorder and posttraumatic stress disorder in comparison to treatment as usual. The study is powered to determine effectiveness of the intervention for both the individual anxiety disorders and the aggregate group of disorders. In addition, quality of care is being compared between the two treatment conditions. Another goal is to examine acceptability and perceived effectiveness of the intervention for low SES whites, African-Americans, and Hispanics. Secondary goals are to use quasi-experimental methods to estimate the effect of appropriate anxiety treatment on clinical and functioning outcomes, and to estimate health care costs and the cost-effectiveness.

CALM is the largest randomized trial of collaborative care for anxiety disorders conducted to date (Sullivan, Craske, Sherbourne et al., 2007). Participating research institutions are: University of Washington, Seattle, University of California at San Diego and Los Angeles, the University of Arkansas at Little Rock, Arkansas and the RAND Corporation. In contrast to our prior CCAP study, each site worked to select non-university affiliated clinics in their geographic area to participate. Candidate clinics were evaluated and 17 were purposively selected based on a number of considerations, including provider interest, space availability, size and diversity of the patient population, and insurance mix.

A “facilitated referral” approach used multiple strategies to recruit subjects. Primary care providers and clinic nursing staff directly referred potential subjects, and sites actively publicized the study within each clinic, allowing for self-referral. In addition, at some sites, a simple five-question anxiety screener was used to identify patients who had potential anxiety disorders (Means-Christensen, Sherbourne, Roy-Byrne, Craske, & Stein, 2006).

A specially trained study clinician (usually a nurse or a social worker), termed the Anxiety Clinical Specialist (ACS), functioned as both the main care manager/interventionist, as well as the diagnostician who met with referred subjects to determine eligibility for CALM. Having the eligibility diagnostic assessment completed by a more clinically oriented individual, as opposed to a research specialist, was a departure from many effectiveness studies, but more consistent with a model that was likely to be sustained after the study was over. An eligible subject had to be a patient at one of the participating clinics, be at least 18 years old, meet DSM-IV criteria for generalized anxiety disorder, panic disorder, social anxiety disorder or posttraumatic stress disorder (based on the Mini International Neuropsychiatric Interview) (Sheehan, Lecrubier, Sheehan et al., 1998), and score at least 8 (moderate but clinically significant anxiety symptoms on a scale ranging from 0-20) on the Overall Anxiety Severity And Impairment Scale (OASIS) (Campbell-Sills, Norman, Craske et al., 2009). Co-occurring major depression, expected to be common among anxious outpatients, was permitted.

Exclusion criteria were few and were intended to exclude persons who would not likely benefit from the intervention or for whom the intervention could be risky. They included serious alcohol or drug use (specifically, alcohol or marijuana dependence or any other drug abuse or dependence, including methadone), unstable medical conditions, marked cognitive impairment, active suicidal intent or plan, psychosis, or unstable bipolar I disorder. Subjects already receiving ongoing cognitive behavioral therapy (CBT) were excluded. Finally, persons without routine access to a telephone, or who could not speak English or Spanish were excluded.

ACSs who delivered the intervention (n=14) had some patient care experience (although only 8 had prior mental health care experience) and some exposure to primary care settings, but did not have expertise in anxiety management or CBT. The ACS ranged in age from 25 to 59 years, were mostly women, and typically had master's degrees in social work or bachelor's or master's degrees in nursing. Most were selected by study personnel and employed by the study site academic institution, but some were working clinically at affiliated sites and were employed by the health care provider organization operating the clinic (see Sullivan et al., 2007).

Patients who entered the CALM study were randomized to either treatment as usual or our collaborative care intervention. The latter involved either CBT, medication management or both, a decision that was made jointly by the patient and the ACS. The choice was complicated by the fact that over 50% of subjects were already taking medication and virtually all of these chose CBT. While they were encouraged to continue their medication and consider optimizing it by dose-adjustment (many subjects were taking medications at sub-optimal doses), in practice, many chose to focus only on the CBT and not make any changes in their medication (which was, at best, partially effective since they were still eligible for this study because of significant anxiety). After an initial course of treatment and if sufficient improvement had not been achieved, patients could choose another course of treatment with the same modality (CBT or medications) or the alternate modality. Further “steps” of treatment could be negotiated at three month intervals during the course of the year-long treatment protocol. In practice, however, continued treatment was only encouraged for individuals judged by the ACS (and his/her supervisor[s]) likely to benefit from an added course of either CBT or pharmacotherapy, and not simply because they remained symptomatic. Some individuals, after experiencing measureable improvement, remained anxious and had longer term problems (e.g., relationship difficulties; personality disorders) unlikely to respond to an additional short term course of CBT or medication.

Progress throughout our CALM intervention was monitored using a web-based tracking system which allowed for real time monitoring of recruitment, enrollment, diagnoses, ineligibility, patient contact information and continuous, session by session symptom assessments. At the initial visit with the patient, the ACS recorded information such as social, medical and mental health history, and medication type and dose, on the web based system. Subsequent treatment session notes captured changes in treatment and patient symptom scores that were graphed over time so that trends could be easily observed by the ACS and the supervisors who also had access to the information. Thereby, psychiatrist and psychologist supervisors monitored patient progress and focused discussion with ACSs upon patients who were not progressing optimally. The system also reminded ACSs to follow-up with patients and record successful or unsuccessful contacts, as well as record contacts with the primary care physician, thereby facilitating the collaborative care model. Overall, the web based system allowed for a measurement based approach to care, with ongoing data about patient progress serving to guide treatment planning. This more simple “outcomes” oriented approach stands in contrast to a more time-intensive “process-oriented” approach that monitors the care process for each individual, and may be more realistic in a busy clinical setting. If a patient is progressing well, there is no focus on whether or not the treatment received had the required “fidelity”. Only if the outcome is suboptimal would there be more scrutiny of the type of intensity of treatment. In essence, the idea is “whatever you're doing now is not working so we better do something else, or do this better”.

Pharmacotherapy component of CALM

A simple algorithm for medication management of anxiety was provided to the ACS along with several hours of didactic training to familiarize them with the basic evidence-based medications for anxiety, and common pitfalls that contribute to medication non-adherence. In practice, the algorithm recommended prescription of an SSRI or SNRI antidepressant, with the goal of increasing the dose to the maximum tolerable dose, to “optimize” dose and increase the likelihood of response. A lack of response would prompt substitution of a different antidepressant. Sub-optimal improvement would prompt a second step, in which another antidepressant, or possibly a benzodiazepine in select cases, would be added to the first agent. More elaborate interventions could be considered after consultation with the expert study psychiatrist. In practice, few patients received medication alone. A substantially larger number received medication in the course of receiving CBT. For these patients, ACSs had to take time from an already time-intensive CBT program to talk about medications. In practice, this was not terribly feasible and may not have happened very reliably.

CBT component of CALM

The CBT component was developed with the goal of maximizing user friendliness and ease of implementation. We aimed to create a brief intervention that could be adopted in a variety of primary care settings that lack existing mental health expertise. To this end, we developed a novel, computerized system for supporting delivery of CBT for anxiety disorders by novice clinicians, called CALM Tools for Living (see Craske, Rose, Lang et al., 2009). In contrast to separate, uniquely tailored CBT manuals for different disorders, we developed a CBT approach that singly addresses the four most common anxiety disorders in primary care settings (Kroenke et al., 2007): panic disorder with or without agoraphobia, generalized anxiety disorder, social anxiety disorder, and posttraumatic stress disorder. In this program, the core elements of CBT are the same across the four anxiety disorders, whereas other elements are tailored to the features unique to each anxiety disorder through branching mechanisms.

In its application, the focus is maintained upon the most distressing and impairing of the four anxiety disorders, as rated by the patient. Comorbid anxiety as well as mood disorders are presumed to improve as a result of this focused attention, given the evidence that CBT for a targeted anxiety disorder yields positive benefits upon comorbid anxiety and mood disorders (Borkovec, Abel, & Newman, 1995; Brown, Antony, & Barlow, 1995; Tsao, Mystkowski, Zucker et al., 2005). Moreover, we chose not to focus upon more than one anxiety disorder at the same time given the preliminary evidence that CBT programs that simultaneously target a principal anxiety disorder and a comorbid anxiety or mood disorder are less effective than CBT targeting a principal anxiety disorder only (Craske, Farchione, Allen et al., 2007).1

Another novel feature is that the computer program guides both the clinician (the ACS) and patient. The choice to guide the clinician by the computer program was driven by two factors. First was the need to minimize clinician training to enhance eventual implementation in primary care and other real world settings. Second was the need to develop a system that would support minimally trained clinicians’ implementation of a relatively sophisticated CBT for not just one but four anxiety disorders. Thus, we developed a program in which a computer program guides the clinician, session by session, in the delivery of CBT. Computer-assisted programs already have been used in training clinicians in CBT (Carroll & Rounsaville, 2007; Gega, Norman, & Marks, 2007), but they have not before been used for ongoing assistance for delivery of CBT. The program aids novice clinicians by not only providing the structure for delivering CBT for four different anxiety disorders, but also by helping them to remain on target (i.e., focus upon CBT), and to maintain CBT fidelity. Furthermore, the computerized program is expected to contribute to sustainability of evidence-based CBT within real world settings.

The computer program also guides the patient. Stand-alone (i.e., without clinician involvement) computerized/internet versions of CBT are well researched, and have been shown to be generally acceptable and effective in treating depression and anxiety (Gega, Marks, & Mataix-Cols, 2004; Proudfoot, Goldberg, Mann et al., 2003), as well as specific anxiety disorders, including panic disorder (e.g., Carlbring, Bohman, Brunt et al., 2006)), social anxiety disorder (e.g., Botella, Hofmann, & Moscovitch, 2004; Titov et al., 2008), posttraumatic stress disorder (Lange, van de Ven, Schrieken et al., 2001), and obsessive compulsive disorder (Kenwright, Marks, Graham et al., 2005). However, they are associated with higher rates of drop-out or refusals, lower rates of satisfaction with therapy, and less superior outcomes compared to a live clinician: computerized programs are more acceptable and more successful when clinician involvement is offered (Spek, Cuijpers, Nyklicek et al., 2007). Hence, our choice to engage the clinician with the patient as they both proceed simultaneously through the computer program was expected to enhance patient involvement and satisfaction (see Figure 2).

Figure 2
CALM Computer Assisted Delivery of CBT (CALM Tools for Living)

The ACS training for CBT involved readings and five days of workshops addressing CBT in general, CBT in CALM, and the tailoring of CBT to each of the four anxiety disorders. Interspersed between workshops were role-plays where ACSs practiced CBT skills and use of the computer-assisted program. In addition, they took on two to four “training cases” and their proficiency at delivering CBT was evaluated by expert psychologists. Lastly, they received ongoing group telephone supervision for approximately one hour per week from an expert psychologist.

As in our prior CCAP study, the ACSs did more than provide CBT. They conducted initial diagnostic assessments, provided education about anxiety, prepared patients to make treatment choices, addressed barriers to treatment, and assisted the prescribing physician with medication management. Once trained in all of these aspects, ACSswere supervised by study psychiatrists in separate weekly hour long, group phone calls.

Results from CALM Tools for Living

Results from the larger CALM study await finalization of the twelve month follow-up assessment. However, the acceptability and utility of the computerized CBT program has been evaluated (see Craske et al., 2009 for more details). First, 13 ACSs who were trained to use the CALM Tools for Living program and who used the program for at least a year rated the program very highly. They strongly endorsed that the program worked well during the meetings with patients, that it was relatively easy to use, and that their overall opinion was very high as was their judgment as to their patients’ opinion (1-7 point Likert scale scores ranged from 5.69 to 6.23; ease of language was rated slightly lower, 5.39). In responses to open-ended questions, ACSs appreciated the way in which the program provided a clear agenda per session and overall structure, as well as an easy-to-use guide for CBT concepts. They also appreciated the comprehensiveness of the information, the print functions for creating individualized-patient “workbooks”, the video demonstrates of specific skills, and the interactive nature of the program as strengths of Calm Tools for Living. They also noted areas for improvement, including simplification of the language, reduction in the amount of text and elimination of redundancy, and greater flexibility. These data are consistent with results from our analysis of this cohorts’ history of previous care, where satisfaction with mental health care was strongly associated with receipt of CBT-oriented psychotherapy (and the more CBT elements incorporated, the greater the satisfaction) but not with receipt of pharmacological care.

In addition, patients engaged in the program. Of the first set of 261 patients who received at least one CBT session, attendance averaged 7.63 (SD = 3.39) computer-assisted CBT sessions with an ACS (range = 1 - 19). In addition, they reported completing homework assignments between sessions. For example, they practiced CALM breathing skills on average 6.5 times (SD=3.6) between when they were first introduced to this skill and their next session with the ACS (typically this interval was one week). Patients also indicated that they understood the material in the program, with their 0-100 point ratings of understanding of each module ranging from 85.4 to 91.6. Their accuracy on the quiz tests was similarly very high, ranging from 86% to 97%.

Self reported levels of anxiety and depression symptoms were evaluated across randomly selected subsets of patients. Anxiety and depression scores decreased significantly and substantially from the first to the last Calm Tools for Living session, and to the same degree whether the target of CBT was panic disorder, generalized anxiety disorder, social anxiety disorder or posttraumatic stress disorder (see Craske et al., 2009). Also, patient rated understanding of the Calm Living module predicted final session anxiety scores, even after controlling for all other variables, suggesting that symptom outcomes may be due to specifics of the program, and the importance of exposure to feared situations in particular, rather than nonspecific factors.

Implementation and Future Directions

While final results from CALM are not yet available, a number of issues are open for exploration. Assuming that CALM is found to be clinically effective as well as cost-effective, a major issue is how to encourage its uptake in primary care practices or its adoption in other systems of care. In part, this will rely on the buy-in of various stakeholders, many of whom face issues of constrained resources. Mendel et al. (2008) describe the ways in which the UCLA/RAND NIMH Center for Research on Quality in Managed Care is attempting to address dissemination and implementation from the narrowest to the broadest levels. Consideration is given to multiple layers of stakeholders, from individual care providers and consumers to care delivery organizations, insurers and purchases, interest groups, regulatory agencies, to macrosystems of the cultural/normative environment, resource/economic environment and legal/policy environment. Within that framework, dissemination is considered from the stand point of an evaluation process (including a capacity/needs assessment, implementation evaluation and outcome evaluation) and a diffusion process, including consideration of the context of diffusion (such as norms and attitudes, resources, policies and incentives, networks and linkages and media and change agents), stages of diffusion (from adoption to implementation to sustainment), and intervention outcomes in terms of both the patient and the organization.

For CALM, one issue will be to understand why it is or is not adopted by participating clinics after the research project has ended. Interviews with different stakeholders, currently being conducted, will provide data relevant to how to tailor or adapt CALM to particular settings. Natural adaptations that occurred in the different clinical sites (e.g., differences in number of length of sessions delivered; differences in numbers and characteristics of patients offered further steps of care; differences in attention given to medications in the context of CBT) will provide useful information about barriers and challenges to implementation of CALM. In addition, such information can be used to improve future design modifications for CALM and to address potential dissemination strategies.

In addition, it will be possible to explore variations on the current CALM model. For example, it would be possible to examine the degree to which the results generalize to different types of clinicians with different levels of training. Presumably, the program will be most acceptable to those with least expertise and training in CBT whereas experienced CBT clinicians might find the computer based program too constraining. By providing on-line guidance, the computer assisted program keeps novice clinicians on task and thereby maintain CBT integrity. Whether it is superior to CBT conducted by novice therapists with the aid of manuals or workbooks remains open to future investigation. Other alterations in the CALM model could be tested. For example, from a public health and cost-effectiveness perspective, it is possible that a less intensive version of CALM might suffice for some people. It could be parsimonious and more cost-effective to offer a less intensive version first, reserving the full CALM program for those who fail to respond to the less intensive version. It is also possible that approaches to further tailor CALM to groups or to individuals might be studied. Indeed, using qualitative methods, we are examining the acceptability of CALM to a range of study subjects, including those of varying ethnicity and socio-economic status. We expect this work may give us insights into how the current CALM intervention might be more effectively tailored for these groups.

One of the most innovative features of CALM is use of the computer-assisted CBT program. We are now exploring the potential of delivering the CALM intervention at a distance via the internet and telephone. We recognize that there is a mis-match between need for treatment, and particularly in application of evidence-based practices, and availability of mental health professionals. This need may be most extreme in rural areas where very few mental health professionals practice. Delivery of the CALM CBT program at a distance could remedy this mis-match.


This work was supported by the following National Institute of Mental Health grants: U01 MH070018, U01 MH058915, U01 MH057835, UO1 MH057858, U01 MH070022, K24 MH64122, and K24 MH065324.


Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

1Unified CBT protocols that teach cognitive and behavioral skills for managing all forms of negative affect remain under development


  • Alonso J, Angermeyer MC, Bernert S, Bruffaerts R, Brugha TS, Bryson H, et al. Disability and quality of life impact of mental disorders in Europe: results from the European Study of the Epidemiology of Mental Disorders (ESEMeD) project. Acta Psychiatrica Scandinavica. 2004;109:47–54. [PubMed]
  • Borkovec TD, Abel JL, Newman H. Effects of psychotherapy on comorbid conditions in generalized anxiety disorder. Journal of Consulting and Clinical Psychology. 1995;63:479–483. [PubMed]
  • Botella C, Hofmann SG, Moscovitch DA. A self-applied, Internet-based intervention for fear of public speaking. Journal of Clinical Psychology. 2004;60:821–830. [PubMed]
  • Bower P, Rowland H, Hardy R. The clinical effectiveness of counseling in primary care: a systematic review and meta-analysis. Psychological Medicine. 2003;33:203–215. [PubMed]
  • Brown TA, Antony M, Barlow DH. Long-term outcome in cognitive-behavioral treatment of panic disorder: clinical predictors and alternative strategies for assessment. Journal of Consulting and Clinical Psychology. 1995;63:754–765. [PubMed]
  • Butler AC, Chapman JE, Forman E, Beck AT. The empirical status of cognitive-behavioral therapy: a review of meta-analyses. Clinical Psychology Review. 2006;26:17–31. [PubMed]
  • Campbell-Sills L, Norman SB, Craske MG, Sullivan G, Lang AJ, Chavira DA, et al. Validation of a brief measure of anxiety-related severity and impairment: the Overall Anxiety Severity and Impairment Scale (OASIS). Journal of Affective Disorders. 2009;11:92–101. [PMC free article] [PubMed]
  • Carlbring P, Bohman S, Brunt S, Buhrman M, Westling BE, Ekselius L, et al. Remote treatment of panic disorder: a randomized trial of Internet-based cognitive behavior therapy supplemented with telephone calls. American Journal of Psychiatry. 2006;163:2119–2125. [PubMed]
  • Carroll KM, Rounsaville BJ. A vision of the next generation of behavioral therapies research in the addictions. Addiction. 2007;102:850–869. [PMC free article] [PubMed]
  • Craske MG, Farchione TJ, Allen LB, Barrios V, Stoyanova M, Rose RD. Cognitive behavioral therapy for panic disorder and comorbidity: more of the same or less of more? Behaviour Research and Therapy. 2007;45:1096–1109. [PubMed]
  • Craske MG, Golinelli D, Stein MB, Roy-Byrne P, Bystritsky A, Sherbourne C. Does the addition of cognitive behavioral therapy improve panic disorder treatment outcome relative to medication alone in the primary-care stetting? Psychological Medicine. 2005;35:1–10. [PubMed]
  • Craske MG, Rose RD, Lang A, Welch SS, Campbell-Sills L, Sullivan G, Sherbourne C, Bystritsky A, Stein MB, Roy-Byrne PP. Computer-assisted delivery of cognitive behavioral therapy for anxiety disorders in primary-care settings. Depression and Anxiety. 2009;26:235–242. [PMC free article] [PubMed]
  • Craske MG, Roy-Byrne P, Stein MB, Sullivan G, Hazlett-Stevens H, Bystritsky A, et al. CBT Intensity and Outcome for Panic Disorder in a Primary Care Setting. Behavior Therapy. 2006;37:112–119. [PubMed]
  • Fernandez A, Haro JM, Martinez-Alonso M, Demyttenaere K, Brugha TS, Autonell J, et al. Treatment adequacy for anxiety and depressive disorders in six European countries. British Journal of Psychiatry. 2007;190:172–173. [PubMed]
  • Gega L, Marks I, Mataix-Cols D. Computer-aided CBT self-help for anxiety and depressive disorders: experience of a London clinic and future directions. Journal of Consulting and Clinical Psychology. 2004;60:147–157. [PubMed]
  • Gega L, Norman IJ, Marks IM. Computer-aided vs. tutor-delivered teaching of exposure therapy for phobia/panic: randomized controlled trail with pre-registration nursing students. International Journal of Nursing Studies. 2007;44:397–405. [PubMed]
  • Katon W, Russo J, Sherbourne C, Stein MB, Craske MG, Fan M-Y, et al. Incremental cost-effectiveness of a collaborative care intervention for panic disorder. Psychological Medicine. 2006;36:353–363. [PubMed]
  • Kenwright M, Marks I, Graham C, Franses A, Mataiz-Cols D. Brief scheduled phone support from a clinician to enhance computer-aided self-help for obsessive-compulsive disorder: randomized controlled trial. Journal of Clinical Psychology. 2005;61:1499–1508. [PubMed]
  • Kessler RC, Chiu WT, Demler O, Walters EE. Prevalence, severity, and comorbidity of 12-month DSM-IV disorders in the National Comorbidity Survey Replication. Archives of General Psychiatry. 2005;62:617–627. [PMC free article] [PubMed]
  • Kroenke K, Spitzer RL, Williams JB, Monahan PO, Lowe B. Anxiety disorders in primary care: prevalence, impairment, comorbidity, and detection. Annals of Internal Medicine. 2007;146:317–325. [PubMed]
  • Lange A, van de Ven J, Schrieken B, Emmelkamp PMG. Interapy: Treatment of posttraumatic stress through the Internet: a controlled trial. Journal of Behavior Therapy and Experimental Psychiatry. 2001;32:73–90. [PubMed]
  • McLean PD, Woody S, Taylor S, Koch WJ. Comorbid panic disorder and major depression: implications for cognitive-behavioral therapy. Journal of Consulting and Clinical Psychology. 1998;66:240–247. [PubMed]
  • Means-Christensen AJ, Sherbourne CD, Roy-Byrne PP, Craske MG, Stein MB. Using five questions to screen for five common mental disorders in primary care: diagnostic accuracy of the Anxiety and Depression Detector. General Hospital Psychiatry. 2006;28:108–118. [PubMed]
  • Mendel P, Meredith LS, Schoenbaum M, Sherbourne C, Wells KB. Interventions in Organizational and Community Context: A Framework for Building Evidence on Dissemination and Implementation in Health Services Research. Administration and Policy in Mental Health and Mental Health Services Research. 2008;35:21–37. [PMC free article] [PubMed]
  • NIH (National Institutes of Health) Dissemination and Implementation Research in Health (R01 Program Announcement). 2006a. [25 October 2006]. Retrieved from
  • NIH (National Institutes of Health) NIH Roadmap Request for Information (RFI); Translation; The Science of Knowledge Dissemination, implementation and integration. 2006b. Retrieved from
  • Norton PJ, Price EC. A meta-analytic review of adult cognitive-behavioral treatment outcome across the anxiety disorders. Journal of Nervous and Mental Disease. 2007;195:521–531. [PubMed]
  • Pincus HA, Pechura C, Keyser D, et al. Depression in primary care: learning lesions in a National Quality Improvement program. Administration and Policy in Mental Health and Mental Health Services Research. 2006;33:2–15. [PubMed]
  • Proudfoot J, Goldberg D, Mann A, Everitt B, Marks I, Gray JA. Computerized, interactive, multimedia cognitive behavioural program for anxiety and depression in general practice. Psychological Medicine. 2003;33:217–227. [PubMed]
  • Roy-Byrne PP, Cowley DS. Pharmacological treatments for panic disorder, generalized anxiety disorder, specific phobia and social anxiety disorder. In: Nathan PE, Gordon JM, editors. A Guide to Treatments that Work. 3rd ed. Oxford University Press; New York: 2007.
  • Roy-Byrne PP, Craske MG, Stein MB, Sullivan G, Bystritsky A, Katon W, et al. A Randomized Effectiveness Trial of Cognitive-Behavioral Therapy and Medication for Primary Care Panic Disorder. Archives of General Psychiatry. 2005;62:290–298. [PMC free article] [PubMed]
  • Saarni SI, Suvisaari J, Sintonen H, Pirkola S, Koskinen S, Aromaa A, et al. Impact of psychiatric disorders on health-related quality of life: general population survey. British Journal of Psychiatry. 2007;190:326–332. [PubMed]
  • Sareen J, Jacobi F, Cox BJ, Belik SL, Clara I, Stein MB. Disability and poor quality of life associated with comorbid anxiety disorders and physical conditions. Archives of Internal Medicine. 2006;166:2109–2116. [PubMed]
  • Sheehan DV, Lecrubier Y, Sheehan KH, Amorim P, Janavs J, Weiller E, et al. The Mini-International Neuropsychiatric Interview (M.I.N.I.): the development and validation of a structured diagnostic psychiatric interview for the DSM-IV and ICD-10. Journal of Clinical Psychiatry. 1998;59(Suppl 20):22–33. [PubMed]
  • Spek V, Cuijpers P, Nyklicek I, Riper H, Keyzer J, Pop V. Internet-based cognitive behavioural therapy for symptoms of depression and anxiety: a meta-analysis. Psychological Medicine. 2007;37:319–328. [PubMed]
  • Stein MB, Roy-Byrne PP, Craske MG, Bystritsky A, Sullivan G, Pyne JM, et al. Functional Impact and Health Utility of Anxiety Disorders in Primary Care Outpatients. Medical Care. 2005;43:1164–1170. [PubMed]
  • Stein MB, Roy-Byrne PP, Craske MG, Campbell-Sills L, Lang AJ, Golinelli D, et al. Quality of and Patient Satisfaction with Primary Health Care for Anxiety Disorders. in submission.
  • Sullivan G, Craske MG, Sherbourne C, Edlund MJ, Rose RD, Golinelli D, et al. Design of the Coordinated Anxiety Learning and Management (CALM) study: innovations in collaborative care for anxiety disorders. General Hospital Psychiatry. 2007;29:379–387. [PMC free article] [PubMed]
  • Tsao JCI, Mystkowski JL, Zucker BG, Craske MG. Impact of cognitive-behavioral therapy for panic disorder on comorbidity: a controlled investigation. Behaviour Research and Therapy. 2005;43:959–970. [PubMed]
  • Unutzer J, Rubenstein L, Katon WJ, Tang L, Duan N, Lagomasino IT, Wells KB. Two-year effects of quality improvement programs on medication management for depression. Archives of General Psychiatry. 2001;58:935–942. [PubMed]
  • Wagner EH, Austin BT, Davis C, Hindmarsh M, Schaefer J, Bonomi A. Improving chronic illness care: translating evidence into action. Health Affairs. 2001;20(6):64–78. [PubMed]
  • Wang PS, Lane M, Olfson M, Pincus HA, Wells KB, Kessler RC. Twelve-month use of mental health services in the United States: results from the National Comorbidity Survey Replication. Archives of General Psychiatry. 2005;62:629–640. [PubMed]
  • Weisberg RB, Dyck I, Culpepper L, Keller MB. Psychiatric treatment in primary care patients with anxiety disorders: A comparison of care received from primary care providers and psychiatrists. American Journal of Psychiatry. 2007;164:276–282. [PubMed]
  • Young AS, Klap R, Sherbourne CD, Wells KB. The quality of care for depressive and anxiety disorders in the United States. Archives of General Psychiatry. 2001;58:55–61. [PubMed]