A Conceptual and Empirical Review of the Meaning, Measurement, Development, and Teaching of Intervention Competence in Clinical Psychology

doi:10.1016/j.cpr.2008.09.008

Clin Psychol Rev. Author manuscript; available in PMC 2010 February 1.

Published in final edited form as:

Clin Psychol Rev. 2009 February; 29(1): 47–56.

Published online 2008 September 30. doi: 10.1016/j.cpr.2008.09.008

PMCID: PMC2652709

NIHMSID: NIHMS91637

A Conceptual and Empirical Review of the Meaning, Measurement, Development, and Teaching of Intervention Competence in Clinical Psychology

Brian A. Sharpless^a and Jacques P. Barber^a

^aCenter for Psychotherapy Research, The University of Pennsylvania, 3535 Market Street, 6^th Floor, Philadelphia, PA 19104, USA

Corresponding author.

Email Addresses, Phone Numbers, and fax numbers:

Brian A. Sharpless: basharpless/at/gmail.com; office = 1-215-662-2837; fax = 1-215-349-5171

Jacques P. Barber: barberj/at/mail.med.upenn.edu; office = 1-215-662-2306

The publisher's final edited version of this article is available at Clin Psychol Rev

See other articles in PMC that cite the published article.

Publisher's Disclaimer

Abstract

Through the course of this paper we discuss several fundamental issues related to the intervention competence of psychologists. Following definitional clarification and proposals for more strictly distinguishing competence from adherence, we interpret Dreyfus and Dreyfus’s (1986) five stage theory of competence development (from novice to expert) within a strictly clinical framework. Existing methods of competence assessment are then evaluated, and we argue for the use of new and multiple assessment modalities. Next, we utilize the previous sections as a foundation to propose methods for training and evaluating competent psychologists. Lastly, we discuss several potential impediments to large scale competence assessment and education, such as the heterogeneity of therapeutic orientations and what could be termed a lack of transparency in clinical training.

The idea of psychological competence has recently assumed a prominent role in our field. Competencies of various sorts (e.g., research, teaching, assessment) have been widely discussed (e.g, Kaslow, 2004), and recent work has attempted to formalize these multifarious competencies into a widely-known “cube model” (Rodolfa et al., 2005). However, many questions about competence remain unanswered, and these include several which are fundamental (e.g., what competence means, how best to measure it, and how to it develops). This current lack of firm answers is likely due to the number of central theoretical issues that underlie the construct of competence as well as the assortment of psychological approaches that are currently available.

Clinical psychology and the services it provides represent a principal locus where psychological knowledge and the general public converge in an intimate way. As the public is continually searching for a means to ensure that its time and money are invested in trusted and competent sources, the idea of intervention competence (or that particular type of competence demonstrated when remedying psychological difficulties) holds an obvious relevance. This is especially the case given that accountability has recently become something of a cause celebre (e.g., Nelson, 2007). Thus, given this importance, we intend to review the many complexities involved in understanding intervention competence. It is important to note that we will not explicitly discuss other closely-related constructs of competence (e.g., assessment competence). Though important and relevant, and likely related to intervention competence, a decision was made to focus solely on intervention competence. The approach taken in this paper is to raise relevant a priori questions about intervention competence, provide answers where possible (derived from empirical and theoretical sources) and, where not possible, to explore these conceptual difficulties in the hopes of moving towards an eventual “good enough” consensus. After discussing the importance of intervention competence, we will explore its development and measurement and will attempt to derive educational implications from this review. We also hope to provide reasons for the conspicuous difficulties involved in these tasks.

Why Should we Assess Intervention Competence?

We begin by asking two fundamental questions: should we assess intervention competence, and if so, why? The answers to these seemingly obvious questions are, in fact, quite complex. A simple-minded, yet nonetheless accurate response to these questions would be that the APA Ethics Guide (2002) says that we should. These guidelines clearly indicate that a therapist’s competence is an ethical issue that we as psychologists are bound not only to consider and evaluate, but also to increase and undergird. Further, our ethical obligations include providing the best available professional services (i.e., using the most effective techniques [APA, 2005]) and, in the absence of established standards, we are obligated to take necessary steps to ensure that our patients receive adequate care as well as professional interventions.

There are reasons to explore competence beyond APA guidelines, however. For instance, practitioners of any craft desire to maintain high standards for both themselves (viz., personal reputations) and their profession. Thus, we have a broader and supra-personal ethical obligation to ensure that we engage in competent work and treatment. Public protection is also a factor, as we obviously want to provide assurance that we as psychologists cause no harm and actually bring at least a modicum of relief to our patients. This protection is partially safeguarded by licensing boards, but an individual duty also exists. And, finally, exploring intervention competence is also important for adequately training the next generation of therapists.

What is Intervention Competence, and What is Not Intervention Competence?

Before we explore specific aspects of intervention competence, it is necessary to possess both positive and negative definitions of the term (viz., what it is, and what it is not). However, a plurality of definitions currently exists, and none is self-evident. Dictionary definitions provide some conceptual guidance, but lack specificity. For instance, competence can be defined as: (1) the state or quality of being adequately or well-qualified, (2) demonstrating ability, or (3) it can have a legal definition (i.e., being legally qualified to perform some action). But what does this mean within the confines of intervention competence? Can competence be reduced to successfully passing legally-required licensure exams? Is it merely a minimally acceptable criterion (i.e., not harmful, and maybe a little bit helpful) or a particular rung on the ladder of therapeutic skill ranging somewhere between flagrantly incompetent and the most competent therapist who ever lived (e.g., one standard deviation above the mean of all therapists)? Answers to these questions are not unambiguously forthcoming.

Securing a workable definition is also hampered by the fact that it is frequently conjoined with other terms (Barber, Sharpless, Klostermann, & McCarthy, 2007). Intervention competence (along with adherence and treatment differentiation) is a core component of treatment integrity, or the extent to which a treatment was implemented as intended and able to be differentiated from other approaches (Perepletchikova & Kazdin, 2005). It is (arguably) a necessary prerequisite for making valid determinations of any therapy’s effectiveness. Treatment integrity is thought to be raised through the use of psychotherapy manuals, and adherence is typically considered to be the extent to which a therapist utilizes prescribed interventions without taking recourse to proscribed techniques. However, adherence and competence are often comingled, and this may result in a lack of definitional clarity. Thus, we will elaborate on the crucial distinction, as we see it, between adherence and competence.

How do Adherence and Intervention Competence Differ?

We would argue that the distinction between adherence and competence hinges on the type of knowledge that each respectively demonstrates. As is often the case in matters of intellectual importance, the Greeks can provide guidance. For example, in Aristotle’s (1984) Nichomachean Ethics (trans. 1894), three types of knowledge/intellectual virtues were described: episteme, techne, and phronesis. As episteme refers to context independent knowledge of that which is universal and invariant, it typically falls outside the purview of psychotherapy research and will not be further considered. Techne, however, is knowledge of how to create something/complete a task. It reflects knowledge of how to envision and achieve a particular goal (e.g., how to teach a patient relaxation training in order to reduce anxiety). This is not to say that it disallows creativity, but in exercising techne, one will know and be directed towards the ultimate outcome in advance. It can be considered a rule following activity. As such, it involves fixed endpoints that are achieved in a relatively orderly sequence of intermediary activities. You know when you reach the goal, and you also generally know how you will achieve it. This could be the case in a stereotyped treatment manual possessing a number of fixed and sequential interventions.

Phronesis, on the other hand, is guided not by rules and pre-determined goals, but by values which serve as regulative ideals. It is “the excellence by which one deliberates well about what to do in the human realm. It is the process of reasoning used to make the appropriate practical choices that constitute a good life (Polkinghorne, 2004, pg. 111).” Thus, it can be considered a practical wisdom, or capacity to exercise good judgment that is reliant on the particularity of the situation (e.g., knowing when and how to make a dynamic interpretation that will be appropriate and helpful to a particular patient at a particular point in time). In contrast to techne, which can be considered an almost algorithmic and rule governed process, phronesis involves a continuous and mutually-influencing interplay between the process and the regulative ideals (i.e., values) guiding the process. It is fluid and open. Further, the endpoint in a value-driven enterprise may even change over time as the process of moving towards the regulative ideal unfolds. Endpoints in this case are flexible and context-dependent. To return this to the clinical realm, we suggest that skill in adhering to a treatment manual would be more reflective of techne. However, appropriateness and judgment used in applying treatments, manualized or not, would be indicative of phronesis. Therefore, the crucial difference between adherence and competence is that adherence demonstrates knowledge of “how” to intervene (i.e., one possesses knowledge of how to implement a particular treatment technique and is capable of doing so) and “what” to intervene on (viz., the therapist has the knowledge to identify particular problems requiring intervention such as depressive symptoms), whereas competence is knowledge of “when and where (and possibly why or why not)” to intervene.

As a side note, there is a parallel between good, competent therapy and the process of general scientific discovery. In contrast to popular conceptions, rule following does not always appear to work well for science, and certainly does not exhaust the procedures scientists utilize. The conception of the scientific method as an orderly application of sequential rules has been trenchantly critiqued by many, and has been found to be both historically inaccurate and insufficient to explain discovery (Kuhn, 1996; Feyerabend, 1975). Thus, a simplistic view of science as inherently rule following is somewhat naïve. A perusal of scientific breakthroughs reveals that non-adherence to a scientific method, serendipity, the utilization of extra-scientific criteria for guidance, and even outright non-rational thought have all had stronger roles to play in scientific discovery than is often acknowledged (ibid, Foucault, 1970). We believe that therapeutic breakthroughs can sometimes occur through similar, non-algorithmic processes. Further, good science and good psychotherapy appear to necessitate both the humble acceptance of uncertainties and the real fact that there are more elements occurring in any moment-to-moment interaction than could ever be completely captured by a finite individual. Manuals may risk engendering the false belief that a good answer is always available if you follow the rules. But this is obviously not always the case.

The techne-phronesis distinction is very important, as the relationship between adherence and competence may be highly complex (e.g., as seen in Barber, Gallop et al., 2008). In many situations, adherence and competence may be synchronous, and it may even be difficult to distinguish them (in fact, one study [Barber & Crits-Christoph, 1996] found them to be correlated at .58). However, we are quite certain that most practicing psychotherapists can identify examples from their own practices in which the most competent action to undertake with a particular patient did not correspond at all with what a manual would prescribe. In these (possibly uncommon) circumstances, maintaining adherence could in itself be viewed as incompetent due to the lack of responsiveness to the particularity of the situation (i.e., a lack of therapist flexibility/situational attunement).

What Should be Considered Incompetent?

As seems to be the case with competence, there is a tacit, “I know it when I see it” sense about incompetent therapeutic behavior. Causing harm to a patient through crimes of omission (e.g., not thoroughly assessing suicidality) or crimes of commission (e.g., seducing the patient) are clearly incompetent actions. Presumably, a high frequency of such behaviors would indicate a trait, and not just a state, of incompetence. However, Overholser and Fine (1990) discuss several other subtler domains of incompetence. Some of their categories, when considered within the context of our definitions above, would be more appropriately labeled non-adherence, yet several clearly refer to competence. For instance, Overholser and Fine’s category of “incompetence due to inadequate clinical skills (pg. 465)” encompasses difficulties forming alliances, providing too much advice, and excessive therapist self-disclosure. Similarly, incompetence can arise due to poor judgment. As we argue that the concept of intervention competence includes a high degree of professional judgment, no further explanation here is necessary. Finally, a psychologist can be incompetent due to the presence of disturbing interpersonal attributes (Wood et al., 1985). It is certainly not unheard of for therapists to be affected by their own psychopathologies/problems such that they occlude aspects of their patient’s interpersonal world (e.g., failing to take important contextual information into account); overly identify, personalize, or react to a patient’s situation due to unresolved issues; or even lack the ability to accurately empathize. A competent therapist knows their professional limitations, practices adequate self care, and knows their limits of personal comfort. However, self-care may not be as common as we would hope (Pope, Tabachnick, & Keith-Spiegel, 1987; Guy, Poelestra, & Stark, 1989).

What is the Meaning of Competence Displayed in “Fringe” Practices?

Our discussion of intervention competence also raises the question of what it means to be competent in non-mainstream, or what could be termed “fringe” clinical practices. As an example, what does it mean to be competent in an approach such as “rebirthing therapy?” Rebirthing therapy (Mercer, 2001) consists of swaddling an individual (usually a child) in tight cloth, holding them down, and simultaneously applying rhythmic pressure in order to simulate uterine contractions. When the individual is “reborn” in such a manner, this is thought to eventuate in the removal of negative emotions and increase his/her readiness to enter into healthier relationships. This practice achieved some degree of notoriety when it resulted in a child’s death (Crosson, 2000). Thus, the question is whether one would consider a therapist who is competent in the delivery of rebirthing therapy to be competent.

Presumably, any therapeutic modality with at least several adherents will have guidelines as well as some degree of training and peer review, even if both of these are conducted fairly informally. Thus, with consensus and operationalization, the possibility exists for the assessment of adherence and competence in any of these practices. In spite of this possibility, however, it seems somewhat odd to label clinicians as ‘competent’ who poorly serve the public by using idiosyncratic approaches having no empirical and (at best) questionable theoretical reason to believe these methods are valid or effective. However, it is difficult to create a firm boundary between fringe and non-fringe practices, and these judgments likely vary according to psychologist and/or therapeutic orientation. Nevertheless, for us it seems reasonable to assume that being a “competent” psychologist would include both an implicit ethical exhortation to choose the best type of treatment for your patient’s particular situation as well as an ability to justify treatment decisions with compelling empirical and/or clinical evidence.

Are There Different Meanings of Intervention Competence?

There also appear to be at least two meanings of intervention competence. Barber, Sharpless et al. (2007) recently elaborated upon two of these: global competence and limited-domain competence. The former is “the idea that a therapist possesses clinical acumen, and that competence pervades their interventions. It is the sense that a therapist appropriately and independently manages a number of clinical problems and can adequately help patients realize their treatment goals” (p. 494). This appears to be the type of competence that professionals have in mind when attempting to determine readiness for independent practice or readiness for internship. In contrast, limited-domain competence refers to competence expressed within a specific type of intervention (e.g., supportive-expressive therapy for depression). This type of competence may be what is desired for certain types of specialties (e.g., sex therapy certification), but not for clinical psychology as a whole and its competing orientations.

Does Intervention Competence Vary According to Orientation?

Psychology, even more so than other disciplines, involves a multiplicity of competing paradigms. This heterogeneity is perhaps even more evident in clinical and counseling spheres. If therapists possessing different orientations are truly operating within different paradigms (in the Kuhnian (1996) sense of the term), they would presumably be viewing clinical phenomena as differently as physical phenomena would be viewed by Einsteinian and Newtonian physicists. It seems reasonable to conclude that psychologists hoping to understand and also assess competence must come to terms with the ramifications of this fact. For instance, would a behavior therapist and a psychoanalyst view competence in exactly the same way? This seems unlikely, as fundamental therapeutic techniques of both orientations are in such stark contrast. Their respective therapeutic worlds would be viewed through different colored lenses. In fact, it is the opinion of the writers that the competent expression of techniques viewed from within the perspective of adherents of the same paradigm may sometimes be viewed as incompetent therapeutic behavior by the representatives of the other. For example, whereas both psychoanalysts and behavior therapists would agree that “exposure” to traumatic events is important for the treatment of trauma, the timing and judgment of when (and why) this should be accomplished may be discrepant. A behavior therapist would likely begin prolonged exposure by the third session (Foa et al., 1991), and waiting months before doing so would be considered incompetent, if not unethical. A psychoanalyst, on the other hand, would likely view such early exposure to a trauma as a re-enactment of the trauma, and therefore an incompetent (if not unethical) action. Although we acknowledge that examples such as this may be uncommon and that enough overlap exists between modalities to evaluate inter-modality competence on some level, we would argue that there are some incommensurable elements remaining between them.

Further, it seems reasonable to hypothesize that extensive knowledge in more than one type of therapeutic orientation/modality may allow a psychologist to “step outside” of his/her predominant orientation in order to assume a position of evaluation for another modality. For instance, even though we may characterize ourselves as psychodynamically-inclined psychotherapists, we have both been trained in behavioral and cognitive therapies and feel confident that we could accurately differentiate between competent and inferior expressions of these modalities in spite of the fact that they lay outside of our primary therapeutic orientation. Using our particular example, the knowledge of “why” may be more tacit, and the ability to clearly articulate the reasons for viewing a behavior as competent would be richer for more competent practitioners of that same paradigm. Further, there would likely be occasional instances where we would make judgments (as above) that some competent actions reflect incompetence.

How Might Intervention Competence Develop?

Another fundamental question about intervention competence concerns the nature of its development. Most would agree that it is a dimensional construct (although qualitative disjunctures in competence may exist) and that the development of competence “is an ongoing process within an individual that is, ideally, in a state of constant flux and renewal (Nagy, 2005, pg. 29).” Thus, development of competence is not an absolute and static notion possessing a fixed endpoint, but more a lifelong endeavor. However, we as researchers and practitioners are interested in establishing thresholds and benchmarks for competence and, consequently, must make important decisions about where to artificially demarcate or “carve nature at its joints.”

Given the fact that all professions possess notions of competent and incompetent behavior, we will avail ourselves of what other disciplines have had to say about this issue. In particular, we will discuss an important aspect of Dreyfus and Dreyfus’ (1986) landmark work (written from philosophical and computer science perspectives). In it, they describe the subtle (and not-so subtle) qualitative shifts that occur throughout the development of competence and present five stages from novice to expert. Though not directly linking their work with psychotherapy, we will reinterpret Dreyfus and Dreyfus’s five stages (1986, pp. 16-36) within this particular context and provide clinical examples relevant to both competence and adherence. Although Dreyfus and Dreyfus’s work has been adapted to other disciplines (e.g. nursing) and other psychologists have cited it (Hatcher & Lassiter, 2007), our review will differ in that we will interpret it in a purely intervention-based framework, will link it to the definitions above, and will attempt to discern several stage-specific clinical benchmarks for future empirical exploration. Before proceeding, though, we must provide the caveat that, as in all developmental models, stages are never as distinct in reality as they are in theory (i.e., not everyone moves through the same stages in the same order). Further, there is always the danger that stages may become conceptually reified. However, stages can also serve a heuristic value by indicating important developmental distinctions that warrant additional attention.

Stage 1: Novice

Psychotherapy novices, like all novices, could be said to operate in a state of rule learning and rule governedness. They spend time memorizing the objective “signs” and indicators of different clinical problems and acquire rules to govern their actions (e.g., If a patient has __, it means __, and I should do __). Time may even be spent formulating pat “responses” to certain comments or situations (e.g., how to manage patient requests for personal information). These various “rules” delimit the novice’s allowable range of behavior, and they tend to evaluate their performance based on adherence to these algorithms. As a result, an overall understanding of therapy (and their patients) is lacking, and their focus on context-free knowledge and rules may make it difficult for them to authentically encounter their patients. In supervision, they may list symptoms without providing any overarching coherence or sense of the patient. The novice stage is also limited due to concentration demands, as it is difficult to keep so many rules in mind. As Dreyfus and Dreyfus (1986) state, “the first rules allow the accumulation of experience, but soon they must be put aside to proceed (pg. 23).”

Stage 2: Advanced Beginner

Trainees at this stage have worked with a number of patients at different stages of therapy (e.g., beginning and termination stages), received supervision, enlarged their repertoire of “allowable” responses, and may have taken preliminary forays into several orientations. This real-world experience with context free facts allows them to perceive the similarities of new situations to old ones and also facilitates recall of the rules that should be applied. Thus, they can form more sophisticated rules and discriminate at higher clinical levels (e.g., knowing whether a depressed patient would benefit more from behavioral activation or cognitive therapy). One could say the rules have changed from “if __, do __” to “if __ is similar to this situation, do __.” This does not imply an explicit awareness of why the situations are similar, however, as there remains an ineffable quality to the advanced beginner’s perceptions. Most therapeutic actions remain deliberate (and deliberative), but simple actions can be undertaken alone (viz., supervision remains necessary). We would argue that achieving such a level of competence is appropriate for determining readiness for pre-doctoral internship.

Stage 3: Competence

The leap from stage 2 to 3 appears to represent a binary shift. First, there is a global rearrangement of clinical decisions. As a focus on context free elements eventually becomes a drain on attentional resources, prioritization is necessary, and organizational and hierarchical plans are adopted according to urgency of need and their relation to a superordinate plan or value (see also the automaticity literature, as in Bargh & Williams, 2006). The importance of a fact is now predicated on its relation to other facts. Thus, competent therapists distill clinical presentations to their essential features and treat patients according to their particular needs. Their communication about patients also changes. Whereas the patient narratives of less experienced therapists consist largely of isolated facts or situational similarities, those of competent therapists are likely shorter and less detailed, but more organized, nuanced, contextual, and coherent.

The second shift is towards an increased sense of investment and immersion. Whereas poor outcomes in stage 2 could be seen as resulting from inadequately specified elements or poor rules, stage 3 is characterized by feelings of personal responsibility due to the fact that plans for therapy are now largely personal choices and not just applications of disembodied algorithms. The sequelae of decisions are intensely felt even though these decisions are still made in a detached and deliberative manner. This sense of accountability is ethically important (Leach, 2002), and we believe that this should be the minimum stage required for independent practice.

Stage 4: Proficiency

Proficient therapists no longer base decisions upon detached and deliberative selections among alternatives. With copious experience and a deeper involvement in therapy, features of situations become intuitively graspable without recourse to rule following. “It just happens” that certain elements become salient while others recede, and no conscious deliberation seems to occur or be necessary. There is an increased responsiveness (Stiles, Honos-Webb, & Surko, 1998), as new clinical events modify the plans, expectations, and salience of other relevant features in a to-and-fro movement (viz., akin to a hermeneutic circle, Bernstein, 1983). However, the particular manner in which action is taken may be reflectively chosen and could break this sense of therapeutic attunement. For instance, the therapist intuitively grasps that a patient is ready to explore a thorny or difficult life issue, yet carefully/consciously chooses the needed words to move him/her in that direction.

Stage 5: Expertise

An expert’s skill, “has become so much a part of him that he need be no more aware of it than he is of his own body (Dreyfus & Dreyfus, 1986, pg. 30, see also Schön, 1983 and Polanyi, 1967 for other descriptions of tacit knowledge).” There is a fluidity present, and expert therapists act appropriately and seemingly without conscious choice of action (i.e., they just do “what normally works”). They view clinical problems in an immersed, not detached way, and respond not with rules, but with what experience has taught them. Deliberation may still be present, but is focused not on calculation, but on a critical evaluation of their “intuitions” (i.e., the collection of impulses towards action derived from what previous experiences in similar situations have taught them – some of which may be contradictory). When engaged in therapy, there is a palpable sense that they lose themselves in the experience. As one example from the often-cited “Gloria tapes” (Shostrum, 1966), Carl Rogers, when reviewing the session, comments that his own level of engagement made it difficult for him to remember particular facts of the session. Expert therapists are fully immersed, similar to a child at play (Gadamer, 1975). Further, their enormous breadth and depth of knowledge is inspiring to others.

Summary of the Development of Intervention Competence

What may be apparent in the progression through these stages is a movement away from a detached and rule governed psychologist consciously deliberating on “facts” about a patient and towards an embodied, experienced psychologist engaged in the moment with the full particularity of the individual before them. It is an increasingly fluid/flexible responding to important and tacitly-perceived nuances of the situation and what is best for the patient. Thus, it can be seen as a movement from an exclusive reliance on rules (adherence in the strict sense) to context, appropriateness, values, and accountability (viz., competence). However, this progression does not mean that one becomes “non-adherent” or ceases using rules, but merely implies the development of increased flexibility and responsiveness (i.e., adapting to situations that may be unexpected or which require an appreciation of the patient’s idiosyncratic context).

This progression has an obvious relevance for assessment. Assuming the model outlined above seems face valid and may be indeed reflective of how intervention competence develops, the field may be better served by evaluations which focus less exclusively on skills and knowledge assessment (e.g., standardized tests), and more on the development of sound, nuanced, and well-reasoned judgments in applying these skills. Of course, competence assessment ineluctably requires some level of skill assessment, as psychologists require skills to competently apply. However, we argue that the judgment and timing used in the moments of application are critical.

As will be outlined below, many approaches to measuring competence have been proposed, but few have been empirically applied, and no gold standards currently exist. There are many obvious challenges and dangers in assessing competence, not the least of which is ensuring fidelity to actual practice (Kaslow et al., 2007). Further, assessors must necessarily face the reality of multiple paradigms. The particular difficulties involved in this issue may be responsible for some of the resistances to assessment described by Lichtenberg et al. (2007).

On Measuring Intervention Competence

Assessing Patient Outcome

Measuring intervention competence through assessment of patient outcome has been discussed in the literature. It makes intuitive sense that psychologists with high levels of competence should evidence better patient outcomes than less competent psychologists. Though not a prevalent approach, examples in the literature do exist (Brown, Lambert, Jones, & Minami, 2004; Okiishi et al., 2006) and a more widespread application could be easily accomplished through the use of pre- and post-therapy assessments conducted by independent raters, patient surveys, or self-reports. Strengths of this approach include the fact that it could be easily and cheaply accomplished and also relies on direct feedback from patients (as therapists tend to be rather poor judges of their own abilities, as in O’Donovan, Bain, & Dyck, 2006). Most research studies already repeatedly administer measures, so outcome ratings of protocol therapists could be readily catalogued. General practitioners could utilize standardized or practice-specific batteries, and these batteries could be formulated to be either fairly specific (e.g. paraphilias) or general enough to use with all patients. A relatively large sampling of a psychologist’s ability to achieve various outcomes (or levels of endstate functioning) could be obtained in a short amount of time. In addition, this approach holds more promise for assessment of global competence than do others at this time, as it could conceivably capture information across different types of patients and situations (e.g., preliminary data showing consistency across patients can be found in Luborsky et al., 1997).

Though face valid (and likely to be readily endorsed by HMOs and insurance companies), this approach entails several notable problems. Primarily, it is more indirect than the others. What outcome in this case would represent is the sum effect of many patient and therapist variables, expectancy and maturation effects, and other factors that add “noise” to the competence rating obtained. Moreover, patient selection varies widely, with some psychologists seeking out the “tough cases.” Thus, measures of overall patient difficulty would need to be taken into consideration (Perepletchikova & Kazdin, 2005). From a practical standpoint, some practitioners may not be willing to assess (expose?) themselves in this manner. Further, except for general agreement that lower symptom levels are important, not all orientations agree on what constitutes a good therapy outcome. Different paradigms of psychotherapy (and the split between researchers and practitioners) make consensus difficult, and it seems to us that the assessment of outcome for a particular patient requires a particular theoretical framework. Finally, this definition of intervention competence runs the risk of circularity, as competent therapists are those with better patient outcomes, and better patient outcomes imply competence.

Clinical Portfolios/Dossiers

Psychologists’ various patient outcomes could also be collected into what could be termed therapist portfolios or dossiers. Although we are not aware of their use in the evaluation of intervention competence, they would allow for the collection and compilation of evidence about performance that, if updated regularly, could also indicate improvement over time (and, thus, may be useful in training programs). One of the strengths of this approach is that such dossiers could incorporate detailed narrative descriptions (written by the clinician) of patients’ treatments along with more objective outcome information (e.g., regularly administered assessment batteries or diagnostic information). With regard to the narratives, these portfolios could prove useful as a tool for assessing the complexity of case conceptualization/treatment planning which, as we hypothesized above, may also give information about level of intervention competence. Similar approaches have been fruitfully used in the educational and business areas of psychology where they have also been called reflective learning portfolios (e.g., Tillema, 2001).

Alliance

Another possibility for evaluating the competence of psychologists involves assessing the therapeutic alliance (Summers & Barber, 2003). Alliance is easily measured using several different scales (e.g., Working Alliance Inventory, Horvath & Greenberg, 1989), and has demonstrated reliable (yet fairly small) relationships with therapy outcome (Martin, Garske, & Davis, 2000). Moreover, it appears to be a ubiquitous aspect of treatment in therapies, especially in those modalities that emphasize interpersonal relationships.

However, using alliance as a measure of competence also possesses sizable limitations. First, by their very nature alliance measures only capture one aspect of competence, and would necessarily omit other information (the skillfulness with which one applies techniques, for example). Second, most patients rate alliance quite highly (e.g., Hatcher & Barends, 1996; Barber et al., 1999), a factor making the alliance’s absence more telling than its presence. This may limit the ceiling for this particular competence assessment, as it is not necessarily indicative of excellence in therapy. However, these two factors may make it useful as a benchmark. Further, in some cases a highly rated alliance may only be indicative of a patient idealizing their therapist, and a lower alliance rating may actually be a sign of stronger integrity/maturity on part of the patient.

Examinations such as the Examination for Professional Practice in Psychology (EPPP)

Passing the EPPP is a requirement for all psychologists seeking licensure in the US. It is a multiple choice exam that can be administered in 255 minutes (Rehm & Lipkins, 2004). However, in spite of its ease of administration and computerized scoring, it possesses notable difficulties. Due to the lack of criterion validity testing, the excessive heterogeneity of questions (e.g., only 15% of questions are focused on interventions), and the seemingly inappropriate test format for assessing subtle psychological constructs (viz., multiple choice answers based upon limited information), we are not optimistic that the EPPP as currently constructed will be useful for assessing clinical psychologists’ competence (Sharpless & Barber, in press). We raise particular issue with ethical items that are posed in this format, as the process and manner of reflecting upon ethical questions seems equally important as the actual action taken. However, we do not wish to imply that standardized testing is necessarily inappropriate. If formulated and conducted in a manner which provides sufficient contextual detail and allows for (and accurately captures) nuanced clinical judgments and behavior, standardized measures may prove useful. We recommend that such exams allow test takers to explain their answers and the reasoning behind their particular clinical decisions.

Measures of Intervention Competence Derived from Randomized Clinical Trials (RCTs)

As a recent article reviewed competence measures derived from the therapy outcome literature (Barber, Sharpless et al., 2007), only a summary of these findings will be presented here. All of the measures reviewed were found to assess limited-domain competence only, and the majority required raters who were experts in the type of psychotherapy that was assessed. Moreover, several measures were found to conflate competence and adherence. Also troubling was the limited across-site reliability information available as well as the specificity of the competence measures (assessing specific treatments for a specific disorder). Factors such as these led Barber, Sharpless et al. to call for the development of scales possessing better conceptual clarity that would be applicable to a wider range of therapists and situations.

These methods derived from clinical trials could also be combined with those from Industrial/Organizational (I/O) psychology. Although I/O psychologists obviously do not utilize the same definitions of core concepts that we outlined above and do not have the same assessment purposes (i.e., determining the competence levels of psychotherapists), there may be enough conceptual overlap to prove useful. For instance, 360 degree feedback assessments (e.g., Ward, 1997) could be used. These are being increasingly used in industry and possess some favorable validity evidence (Atkins & Wood, 2002). Adapted to clinical purposes, this method could include the rating of competence not only by recognized experts in the therapist’s modality, but also from clinical supervisors (if any), peers, clinical supervisees, the psychologist’s self ratings, and the psychologist’s patients. This may provide a richer perspective on competence.

Business as Usual Therapy vs. Therapy at the Extremes

We would like to raise another issue for exploration. Namely, would competence be more evident (and more easily assessed), during “business as usual therapy” (therapy where patient and therapist are both focused and engaged on the task at hand in a facilitative manner) or in the handling of the unexpected or extreme aspects of therapy (e.g., intense anger or patient attempts at seduction)? In other words, is it possible that competence is best indicated by an ability to be flexible, unflappable, and adaptable to the unusual (viz., able to “step outside the box”) while simultaneously being “therapeutic” and using appropriate techniques? In our experience, situations such as these appear to cast competence levels in sharp relief. During “business as usual” therapy, an expert therapist may appear similar to a non-expert. However, in situations that befuddle the novice or even the advanced beginner (e.g., how to respond to patients who begin to incorporate the therapist into a delusional system), an expert therapist responds without batting an eye, and can often turn adversity into progress (e.g., increase reality testing). Business as usual therapy may obscure the fact that a therapist lacks flexibility and adaptability. Of course, patients can always be surprising regardless of the level of expertise, but the frequency of such events likely lessens with experience (and as Schön [1983] aptly notes, surprises are not necessarily negative, but instead often lead to novel insights [pg. 328]). No data yet exist on this potential assessment domain, and it may be difficult to evaluate. One possible option may be the use of simulators of various sorts, and these will be discussed next.

Use of simulators

A method advocated by Roberts, Borden, Christiansen, and Lopez (2005) involves use of computer technology. In a parallel fashion to certain trends in medical education, standardized and/or simulated patients could be utilized in order to assess specific and general clinical skills (Leigh et al., 2007, for examples). Several problems exist with this approach, however. First, medical complaints do not always parallel the format of psychological complaints, and computerized simulations of psychopathology would probably require significant simplification. Second, the experience of witnessing a videotape/computer screen/actor is a different subjective experience than being in the room with a real and legitimately suffering individual. Use of “virtual reality” simulators may be able to make the experience feel more organic and natural, but we would argue that an important and authentic subjective component would still be missing. These two factors place realistic limits on the validity of this method for assessment, but do not take away from its potential as a didactic tool. As stated above, this may be a promising method for safely exposing trainees to situations at the “extremes” of therapy and could also be used didactically in order to model possible responses.

Assessment Centers

Another possible adaptation from the I/O literature would be the use of assessment centers (Bray & Grant, 1966) which, contrary to their name, are not places per se, but events. Assessment centers are widely used and have demonstrated validity (Woehr & Arthur, 2002). The hallmark of an assessment center is a standardized evaluation of behaviors based upon multiple job-relevant tasks rated by multiple judges (Ballantyne & Povah, 2004). These tasks/job samples are usually fairly wide-ranging in order to allow for the development of a fairly rich picture of individual performance. Conforming this method to an intervention framework, we could envision the clinical assessment of some of the options discussed above (e.g., simulated patient situations of various types as well as case conceptualization/treatment planning tasks) in order to systematically assess competence in various clinical situations. An obvious drawback of this approach would be the high costs in terms of time and money.

Multi-Trait, Multi-Method, Multi-Informant Method

In summary, no one method yet holds the distinction of being a gold-standard, and many have yet to be systematically evaluated. As with most psychological constructs, it seems reasonable to conclude that multi-method approaches are preferable (Kazdin, 1998) to any uni-method approaches because the sophistication of the construct (viz., intervention competence) is never going to be fully encompassed by our operational definitions. The methods detailed above may prove insufficient on their own, but we wonder if they each tap into a more general factor of competence as described in the previous two sections of this paper. This awaits further data. Similar to the movement in RCTs to include multiple indicators of “high endstate functioning,” using combinations of the above-mentioned approaches appears to be a prudent way to proceed. Prior to a general consensus on what this should entail, care and judgment are necessary for determining the appropriate methods for each modality of therapy and each specific purpose of assessment.

Lingering Questions about Assessing Intervention Competence

Clearly, more empirical work is required, and we will end this section by listing some problems that must be faced as part of a comprehensive understanding of the measurement of intervention competence. First, what is the acceptable level of competence for each desired purpose (e.g., when should a therapist be allowed to apply for internship)? Although work is currently being done to formulate benchmarks (e.g., Assessment of Competency Benchmarks Workgroup, 2007), this is a large question which betrays no obvious answers. However, we hope that our adaptation of Dreyfus and Dreyfus’s (1986) model of competence development may prove helpful. Relatedly, certain competent behaviors are easier to rate/evaluate than others, but encompassing and capturing the breadth of competence is an important task when considering benchmarks (Manring, Beitman, & Dewan, 2003; Kaslow et al., 2007). Second, how many observed instances of competent therapeutic behaviors would be required for a psychologist to be viewed as competent? Does assessment require multiple cross-sectional observations (samples of tapes) or longitudinal observations (full therapies from start to finish)? Regardless, it seems reasonable to conclude that any valid assessment of competence would require multiple and repeated observations. Third, must evaluators of competence be at an equal or higher level of skill than the ratee? Dreyfus and Dreyfus (1986) would seem to imply that recognition of competence is possible at the lower levels, but that the ability to explicitly articulate “why” is a skill that becomes more advanced with experience and training.

Recommendations for Engendering Intervention Competence in Future Psychologists

Providing suggestions for training future psychotherapists is a difficult venture, especially when data are lacking. There currently exists a plurality of program types and methods already slated with this task and, given the limited knowledge of how best to cultivate competence (if there is an objectively “best way”), this is probably a reasonable state of affairs. However, there may be pedagogical deductions from the reviews above that are applicable to future therapists. From a developmental perspective, Dreyfus and Dreyfus’s (1986) stages from novice to expert imply changes in the levels of rule usage and immersion in the therapy process. As for the former, trainees systematically learn rules and then learn when to follow, break, and modify rules. How can this best be accomplished? First, we as educators should ensure that students are presented with a wealth of source materials on theory and practice and should be encouraged to read widely and deeply. Presumably, this is already occurring in graduate programs. Through emphasizing the underlying principles of therapy and therapeutic change rather than just techniques alone (Flannery-Schroeder, 2005), novice trainees may be in better position for moving towards competence. In a similar vein, Rosen and Davidson (2003) argue convincingly that graduate training programs should devote less time to the promulgation of empirically-supported (and proprietary) treatments for specific disorders (a practice that may be more reflective of and conducive to the development of techne), and should instead focus on empirically-supported principles of change and the range of their potential applications (which we feel may better allow for the development and expression of phronesis).

Further, students should be taught those aspects of competence that have been demonstrated to be teachable. For instance, evidence indicates that the ability to form a good therapeutic alliance, while not an algorithmic process, is a learnable skill enhanced with training (Crits-Christoph et al., 2006). No training sites that we are aware of utilize alliance assessments in their student evaluations, but this could easily be remedied.

It also seems to us that competence could be augmented by increasing what could be termed the transparency of our discipline. Clinical psychology has the unusual distinction of being perhaps the only helping profession in which a member could be licensed without ever having witnessed a successful treatment from beginning to end. Though this would hopefully be a rare occurrence, it is possible nonetheless (note: psychoanalysts may be exempt from this charge if their training analysis was considered successful, and though we have not seen it explicitly stated in the analytic literature, this may be one implicit reason for the training analysis requirement). This is at least partly due to the secretive nature of therapy. In surgery, for instance, a surgeon’s craft can be demonstrated from start to finish within a large amphitheatre. The anaesthetized patient will not respond differently if 2 or 200 people were privy to the process. This is not the case in psychotherapy, and a veil of secrecy surrounds so much of our work and the presentation of our work. These needs for privacy and confidentiality conflict with the didactic methods traditionally used in other, related disciplines. In psychotherapy, process notes continue to be used in many programs, but are retrospective and vulnerable to witting and unwitting distortions. Audio and video-recorders, if widely utilized, could make didactics significantly more transparent and allow us to catalogue numerous exemplars of “good” (and bad) psychotherapy from start to finish. Thus, tape libraries of competent therapists (as advocated by Yager & Bienenfeld, 2003) could be compiled. A reason why this is not widely done is perhaps because no one thought it would be important to have a trainee observe an entire treatment. We suggest that this could be a great learning experience. Having such readily available “models” of psychotherapy would allow students to “learn the rules” and also learn when it is ok (and that it is ok) to deviate when context warrants it. However, care should obviously be taken to ensure that the lessons learned from videotapes are not converted into technical dogmas. Discussions of important therapeutic decision points (and alternate routes that could have been taken) in the course of these recorded sessions may also facilitate transitions between Dreyfus and Dreyfus’s (1986) first three stages. Finally, showing trainees real-world examples of extreme situations in therapy, and how competent therapists handled them, would emphasize the importance of flexibility and nuance (viz., phronesis) that might otherwise be missed by other pedagogical techniques (e.g., reading manuals). Role playing these situations may be helpful, but having trainees respond to actual clinical impasses may be more beneficial.

Further, videotaping trainee’s own sessions (preferably with both therapist and patient views) would allow for better and more accurate supervision. In RCTs, an actual recording of session content is the prerequisite for competence assessment (and in our experience, both patients and therapists quickly adapt to this arrangement). Independent raters could even be used in order to circumvent the potential biases engendered by the supervisory relationship (Sakinofsky, 1979). In addition, watching trainee’s sessions would allow for an early recognition of disturbing or non-therapeutic interpersonal attributes or psychopathological “blind spots.” However, it must be noted that video will not capture all of the many subtleties that are occurring in the therapy room and that absolute transparency of the therapeutic process is an unreachable goal. But these factors do not detract from their potential educational significance.

Technology could also be used to make immediate observation and feedback possible during a trainee’s therapy sessions. Use of dual microphones and earphone receivers would allow for supervisors to closely monitor the moment-to-moment therapy process as it unfolds from another room. When the trainee becomes “stuck” or confounded or begins to travel down non-therapeutic paths, the supervisor could provide direct in vivo feedback and suggestions to the trainee (family therapists have been using similar procedures for a long time (Gallant, Thyer, & Bailey, 1991). Thus, when therapy is progressing appropriately and patient behavior is able to be comfortably handled, supervision would be unnecessary. However, when events surpass what could be called the trainee’s ”therapeutic zone of proximal development,” he or she could benefit from the thoughts and techniques of an expert (Leiman & Stiles, 2001, for a discussion of importance of recognizing patients’ zones of proximal development when conducting psychotherapy).

In summary, there do not appear to be many shortcuts for developing competence. It is hopefully a lifelong project that is fervently and earnestly undertaken, as training and experience are obviously the primary vehicles for gaining clinical judgment and progressing through the outlined stages. The sections above, however, do seem to indicate certain competence benchmarks. For instance, the style of a trainee’s description of session detail implies the stage where he or she may be placed. The use of a hierarchical and parsimonious organization would be indicative of a higher stage than would a “shotgun” approach in which a multitude of facts are presented in a haphazard and disorganized fashion. In addition, the ability and skill with which one develops a therapeutic alliance appears to be an easily assessed benchmark.

Many questions related to the training of competent psychologists remain incompletely answered. One having sizable importance, though no consensus, is whether it is better to initially train psychologists in one paradigm or to simultaneously immerse them in several? This does not appear to have a straightforward answer, and it seems to us that persuasive arguments could be made for either position. For instance, it could be argued that it is important for beginning therapists to be introduced to the in-depth workings of one particular paradigm (especially in the beginning stages). Kuhn (1996) would warn us that this would necessarily place limits on what a person could “see,” but these same lenses also allow trainees to see the deep inner workings of theory. By immersing students in one theory, they will be able to “take it to the extremes” or “run it into the ground,” revealing both strengths and limitations of the paradigm itself. On the other hand, it could be argued that a sophisticated familiarity with multiple paradigms, and an ability to switch between them, could be associated with greater flexibility (both cognitively and technically) and an ability to view patients from several different (and possibly mutually exclusive) ways. Theorists who advocate either psychotherapy integration or eclecticism would likely view the latter option as more attractive (Norcross & Goldfried, 2005; Lebow, 2002, for reviews).

Conclusion

We have explored the concept of intervention competence, further differentiated it from adherence, and clinically elaborated upon an existing framework for its development. We hope that our psychotherapeutic adaptation of Dreyfus and Dreyfus’s (1986) model and its progressive stages (namely the movement from a caricatured “adherence” to nuanced contextual judgment and increased levels of accountability and immersion) may prove useful in formulating competence benchmarks. Additionally, we propose that such benchmarks should take into account intervention competence proper, and not merely rely upon factual knowledge and “knowing how” to do specific techniques (i.e., adherence). Further, the particular way in which clinicians organize and discuss their clinical material may serve as an important reference point for determining competence levels.

Several major competence assessment methods were also surveyed. Whereas all of these have relative strengths and probable utility for specific purposes, none yet developed are encompassing enough to be used as “stand alone” measures that can be widely applied, and additional data are needed for each. Given this current state of the art, usage of multiple approaches and the development of new methods appears reasonable.

Suggestions for educating competent psychotherapists were also made. Building on our earlier discussions of the techne-phronesis distinction as well as the work of Dreyfus and Dreyfus, we emphasized the important roles of experience, increasingly “transparent” supervision, and the ready availability of real-world exemplars of good (and bad) psychotherapy (e.g., watching entire treatments) in order to broaden and strengthen clinical judgment. Given that appropriate and reliable competence thresholds remain somewhat elusive and “fuzzy,” we also suggested emphasizing the broader aspects of psychotherapy (viz. principles and contextual responsiveness) in addition to techniques and to teach that which is teachable (e.g., listening skills and supportive interventions skills) to beginning therapists.

In closing, we have also attempted to emphasize two important themes throughout the sections of this paper. First, we have argued for the importance of capturing flexibility, contextual responsiveness, nuance, and immersion when assessing competence. Unfortunately, adherence and factual knowledge are much easier to assess and operationalize than competence proper. However, given the many potential implications for the field, it is important that we avoid any tendencies towards oversimplification or an exclusive focus on that which is easiest to observe and catalogue. Second, we have discussed the fact that clinical psychology is characterized by multiple competing (and possibly incommensurable) paradigms. The multiple implications of this fact necessitate a serious discussion of what this may mean and the ways in which this may impact or limit widespread competence assessment.

Acknowledgments

The authors would like to thank Shabad-Ratan Khalsa and Taylor DiGravina for their assistance in preparing this manuscript.

Footnotes

This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

References

American Psychological Association. Ethical Principles of Psychologists and Code of Conduct. American Psychologist. 2002;57:1060–1073. [PubMed]
American Psychological Association. Policy statement on evidence-based practice in psychology. 2005. Aug, Retrieved September 2^nd, 2008 from http://www2.apa.org/practice/ebpstatement.pdf.
Assessment of Competency Benchmarks Work Group. A developmental model for defining and measuring competence in professional psychology. 2007. Retrieved September 2^nd, 2008, from http://www.apa.org/ed/graduate/comp_benchmark.pdf.
Aristotle . Nichomachean ethics. In: Ross WD, translator; Barnes’ J, editor. The complete works of Aristotle. Vol. 2. Princeton, NJ: Princeton University Press; 1984.
Ballantyne O, Povah N. Assessment and development centres. 2. Aldershot, England: Gower; 2004.
Barber JP, Crits-Christoph P. Development of an adherence/competence scale for dynamic therapy: Preliminary findings. Psychotherapy Research. 1996;6:81–94. [PubMed]
Barber JP, Gallop R, Crits-Christoph P, Barrettt MS, Klostermann S, McCarthy KS, Sharpless BA. The Role of the Alliance and Techniques in Predicting Outcome of Supportive-Expressive Dynamic Therapy for Cocaine Dependence. Psychoanalytic Psychology. 2008;25(3):461–482.
Barber JP, Luborsky L, Crits-Christoph P, Thase ME, Weiss R, Frank A, Onken L, Gallop R. Therapeutic alliance as a predictor of outcome in treatment of cocaine dependence. Psychotherapy Research. 1999;9:543–73.
Barber JP, Sharpless BA, Klostermann S, McCarthy KS. Assessing intervention competence and its relation to therapy outcome: A selected review derived from the empirical literature. Professional Psychology: Research and Practice. 2007;38(5):493–500.
Bargh JA, Williams EL. The automaticity of social life. Current Directions in Psychological Science. 2006;15(1):1–4. [PMC free article] [PubMed]
Bernstein RJ. Beyond objectivism and relativism: Science, hermeneutics, and praxis. Philadelphia: University of Pennsylvania Press; 1983.
Bray DW, Grant DL. The assessment center in the measurement of potential for business management. Psychological Monographs. 1966;80(625) [PubMed]
Brown GS, Lambert MJ, Jones ER, Minami T. Identifying highly effective therapists in a managed care environment. American Journal of Managed Care. 2005;11:513–520. [PubMed]
Crits-Christoph P, Gibbons MBC, Crits-Christoph K, Narducci J, Schamberger M, Gallop R. Can therapists be trained to improve their alliances? A preliminary study of alliance-fostering psychotherapy. Psychotherapy Research. 2006;16(3):268–281.
Crosson J. “Rebirthing” halted after a death. Philadelphia Inquirer. 2000 May 22;:A5.
Dreyfus HL, Dreyfus SE. Mind over machine: The power of human intuition and expertise in the age of the computer. New York: Free Press; 1986.
Feyerabend PK. Against method. 3. London: Verso; 1975.
Flannery-Schroeder E. Treatment integrity: Implications for training. Clinical Psychology: Science and Practice. 2005;12(4):388–390.
Foa EB, Rothbaum BO, Riggs DS, Murdock TB. Treatment of post-traumatic stress disorder in rape victims: A comparison between cognitive-behavioral procedures and counseling. Journal of Consulting and Clinical Psychology. 1991;59(5):715–725. [PubMed]
Foucault M. The order of things. NY: Random House; 1970. Original work published 1966.
Gadamer HG. In: Truth and method. Weinsheimer J, Marshall DG, translators. New York: Continuum; 1975. Original work published 1960.
Gallant PJ, Thyer BA, Bailey JS. Using bug-in-the-ear feedback in clinical supervision: Preliminary evaluations. Research on Social Work Practice. 1991;1(2):175–187.
Guy JD, Poelstra PL, Stark MJ. Personal distress and therapeutic effectiveness: National survey of psychologists practicing psychotherapy. Professional Psychology: Research and Practice. 1989;20:48–50.
Hatcher RL, Barends AW. Patients’ view of the alliance of psychotherapy: Exploratory factor analysis of three alliance measures. Journal of Consulting and Clinical Psychology. 1996;64(6):1326–1336. [PubMed]
Hatcher RL, Lassiter KD. Initial training in professional psychology: The practicum competencies outline. Training and Education in Professional Psychology. 2007;1(1):49–63.
Horvath HO, Greenberg LS. Development and validation of the Working Alliance Inventory. Journal of Counseling Psychology. 1989;36(2):223–233.
Kaslow NJ. Competencies in professional psychology. American Psychologist. 2004;59:774–781. [PubMed]
Kaslow NJ, Rubin NJ, Bebeau MJ, Leigh IW, Lichtenberg JW, Nelson PD, Portnoy SM, Smith IL. Guiding principles and recommendations for the assessment of competence. Professional Psychology: Research and Practice. 2007;38(5):441–451.
Kazdin AE. Research design in clinical psychology. 3. Needham Heights, MA: Allyn and Bacon; 1998.
Kuhn TS. The structure of scientific revolutions. 3. University of Chicago Press; 1996.
Leach DC. Competence is a habit. Journal of the American Medical Association. 2002;287(2):243–244. [PubMed]
Lebow J. Integrative and eclectic therapies at the beginning of the twenty-first century. In: Lebow J, editor. Integrative and eclectic psychotherapies. Vol. 4 2002. Kaslow F, editor. Comprehensive Handbook of Psychotherapy. New York: John Wiley & Sons;
Leigh IW, Smith IL, Bebeau MJ, Lichtenberg JW, Nelson PD, Portnoy S, Rubin NJ, Kaslow NJ. Competency assessment models. Professional Psychology: Research and Practice. 2007;38(5):463–473.
Leiman M, Stiles WB. Dialogical sequence analysis and the zone of proximal development as conceptual enhancements to the assimilation model: The case of Jan revisited. Psychotherapy Research. 2001;11(3):311–330.
Lichtenberg JW, Portnoy SM, Bebeau MJ, Leigh IW, Nelson PD, Rubin NJ, Smith IL, Kaslow NJ. Challenges to the assessment of competence and competencies. Professional Psychology: Research and Practice. 2007;38(5):474–478.
Luborsky LA, McLellan T, Diguer L, Woody G, Seligman DA. The psychotherapist matters: Comparison of outcomes across twenty-two therapists and seven patient samples. Clinical Psychology: Science and Practice. 1997;4(1):53–65.
Manring J, Beitman BD, Dewan MJ. Evaluating competence in psychotherapy. Academic Psychiatry. 2003;27(3):136–144. [PubMed]
Martin DJ, Garske JP, Davis MK. Relation of the therapeutic alliance with outcome and other variables. Journal of Consulting and Clinical Psychology. 2000;68:438–450. [PubMed]
Mullen LS, Rieder RO, Glick RA, Luber B, Rosen PJ. Testing psychodynamic therapy skills among psychiatric residents: The Psychodynamic Psychotherapy Competency Test. American Journal of Psychiatry. 2004;161(9):1658–1664. [PubMed]
Nagy TF. Competence. Journal of Aggression, Maltreatment, and Trauma. 2005;11:27–49.
Nelson PD. Striving for competence in the assessment of competence: Psychology’s professional education and credentialing journey of public accountability. Training and Education in Professional Psychology. 2007;1(1):3–12.
Norcross JC, Goldfried MR, editors. Handbook of psychotherapy integration. 2. New York: Oxford University Press; 2005.
O’Donovan A, Bain JD, Dyck MJ. Does clinical psychology education enhance the clinical competence of practitioners? Professional Psychology: Research and Practice. 2006;36(1):104–111.
Okiishi JC, Lambert MJ, Eggett D, Nielsen L, Dayton DD. An analysis of therapist treatment effects: Toward providing feedback to individual therapists on their clients’ psychotherapy outcome. Journal of Clinical Psychology. 2006;62(9):1157–1172. [PubMed]
Overholser JC, Fine MA. Defining the boundaries of professional competence: Managing the subtle cases of clinical incompetence. Professional Psychology: Research and Practice. 1990;21(6):462–469.
Perepletchikova F, Kazdin AE. Treatment integrity and therapeutic change: Issues and research recommendations. Clinical Psychology: Science and Practice. 2005;12:365–383.
Polanyi M. The tacit dimension. New York: Doubleday; 1967.
Polkinghorne DE. Practice and the human sciences: The case for a judgment-based practice of care. Albany: State University of New York Press; 2004.
Pope K, Tabachnick B, Keith-Spiegel P. Ethics of practice: The beliefs and behaviors of psychologists as therapists. American Psychologist. 1987;42:993–1006. [PubMed]
Rehm LP, Lipkins RH. The Examination for Professional Practice in Psychology. In: Vaughn’s TJ, editor. Psychology Licensure and Certification: What Students Need to Know. Washington DC: APA; 2006. pp. 39–53.
Roberts MC, Bordern KA, Christiansen MD, Lopez SJ. Fostering a culture shift: Assessment of competence in the education and careers of professional psychologists. Professional Psychology: Research and Practice. 2005;36(4):355–361.
Rodolfa E, Bent R, Eisman E, Nelson P, Rehm L, Ritchie P. A cube model for competency development: Implications for psychology educators and regulators. Professional Psychology: Research and Practice. 2005;36(4):347–354.
Sakinofsky I. Evaluating the competence of psychotherapists. Canadian Journal of Psychiatry. 1979;24(3):193–205. [PubMed]
Schön DA. The reflective practitioner: How professionals think in action. London: Temple Smith; 1983.
Sharpless BA, Barber JP. The EPPP in the era of evidence-based practice. Professional Psychology: Research and Practice in press.
Shostrum EL. Three approaches to psychotherapy, Series I [Motion picture] 1965 Available from Psychological & Educational Films, 3334 East Coast Highway #252, Corona Del Mar, CA 92625.
Stiles WB, Honos-Webb L, Surko M. Responsiveness in psychotherapy. Clinical Psychology: Science and Practice. 1998;5(4):439–458.
Summers RF, Barber JP. Therapeutic alliance as a measurable psychotherapy skill. American Psychiatry. 2003;27(3):160–165. [PubMed]
Tillema HH. Portfolios as developmental assessment tools. International Journal of Training and Development. 2001;5(2):126–135.
Ward P. 360-degree feedback. London: Institute of Personnel Development; 1997.
Woehr DJ, Arthur A. A construct-related validity of assessment center ratings: A review and meta-analysis of the role of methodological factors. Journal of Management. 2003;29(2):231–258.
Wood B, Klein S, Cross H, Lammers C, Elliott J. Impaired practitioners: Psychologists’ opinions about prevalence, and proposals for intervention. Professional Psychology: Research and Practice. 1985;16:843–850.
Yager J, Bienenfeld D. How competent are we to assess psychotherapeutic competence in psychiatric residents? Academic Psychiatry. 2003;27(3):174–181. [PubMed]