We wrote an article entitled “Expertise in psychotherapy: An elusive goal” (Tracey, Wampold, Lichtenberg, & Goodyear, 2014), which has garnered several written (Hendlin, 2014; McMahan, 2014; Oddli, Halvorsen, & Rønnestad, 2014; Shanteau & Weiss, 2014) and oral reactions. We take this opportunity to clarify and expand our view. In our article, we argued that there is no evidence of expertise within the professional practice of psychotherapy. We focused our discussion on the profession itself, arguing that there was no relation between time spent in professional practice and improved practice. Among professional therapists, there is common lore that experience results in greater clinical “wisdom” and better skill. We found no support for this view with respect to clinical decision-making or clinical outcomes.

While this is a regrettable state of affairs, we do not see it as inherent to the profession. We then proposed that expertise can be developed with explicit feedback of clinical outcome in several ways: within clients over time (“Is this client improving relative to last session?”), across clients (“Is this client doing better than my other clients?”), and across therapists (“How do the outcomes that I obtain compare to those of other therapists?”). While we view this feedback as necessary, we do not view it as sufficient. There needs to be a reflective practice that accompanies clinical work and that such practice should manifest explicit hypothesis testing in a disconfirmatory manner, whereby common heuristics are minimized.

Thus our arguments focused on the profession rather than on the identification of expertise in individuals. Hendlin (2014) includes both definitions (profession and individual) when he takes umbrage at the implication that there are no therapists with expertise. There may indeed be individuals who have expertise but such a determination is different from our focus in the article. The issue of the determination of individual expertise is an interesting one, though, and merits review to clarify where it differs from and overlaps with the issue of the expertise of the profession.

Defining Individual Expertise in Psychotherapy

A therapist’s expertise has been variously defined or understood in terms of (a) reputation, degree attainment, professional distinction, and experience, (b) skill, competence, or adherence to a prescribed standard of performance, (d) clinical accuracy, or (d) outcomes—that is, success with clients. We will review each in turn.

Reputation, degree, professional distinction, or experience as expertise criterion

Reputation as an expert often is employed as a criterion for expertise. In studies of expert performance in psychotherapy, researchers often use nominations of peers (see, e.g., Burlingame & Barlow, 1996; Linehan et al., 2006; Jennings & Skovholt, 1999Skovolt & Jennings, 2004, as reputation is presumably derived from both skills and outcomes. Overholser (2010), in his attempt to identify the core elements of clinical expertise in psychotherapy, indicated that for a professional to be an expert “the professional must possess a terminal degree in the field… [and] the professional is visible in the professional community at a national level” (p. 131). Unfortunately, reputation, degrees, and professional recognition may not relate to performance. For example, therapists others view as experts have been found to fare no better than non-experts in the accuracy of their decisions (Faust, 1991; Faust & Ziskin, 1988; Garb, 1998, 2005; Witteman, Weiss, & Metzmacher, 2013).

Professional experience also has been used in definitions of expertise. Overholser (2010) defines the expert, in part, as a “professional [who] has accumulated multiple years of clinical experience in the direct provision of clinical assessment, psychological testing, or psychological treatment” (p. 131). We noted that there is little evidence of the superiority of more experienced therapists (e.g., Berman & Norton, 1985; Beutler, 1997; Beutler et al., 2004; Budge et al., 2013; Hattie, Sharpley, & Rogers, 1984; Laska, Smith, Wisclocki, & Wampold, 2013; Minami et al., 2008; Okiishi et al., 2003; Stein & Lambert, 1984, 1995; Strupp & Hadley, 1979; Wampold & Brown, 2005). So expertise definitions that include reputation, degree attainment, and experience are insufficient.

Task performance as expertise criterion

Many definitions of expertise stipulate that experts are those who perform various tasks with a high level of skill. Overholser (2010) made such a stipulation: “The [expert] professional has demonstrated evidence of superior clinical skills in a specific application of psychology” (p. 131). Research on aspects of psychotherapy often identifies experts according to these criteria, such as using psychologists who have been awarded diplomate status by the American Board of Professional Psychology (e.g., Eugster & Wampold, 1996; Tracey, Lichtenberg, Goodyear, Claiborn, & Wampold, 2003). Of course, this is general and others have been more specific, usually referring to the core skills as competencies or the psychotherapist who has the skills as being competent. This definition of expertise, that of matching standards of practice, is also advocated by Shanteau and Weiss (2014) in their comment on our article.

Another example would be those who advocate specific treatments and emphasize that expertise depends selecting and administering the optimal evidence-based treatment protocol (Waltz, Addis, Koerner, & Jacobson, 1993). Most evidence-based treatments define what constitutes competence within such treatments. For example, Rector and Cassin (2010) have recently outlined the competencies necessary for expertise in delivering cognitive-behavioral treatments

The identification of core competencies of professional practice (e.g., Kaslow, 2004) provides criteria against which to assess therapist behavior. But these competencies are important for understanding expertise only if those who have these competencies demonstrate levels of performance greater than those with only basic levels of competence. That is, are therapists who possess the core competencies, or have higher levels of such competencies, more effective than others? Shanteau and Weiss (2014) note the inherent problem with defining expertise as adherence to established standards of practice using the example of expertise in astrology. While one may indeed be an expert with respect to adhering to astrology standards, one is no more accurate. When discussing the competencies, Kaslow (2004) noted,

Competencies are composed of knowledge, skills, and attitudes, which, as a coherent group, are necessary for professional practice. They correlate with performance, can be evaluated against well accepted standards, and can be enhanced through training and development. (p. 775, emphasis added)

However, the literature that attempts to identify core competencies rarely, if ever, relies on evidence that the competencies are causally, or even correlationally, related to outcomes. It appears that (a) experts’ ratings of competence after observing therapists’ performance with clients do not predict outcome in general (Webb, DeRubeis, & Barber, 2010), although they did find some support for the treatment of depression (Wampold & Imel, 2015); (b) adherence to clinical protocols (i.e., degree to which the therapist follows the treatment manual) appears to be unrelated to outcome (Webb et al., 2010; Wampold & Imel, 2015); and (c) strict adherence to those protocols might even attenuate therapeutic outcomes (Castonguay, Goldfried, Wiser, Raue, & Hayes, 1996; Henry, Strupp, Butler, Schacht, & Binder, 1993). So there is little basis for the validity of using task performance as a criterion of expertise.

Clinical accuracy as expertise criterion

Much of the research on expertise in psychotherapy has concerned therapists’ cognitive accuracy in certain clinical task domains—specifically, their clinical judgment, clinical decision making, and case formulations (Betan & Binder, 2010; Eells et al., 2005; Faust, 2007; Garb, 1989; Spengler et al., 2009; Wierzbicki, 1993)—and there is little support that skill increases with greater experience. For example, Witteman et al. (2012) found that master’s level students were better at diagnosing depression than experienced therapists. In fact, Shanteau’s (1992) determination that clinical psychologists do not demonstrate expertise was made in part on the basis of the accuracy of their clinical judgment. Perhaps it is relevant as well that mechanical, actuarial approaches to prediction consistently outperform psychologists’ clinical judgments (Meehl, 1954; 1986), though the extent to which this is true depends on the type of data being employed in prediction (Grove, Zald, Lebow, Snitz & Nelson, 2000). Despite the lack of evidence regarding the relation of experience and clinical decision making, there has never been a demonstration that more accurate clinical judgments are related to better outcomes in psychotherapy. While this connection appears logical and necessary, it still remains to be empirically evaluated.

Clinical judgment and decision making also has been discussed with respect to experts’ thinking processes. That is, experts should perform cognitive tasks differently than novices (Bereiter & Scardamalia, 1986), at least with respect to reasoning about and solving domain-relevant problems. Oddli et al. (2014), in their comment on our article, focus on this by noting that there is a body of literature that demonstrates that more experienced clinicians think differently from less experienced clinicians (e.g., Huppert et al., 2001; Kivlighan & Quigley, 1991; Martin et al, 1989; Miller, Hubble & Duncan, 2008; Ronnestad & Skovholt, 2013). While this may be true, there is no direct connection between the different processes used and better outcome. Indeed, with respect to expertise in general, these cognitive differences can cause non-experts to match or out-perform experts (e.g., Ericsson & Lehmann, 1996). Given this, any definition of individual expertise needs to incorporate information on actual performance as indicated in outcomes.

Outcome as expertise criterion

Client outcome might seem the ultimate definition of expertise. Indeed, it would seem curious to claim expertise for any therapist who did not reliably produce exemplary client outcomes. The classic expertise research on chess masters (Chi, 2006) identified those individuals who were extremely skilled in the domain (i.e., those who had long histories of winning) and then examined how these winning individuals approached the game and thought about it differently from those who were not as skilled. Hence strong outcomes are essential in any definition of expertise as it is a means of identifying those who are skilled.

Achieving consistently better outcomes across a range of clients is one means of defining expert therapists (Wampold & Brown, 2005). Baldwin, Wampold and Imel (2007) found that therapists who generally have better outcomes also are better able to form alliances with their clients. However, recent research (Budge et al., 2013) has found that, even using client outcomes, experienced psychologists actually perform worse than intern/postdocs. Based on a longitudinal study of 173 therapists treating over 6000 clients over a period of an average of 5 years, it appears that over time therapists gradually achieve poorer outcomes (Goldberg et al., submitted). Thus there appears to be little relation between professional experience and increased skill when client outcome is used as the basis of skill definition.

But there are difficult definitional issues even with outcome as the expertise criterion. It is recognized, for example, that outcomes of psychotherapy are due in large part to client variables, including severity of dysfunction, diagnosis, motivation (e.g., stage of change), social support, and resources (e.g., Bohart & Tallman, 2010; Groth‐Marnat, Roberts, & Beutler, 2001). So the issue of how to use outcome information is important. Caseloads vary in client difficulty (e.g., more experienced clinicians often carry more difficult clients), so any mean outcome may not take this into account. Caseloads will also vary by predominant diagnoses. Can outcome scores be compared across clients with depression, personality disorders, and substance abuse?

The possibility of needing separate outcome cutoffs for each diagnostic grouping may seem warranted but this also makes it difficult to make determinations of who is an expert. The metric of outcome varies. This could be taken to imply that expertise should only be determined within diagnostic grouping such that one would get a mean outcome score for work with clients with a diagnosis of depression (e.g., the work Minami et al., 2008, who focused on benchmarks for depression outcomes) and perhaps another one with clients with a diagnosis of borderline personality disorder. Of course, the metric could be standardized such that comparisons can be made (or aggregated) across different diagnostic groups, although such a procedure would obviously require an adequate set of base rates against which to compare the outcome of any individual’s caseload. Unfortunately, there are no such norms of group outcome readily available to clinicians, much less quality information on the outcomes of individual clinicians themselves.

In conclusion, there are issues with any definition of individual expertise in psychotherapy. These are emblematic of the difficulties of the development of expertise: If it can’t be defined, then it is difficult to say when it has been attained. But with respect to any definition of expertise, we see actual outcomes as a necessary ingredient and more work is needed to specifically examine expertise at both the individual and profession level.

Expertise of the profession of psychotherapy

We are concerned about the lack of empirical demonstration of expertise in the profession as a whole. Hendlin (2014) expressed outrage that we argued that there is no empirical support for expertise. He felt that our comments were insulting and erroneous and that better research is needed to demonstrate that experience leads to expertise. We agree that more research is needed but differ in that we do not dismiss the research that exists. Shanteau and Weiss (2014) offered two criteria for establishing expertise: coherence and correspondence. Coherence is the agreement with theory/common practice; correspondence is the agreement with an external reality. Psychotherapy would demonstrate expertise if the coherence criterion was adopted, according to Shanteau and Weiss. It is probable that over time clinicians become more aligned with prevailing practice norms or theories. The field fails, however, with the lack of demonstration of correspondence: We have not demonstrated the relation of professional experience with increases in outcomes (the external reality). While agreeing that psychotherapy does not demonstrate expertise using the correspondence definition, Shanteau and Weiss believe that the adoption of the coherence criterion may be appropriate for our profession.

On this we disagree strongly. In our opinion, the correspondence criterion is the only appropriate one and we should use clinical outcomes as the external reality/standard against which to evaluate expertness. At the same time we do not think that the failure to demonstrate this correspondence relation to date must be accepted as an established fact. In this regard, we made explicit arguments for feedback elements that could (and should) be incorporated into practice to enhance the development of expertise of the profession as defined by correspondence with outcome. And we argued further that a lack of quality feedback about what is done in therapy and its clinical outcomes and how such feedback information is used are the sources of limits to establishing expertise in psychotherapy.

Improving expertise

We posited that for the attainment of expertise in psychotherapy it is necessary for therapists to obtain quality information about outcomes. It is still relatively atypical for therapists to obtain quality psychometrically supported information regarding their work with clients. With respect to distal outcomes, such as assessments at termination and 6 months to one year later, assessments are quite rare. As a result, therapists have little knowledge (other than perhaps if the client returns in the future) of eventual outcomes and thus there is no corrective information on impressions gleaned at termination. More information on distal outcomes is essential both with respect to within-therapist comparisons (which are one’s best and worst cases) but also with respect to normative comparisons (how are one’s outcomes relative to those of others?). Such information is rarely provided but is crucial if there is to be any learning from experience.

Furthermore, proximate, session by session information is also important (e.g., symptomology prior to each session and alliance assessments after each session). While there are some good programmatic efforts to study the provision of outcome feedback (both proximate and distal) to the therapist (Duncan, Miller, Wampold, & Hubble, 2010; Lambert et al., 2005; Lambert & Shimokawa, 2011; Miller, Duncan, & Hubble, 2005; Reese, Norsworthy, & Rowlands, 2009), this is just beginning and not yet widely adopted. And while the provision of such feedback is related to better outcomes, it has not been shown to lead to better outcomes over time for a given therapist. As a consequence, although quality outcome feedback (both proximal and distal) is necessary for the development of expertise, we do not see it as sufficient.

One of the findings from the expertise literature is that expertise develops with deliberate practice (Ellis, Carette, Anseel & Lievens, 2014; Ericsson, 2006, 2009). It has been argued (e.g., Miller, Duncan, Sorrell, & Brown, 2005; Miller, Hubble, Chow, & Seidel, 2013) that dedicated time should be set aside for the examination of practice. In this regard, Chow et al. (in press) have found that whereas experience itself did not predict outcomes, the amount of time spent targeted at improving therapeutic skills did. However, the specifics of what should be incorporated into this deliberate practice aimed at improving skills are vague. We specified several aspects that should characterize this deliberate practice (i.e., adopting a disconfirmatory approach, explicitly testing hypotheses, and avoiding heuristics).

Incorporation of these strategies into deliberate practice should increase the chances of learning from the feedback embedded in outcome information. Our article focused on the lack of relation of experience and expertise and possible mechanisms to ameliorate this. Specifically, we called for much more quality outcome information (proximal and distal) and suggested that conscious attempts to use this information in deliberate ways (e.g., minimizing heuristics) would increase the relation of expertise and experience.

Some reactions to our article focused on practices that might lead to skill acquisition that in turn would lead to expertise. McMahan (2014) noted that in our paper we ignored supervision, which is the most widely used mode of improving practice. We see supervision as an important vehicle for obtaining and processing feedback (a key issue). But most supervision is performed while in training and our comments focused on development after training where there is much less supervision provided. In addition, whereas there is some evidence that supervision positively affects client outcome (Bambling, King, Raue, Schweitzer & Lambert, 2006) there also is recent research showing that there are no differences in client outcome across supervisor (Rousmaniere, Swift, Babins-Wagner, Whipple, & Berzins, 2014). Perhaps the issue should not be one of increasing supervision post degree but rather of examining the key elements within supervision that lead to better outcomes. It is our assumption that these too will be similar to our components of deliberate practice.

Oddli et al. (2014) viewed our recommendations for deliberate practice as a bit too narrow in that they would exclude implicit reasoning and intuitive practice decisions. We agree that these are subtle and potentially important aspects of cognitive development; but, without specifying new ways that these can be increased or honed, it is an example of proposing exactly what we already have. How specifically can these context-dependent decisions be enhanced and increased, and would this lead to better outcomes? Further, it is crucial that these intuitive decisions be separated from the common heuristics associated with such processing, because the errors of such processes have been well demonstrated (e.g., Dawes, 1994; Garb, 1997, 1998, 2005). There is some emerging research in this area of logical intuition (e.g., De Neys, 2012) but its application to psychotherapy awaits.

Admittedly psychotherapy is a difficult profession in which there are no clear answers and for which the definition and identification of expertise may be contentious. But we see our practices as open to improvement with careful attention and empirical evaluation. Stronger research-based attempts to help ensure that experience is used appropriately and results in increased skill and improved outcomes are needed. Our recommendations are in line with this view. That said, we do not think that ours are the only potential solutions to gaining expertise as a therapist, but believe that they are promising given the empirical base garnered from other areas.

+6

0

Terence J. G. Tracey, Ph.D., ABPP