At the time of this writing, an estimated one-third of the world’s population is in “lockdown” due to a novel coronavirus (Kaplan et al., 2020). In addition to the loss of life and physical health consequences, the economic and psychological impact of the virus and these containment measures has already been massive and may well be felt for years to come (Brooks et al., in press; Jones et al., 2020).

It would seem tone-deaf to not acknowledge the widespread human suffering at this moment in history, whether in the form of illness, unemployment, or racism (Bui & Wolfers, 2020; Tavernise & Oppel, 2020). This period represents perhaps the most rapid change in human social behavior that has ever occurred (D. Coogan, personal communication, March 27, 2020) and may initiate a mental health crisis (World Health Organization, 2020).

The current and impending psychological pain highlights the hugely important role that mental health providers play. Psychotherapy and psychotherapists will be essential for the collective healing from this pandemic. Thankfully, there is longstanding evidence that psychotherapy is generally effective (Smith & Glass, 1977; Wampold & Imel, 2015). However, there is room for improvement, and unfortunately, many do not benefit (e.g., due to premature termination; Swift & Greenberg, 2012). The questions then become: How can we make psychotherapy more effective? And, how can we best support our patients in these times and in all times?

Most readers of the Psychotherapy Bulletin are likely already convinced that the therapist matters. But this is not just our bias; there is strong scientific evidence that this is the case. Two recent meta-analyses have sought to quantify the degree of variance in patient outcomes attributable to the therapist (i.e., the therapist effect; Baldwin & Imel, 2013; Johns et al., 2019). Both studies found that therapists account for approximately 5% of the variance in patient outcomes. Although 5% may seem small, bear in mind that this effect size is not wildly different from that linking therapeutic alliance and outcome (i.e., 7.7%; Flückiger, Del Re et al., 2018) and is roughly five times greater than that attributable to differences between psychotherapies (i.e., 1%; Wampold & Imel, 2015; Wampold et al., 1997).

Research estimating the size of the therapist effect has been essential for advancing our scientific understanding of psychotherapy. Yet simply saying that therapists account for variance in patient outcomes is akin to saying that diet accounts for variance in health. One still needs to know what to eat or not eat. Crucial next steps for work in this area include determining precisely what therapist characteristics predict this variation and, ultimately, how these therapist-level factors can be selected for and augmented through training.

Although consensus has not been reached on the characteristics of highly effective therapists (Wampold & Imel, 2015), psychotherapy researchers have been investigating the relationship between therapist characteristics and patient outcomes for decades. In the past 40 years, many therapist-level variables have been explored, from therapists’ personal therapy (Garfield & Bergin, 1971) and performance on cognitive tasks (Mintz et al., 1976) to interpersonal skills (Anderson et al., 2009) and professional self-doubt (Nissen-Lie et al., 2013). This literature has been systematically reviewed in two recent studies. Lingiardi and colleagues (2017) examined the association between therapists’ subjective and therapy-nonspecific characteristics with patient outcomes in psychodynamic psychotherapy. Across 30 studies, they found the most consistent evidence for therapists’ interpersonal skills and interpersonal functioning. Patients of therapists who were more affiliative and more interpersonally skilled tended to have better outcomes. Heinonen and Nissen-Lie (in press) reviewed 31 studies examining therapists’ professional and personal characteristics, finding performance-based measures of professional interpersonal skills to be one of the most consistent predictors of patient outcomes. Self-rated social skills, in contrast, did not predict outcomes.

The studies reviewed by Lingiardi and colleagues (2017) and Heinonen and Nissen-Lie (in press) provide an important foundation for clarifying what may account for the therapist effect. However, this literature has substantial shortcomings. Both systematic reviews remarked on methodological limitations, including the use of small samples (of therapists and patients per therapist; Heinonen & Nissen-Lie, in press) and the wide variability in predictors examined (Lingiardi et al., 2017), with an associated small number of direct replications. In addition, with some notable exceptions (Anderson et al., 2009; Schöttke et al., 2017), many studies used self-report measures with known biases (e.g., social desirability). Further, variation in analytic methods and inconsistent use of standardized effect sizes makes it difficult to determine the consistency or magnitude of the association between therapist factors and patient outcomes. At the end of the day, we, as a field, remain without clear empirical guidance specifying precisely which therapist characteristics are most important. The current study is designed to address some of these limitations in the hopes of clarifying measurable therapist characteristics that relate to treatment outcome.

Method

Participants

In partnership with Drs. Jesse Owen and Mark Kopta at CelestHealth, recruitment has occurred through clinics using the Behavioral Health Measure (BHM; Kopta et al., 2015). In addition, we have partnered with Drs. Robbie Babins-Wagner and Amy Bender at the Calgary Counselling Centre. All staff therapists and trainees at partnering sites are eligible to participate in the study. Data collected from therapists will later be matched with patient outcome data routinely collected at participating sites. We aim to recruit ≥ 100 therapists whose responses will then be associated with data from their patients (anticipated patient n ≥ 1000). This target sample size of therapists would provide adequate power (beta = .80) based on a three-level power analysis that assumes a therapist intraclass correlation of .05 (Baldwin & Imel, 2013), 10 patients per therapist, 8 sessions per patient (Goldberg et al., 2018), and a small to moderate effect size (r ≥ .11, d ≥ 0.22).

Procedure

Therapist recruitment is occurring through emails sent by clinic directors at partnering clinics. Study staff direct participants to an online survey that includes approximately two hours of self-report questionnaires and behavioral tasks. Participants are compensated $200 for their time.

Measures

In planning this study, we aimed to collect a wide variety of therapist variables that have been previously shown to predict patient outcomes or are theorized to be linked with patient outcomes. Recent reviews of the literature on therapist characteristics and patient outcomes highlight therapists’ interpersonal skills as one of the more consistent predictors. Therefore, a primary measure we are collecting is the Facilitative Interpersonal Skills task (FIS; Anderson et al., 2009).  The FIS involves collecting participants’ verbal responses to a series of video-based vignettes depicting challenging moments in therapy. Responses are then coded along several domains (e.g., empathy, emotional expression). We have also included self-report and behavioral measures designed to assess component parts of the FIS (e.g., verbal fluency, empathy).

A second primary task we are using is the Multicultural Orientation task (MCO; Owen et al., 2018). Similar to the FIS, the MCO involves viewing a series of video-based vignettes to which participants are asked to respond. These vignettes depict multiculturally salient moments in therapy. Just as for the FIS, we have included other self-report and behavioral measures to more richly assess participants’ multicultural orientation.

Data Analytic Plan

Data will be analyzed using multilevel models that account for the nesting of patients within therapists. In these models, therapist characteristics will be entered as predictors of patient outcomes (e.g., change in outcome measures, early termination). We plan to examine potential interactions between patient demographics and therapist characteristics as predictors of outcomes through random slope models (e.g., Thompson et al., 2018). Further, we plan to use these data to continue the development of machine learning algorithms to automate the scoring of the FIS (Goldberg et al., in press). In keeping with recent efforts to increase transparency and replicability in psychology and science broadly, we plan to pre-register primary study hypotheses before data analysis (Open Science Collaboration, 2012).

Results

We are still actively in the process of data collection for this study. As we have not yet pre-registered our study hypotheses, we have not begun analyzing our data. Nevertheless, we have learned a lot through the design and initiation of this study.

The Importance of Collaboration and Clinical Partners

This project, like so many, has resulted from the combined efforts of many individuals. The individuals noted above and in the acknowledgments have provided a great deal of encouragement and feedback that have been vital for the design and execution of this study.

As discussed elsewhere (McAleavey et al., 2015), partnerships between researchers and clinics are oftentimes complex for a wide variety of reasons. One lesson we have learned through engaging in these partnerships is the central importance of clinic director buy-in and commitment to supporting and encouraging research among their staff. As evidence for this, we began our recruitment efforts by contacting clinic directors at 15 clinics using the BHM. Of these, therapists from five clinics completed the measures, resulting in a sample of 17 participants. Committed to obtaining a sample of therapists that would allow adequate power to detect small to moderate associations between therapist characteristics and patient outcomes, we then expanded our recruitment by reaching out to a clinic director with whom we have formerly collaborated. This expansion has resulted in an additional 40 participants (Figure 1).

Figure 1.

Data collection has likely been more successful since adding our collaborator’s site in part because it is a large clinic with many therapists. Many therapists at this center are in training and may, therefore, be more motivated by the compensation provided and able to devote two hours of their own time to complete our study. It also seems likely that recruitment has been successful due to the established partnership between the clinic director and our staff. This existing relationship may support the sense of shared mission and shared ownership that McAleavey et al. (2015) discussed as crucial for these partnerships.

Technical Partners

With our goal of collecting a multimodal assessment battery with a variety of self-report and behavioral measures, a second equally important partnership has been with a technical team. This partnership has involved collaboration with developers of the performance tasks noted above. In addition, developing our assessment delivery platform was a protracted process, during which we received guidance from programmers with expertise in online behavioral tasks. We conducted several rounds of pilot testing of our platform and made a series of changes based on the feedback we received. We established a collaboration with Theravue.com in order to use their electronic deliberate practice platform for delivering the FIS and MCO online. While the two-hour battery remains a substantial time commitment, we have found that most participants are able to complete the assessments without technical issues. The piloting process delayed the initiation of data collection somewhat, but we are hopeful it will ultimately produce a larger sample, cleaner data, and a more robust scientific product.

Discussion

The current study follows in a long tradition of psychotherapy research seeking to identify therapist characteristics that predict patient outcomes. This work aims both to replicate previous findings by evaluating measures that have been previously linked to patient outcomes (e.g., FIS; Anderson et al., 2009) and to extend this research by addressing several methodological shortcomings of the existing literature (e.g., small sample sizes, reliance on self-report measures, limited range of constructs assessed).

We hope that this study can help clarify what it is that makes effective therapists effective. In other words, we hope to find out what predicts the therapist effect. Of course, this is only an initial step towards the ultimate goal of reducing the burden of mental illness. Depending on what this study and similar efforts reveal, we may uncover certain capacities that could be selected for at the start of graduate training (e.g., through administering an interpersonal skills task as part of the admissions process). Other characteristics could be augmented through targeted training (e.g., mindfulness skills; Pereira et al., 2017) or built into training programs (e.g., deliberate practice; Chow et al., 2015). And yet other characteristics may interact with patient factors and could be used to support efforts to match therapists and patients, providing treatment that considers patient variability (i.e., precision medicine; Collins & Varmus, 2015).

A primary limitation of the current study is its correlational nature. Even if we are able to identify a clear set of therapist characteristics, they will simply be those associated with patient outcomes. Thus, a key future direction will be establishing a causal relationship between potentially malleable therapist characteristics and patient outcomes. Future randomized controlled trials could examine interventions aimed at these characteristics. Results from these experimental studies could then inform training efforts. This may be an ambitious goal, but certainly not outside the realm of possibility for the next decades of psychotherapy research. And, most importantly, these efforts may provide an empirically grounded route for improving the treatment outcomes and lives of our patients, during whatever challenging times the future may hold for human beings on this planet.

Author's Note: The author would like to thank the Society for the Advancement of Psychotherapy for support of this research through the 50^th Anniversary Research Grant. The author is grateful for ongoing discussions with William Hoyt, Jesse Owen, Tony Rousmaniere, Zac Imel, David Atkins, Timothy Anderson, Betty VanLeuven, Bruce Wampold and others regarding this study and its design. The author is grateful to Wing Ng, Dan Fitch, and Robin Goldman for their assistance building the data collection platform and to Anthony Flynn and Ilsa Valdez for assisting with data collection. Correspondence regarding this article can be addressed to Simon B. Goldberg, Department of Counseling Psychology, 335 Education Building, 1000 Bascom Mall, University of Wisconsin, Madison, WI, 53706. Email: sbgoldberg@wisc.edu

+2

0