Ample research suggests that therapists differ in their level of effectiveness (Baldwin & Imel, 2013; Blow, Sprenkle, & Davis, 2007; Wampold, 2001). Even more striking is that therapist effects appear to be larger than treatment effects (Kim, Wampold, & Bolt, 2006; Lindgren, Folkesson, & Almiqvist, 2010). Moreover, therapist training, experience, and theoretical orientation do not appear to explain the majority of therapist effects (Beutler et al., 2013; Okiishi, Lambert, Nielsen, & Ogles, 2003; Stirman & Crits-Cristoph, 2011). Therefore, it has been hypothesized that therapists’ personal characteristics may impact treatment (Heinonen , Linfdors, Laaksonen, & Knekt, 2012; Hersoug, Høglend, Havik, von der Lippe, & Monsen, 2009).

Hypotheses regarding what makes a good therapist often center on constructs such as intelligence (Shedler, 2006), empathic ability (Hill et al., 2008), interpersonal and attachment styles (Marmarosh et al., 2013), and history of personal therapy (Gold & Hilsenroth, 2009). Unfortunately, the empirical literature has largely ignored some of these factors and produced inconclusive or limited results for others. For example, a therapist’s empathic ability is theorized to be critically important (Rogers, 1957). Nonetheless, research on the degree that pre-screening measures of empathy can predict later therapeutic effectiveness is mixed (Hill et al., 2008; Moyers & Miller, 2013).

Thus, while there is considerable evidence that therapist characteristics influence the process and outcome of therapy, much more work is needed (Blatt, Sanislow III, Zuroff, & Pilkonis, 1996; Blow et al., 2007; Lebow, 2006). Greater understanding of which therapist factors are most important and the degree to which these factors are innate versus developed through training will have important implications for graduate school admissions criteria as well as types and methods of training. This study aims to contribute to this need by using a multi-method evaluation to assess students at the beginning of graduate school to determine which trainee characteristics predict later success in forming an alliance and implementing therapeutic techniques.

Method

Participants. Participants in the current study are graduate students enrolled in a Masters in clinical psychology program at a southeastern university. Presently, data have been collected from two cohorts of graduate trainees (N = 19). The participant group is currently 74% female (n = 14) with a mean age of 24 years (SD = 6.74). The racial composition of the sample was reported as 68% European American, 21% African American, 5% Asian American, and 5% biracial. Thirty-seven percent (n = 7) of the present sample endorsed having received therapy, with a mean time spent in therapy of 26 months (SD = 34.66). In addition, participants’ academic records indicated an overall mean undergraduate grade point average (GPA) of 3.57 (SD = 0.23), a mean quantitative Graduate Record Examination (GRE) score of 147.72 (SD = 7.63), mean verbal GRE score of 152.22 (SD = 5.85), and a mean analytic GRE score of 3.81 (SD = 0.57). Finally, at the time of this report, 16% (n = 3) of the sample had either withdrawn from, or been asked to leave, the program. For those participants failing to complete the program, three months was the average amount of time completed (SD = 1.73).

The undergraduate students serving as practice therapy clients are also recruited from the same university. The students who consent to participate in the sessions do so as one of the class project options in a course focused on personal growth and exploration. These students receive course credit for participating in the sessions and writing a reflection essay about their experience. Importantly, none of these practice clients know the researchers in this project and their professor is not provided any information about the therapy sessions other than that the students participated. Presently, the practice client group (N = 16) is 50% female, 50% European American, 25% African American, 12.5% Asian American, and 12.5% Hispanic. The mean age of the group is 20 years, (SD = 2.13). Moreover, 31% (n = 5) reported previous therapy experience, with the mean time spent in therapy of 15 months (SD = 58.38).

Multi-Method Assessment

At the beginning of their graduate training (2nd day of class of the 1st semester), all students enrolled in the Masters of Clinical Psychology program complete a multi-method personality assessment as part of the course work for their Personality Assessment class. The assessment battery is designed to assess the following individual characteristics: personal therapy experience, attachment and interpersonal style, empathy, and implicit dynamics related to self and others. A research assistant (RA) unaffiliated with the program administers, de-identifies, and scores the assessments. Importantly, all student responses are kept confidential and are not shared with anyone in the program (including the professor of the course). The following is a list of the measures used in this assessment:

Academic achievement. The participants’ college GPA and GRE data are acquired from the students’ applications to the program.

Personal therapy experience. Trainees complete a demographic questionnaire that also includes one yes/no question regarding whether they have been in personal therapy and, if so, for approximately how many months.

Experience in Close Relationships-Revised (ECR-R). The ECR-R (Fraley, Waller, & Brennan, 2000) is a five-point Likert-type self-report scale. It consists of 36 statements assessing attachment-related anxiety and avoidance in different types of relationships. The scale includes two subscales: Avoidance and Anxiety (Fraley et. al., 2000). The statements utilized in the questionnaire reflect worries about attachment-related concerns, as well as discomfort with intimacy. Research has confirmed the two factor structure of the measure as well as its temporal stability, with 86% of the variance shared between two administrations of the scale over a six week period (Sibley & Liu, 2004).

The Inventory of Interpersonal Problems-Short Circumplex (IIP-SC). The IIP-SC (Soldz, Budman, Demby, & Merry, 1995) is a 32-item four-point Likert-type self-report scale which assesses eight separate domains of interpersonal problems. The measured domains consist of four items each and yield eight scale scores: Domineering/Controlling, Vindictive/Self-Centered, Cold/Distant, Socially Inhibited, Nonassertive, Overly Accommodating, Self-Sacrificing, and Intrusive/Needy (Horowitz, Alden, Wiggins, & Pincus, 2003). The measure has been reported as having excellent overall reliability (r = .93) and moderate scale reliability, with Cronbach alpha coefficients for the eight scales ranging from .68 for Intrusive/Needy to .87 for Cold/Distant (Horowitz et al., 2003).

Interpersonal Reactivity Index (IRI). The IRI (Davis, 1980, 1983) is a 28–item, five-point Likert-type self-report scale. The measure evaluates four separate aspects of the global construct of empathy with items divided into the following four subscales: Perspective Taking; Fantasy; Empathic Concern; and Personal Distress (Davis, 1983). Research has provided evidence of convergent and discriminant validity of the IRI (Davis, 1983), as well as its four-factor structure (Pulos, Elison, & Lennon, 2004).

Thematic Apperception Test (TAT). The TAT (Murray, 1973) is a set of 32 black-and-white stimulus cards with stylized images depicting specific life scenes. Individuals are shown TAT cards and asked to make up a story in response to each respective card. These stories are then examined in an effort to draw conclusions regarding the respondent’s internal world. For this study, the following seven cards: 1, 2, 3BM, 4, 13MF, 12M, and 14, were administered in a group format in which the cards were projected on a screen and participants were asked to write their responses in a notebook. After the TAT was administrated, responses were transcribed, de-identified, and independently scored by two trained raters using the Social Cognition and Object Relations Scale-Global Rating Method (SCORS-G; Stein, Hilsenroth, Slavin-Mulford, & Pinsker, 2011; Westen, 1995). The SCORS-G is comprised of eight constructs which are rated using a seven-point Likert-type scale, where lower scores are indicative of more pathological aspects of object representations and higher scores are suggestive of more mature and adaptive functioning. The two expert raters used for this project had previously completed manualized training on the SCORS-G (Stein et al., 2011; Westen, 1995) and achieved “good” to “excellent” reliability on the SCORS-G in previous research (Stein et al., 2014).

Therapy Sessions

In the students’ second semester, all graduate trainees take an introductory therapy course. This four-credit course focuses on therapeutic technique with curriculum based on Hill’s three-stage model of helping as presented in Helping Skills: Facilitating Exploration, Insight, and Action, Third Edition (2009). As part of the requirements of the course, all students participate in a series of four practice therapy sessions with undergraduate student volunteers. The first session is a 1.5 hour intake and the remaining three sessions are 45 minutes and focus on whatever issues the client presents.

The practice therapy clients are told that they can use the sessions to work on whatever feels most important to them at the time. However, they are instructed not to share concerns which would necessitate an intervention by a licensed professional, such as suicidal or homicidal ideation, or child or elder abuse. Any clients presenting with these issues are directed to the campus counseling center. Common presenting problems have included difficulties in interpersonal relationships, anxiety related to school performance, and concern regarding choosing a career path.

All sessions are videotaped and the trainees receive supervision from the instructor who is a licensed psychologist. Following sessions one, two, and four, trainees receive 1.5 hours of group supervision (2-3 trainees per group). In addition, students receive 1.5 hours of individual supervision following session three. Supervision focuses heavily on the review of video-recorded case material with emphasis on conceptualization, process, interpretation, and clinical interventions.

Post Session Evaluation

At the end of the third session, clients fill out a measure to assess therapeutic alliance. In addition, the third session videotapes are rated for technique use by two trained raters. The following is a list of the measures used in this assessment:

Working Alliance Inventory, Client Form (WAI-C). The WAI-C (Horvath & Greenberg, 1989) is a 36-item seven-point Likert-type self-report scale designed to assess three facets of the therapeutic relationship: Task, Bond, and Goal (Horvath & Greenberg, 1989). Good reliability (Hanson, Curry, & Bandalos, 2002; Horvath & Greenberg, 1989) and construct validity of the WAI-C (Tichnor & Hill, 1989) have both been reported.

Helping Skill Measure (HSM). The first 13 items of the HSM (Hill & Kellems, 2002) capture basic exploration, insight, and action therapy skills using a five-point Likert format ranging from “strongly disagree” to “strongly agree.” For the purpose of this study, three additional items were added to assess the use of interventions intended to support the client, to employ immediacy in the session, and to utilize personal disclosure. In addition, all negative items (e.g., “In this session, the helper did not encourage the client to express what he/she was thinking or feeling”) were re-worded to have positive content (e.g., “In this session the helper encouraged the client to express what he/she was thinking or feeling”). Hill and Kellems (2002) found estimates of internal consistency to be adequate for the Exploration and Action scales, but less so for the Insight scale. The study also reported low to moderate intercorrelation among the scales, suggesting that the three scales were related, yet distinct.

The instructor and a master’s level research assistant underwent training on the HSM. The training consisted of reading Hill (2009) and Hill and Kellems (2002), practicing coding on 17 videotaped sessions, and discussing the rating categories as a team. After completing the training, the two raters will watch each trainee’s third session videotape in its entirety and then immediately rate the session independently using the HSM. Regular reliability meetings have been held during the coding process to prevent rater drift. The raters have demonstrated inter-rater reliability in the average (ICC(2,2) = .60 – .74; Shrout & Fleiss, 1979) to excellent range (≥ .75) for each of the items. In addition, their intraclass reliability coefficients, ICC(2,2), for the three scale scores Explore, Insight, and Action are in the good to excellent range (.87, .74, and .78).

Planned Statistical Analyses

Two multiple regression analyses will be conducted. The first regression will be used to predict client rated alliance as measured by the total score on the WAI-C. The second regression will be used to predict therapist technique as measured by the average external rater score on the HSM. For both analyses, the independent variables will be the following therapist characteristics: (a) College GPA; (b) Verbal GRE; (c) Performance GRE; (d) Avoidant Attachment scores on the ECR-R; (e) Anxious Attachment scores on the ECR-R; (f) IIP-SC total Interpersonal Problems score; (g) Empathic Concern subscale scores on the IRI; (h) Perspective Taking subscale scores on the IRI; (i) Number of months of personal therapy (none/0 – X); and (j) Average score on the SCORS-G. Additionally, we will examine incremental validity of the above listed variables using hierarchical, alternating, and block regression.

Anticipated Outcomes

Given that past research has shown therapists’ empathic ability, attachment, and interpersonal styles to be related to alliance (Ackerman & Hilsenroth, 2003; Diener & Monroe, 2011), we expect the ECR-R, IIP-SC, and IRI subscales to predict client-rated alliance. Relatedly, we may also expect the SCORS-G to be positively associated with alliance. However, given the lack of previous research on implicit measures of object relations as they relate to the process of therapy, the hypothesis about the SCORS-G is more tentative.

Finally, given that undergraduate GPA and GRE scores have been shown to predict graduate school performance such as graduate GPA, comprehensive exam scores, and faculty ratings (Kuncel, Hezlett, & Ones, 2001), we may expect that higher scores on these two domains will relate to the ability to learn and implement therapeutic techniques (i.e., higher scores on the HSM). However, similarly to the previous hypothesis, due to the paucity of existing research, this prediction is also tentative.

[1] In addition to several student and career awards, The Society for the Advancement of Psychotherapy regularly provides funding for research through two competitive grants—the Norine Johnson, Ph.D., Psychotherapy Research Grant and the Charles J. Gelso, Ph.D., Psychotherapy Research Grant. One Norine Johnson, Ph.D., Psychotherapy Research Grant of up to $10,000 is awarded each year for a project designed to study psychotherapist factors that may impact treatment effectiveness and outcomes. As many as three Charles J. Gelso, Ph.D., Research Grants of up to $5,000 are awarded each year for projects designed to study psychotherapy process and/or psychotherapy outcome. This year, the Psychotherapy Research feature articles will present brief reviews of some of the studies that have recently been funded through these grants.

+1

0

Jenelle Slavin-Mulford, Ph.D.

Jenelle Slavin-Mulford, PhD, is a licensed clinical psychologist, Associate Professor of Psychology at Augusta University, and the chair of the Psychotherapy Research Committee for APA Division 29. She received her bachelor’s degree from Middlebury College and her Ph.D. in Clinical Psychology from the Derner Institute of Advanced Psychological Studies at Adelphi University. She then completed her internship and postdoctoral fellowship at The Massachusetts General Hospital/Harvard Medical School. She has published numerous peer reviewed manuscripts in the areas of personality assessment, psychotherapy process and outcome, and training/supervision.