Psychotherapy Bulletin

Psychotherapy Bulletin

Ample research suggests that therapists differ in their level of effectiveness (Baldwin & Imel, 2013; Blow, Sprenkle, & Davis, 2007; Wampold, 2001). Even more striking is that therapist effects appear to be larger than treatment effects (Kim, Wampold, & Bolt, 2006; Lindgren, Folkesson, & Almiqvist, 2010). Moreover, therapist training, experience, and theoretical orientation do not appear to explain the majority of therapist effects (Beutler et al., 2013; Okiishi, Lambert, Nielsen, & Ogles, 2003; Stirman & Crits-Cristoph, 2011). Therefore, it has been hypothesized that therapists’ personal characteristics may impact treatment (Heinonen , Linfdors, Laaksonen, & Knekt, 2012; Hersoug, Høglend, Havik, von der Lippe, & Monsen, 2009).

Hypotheses regarding what makes a good therapist often center on constructs such as intelligence (Shedler, 2006), empathic ability (Hill et al., 2008), interpersonal and attachment styles (Marmarosh et al., 2013), and history of personal therapy (Gold & Hilsenroth, 2009). Unfortunately, the empirical literature has largely ignored some of these factors and produced inconclusive or limited results for others. For example, a therapist’s empathic ability is theorized to be critically important (Rogers, 1957). Nonetheless, research on the degree that pre-screening measures of empathy can predict later therapeutic effectiveness is mixed (Hill et al., 2008; Moyers & Miller, 2013).

Thus, while there is considerable evidence that therapist characteristics influence the process and outcome of therapy, much more work is needed (Blatt, Sanislow III, Zuroff, & Pilkonis, 1996; Blow et al., 2007; Lebow, 2006). Greater understanding of which therapist factors are most important and the degree to which these factors are innate versus developed through training will have important implications for graduate school admissions criteria as well as types and methods of training. This study aims to contribute to this need by using a multi-method evaluation to assess students at the beginning of graduate school to determine which trainee characteristics predict later success in forming an alliance and implementing therapeutic techniques.


Participants. Participants in the current study are graduate students enrolled in a Masters in clinical psychology program at a southeastern university. Presently, data have been collected from two cohorts of graduate trainees (N = 19). The participant group is currently 74% female (n = 14) with a mean age of 24 years (SD = 6.74). The racial composition of the sample was reported as 68% European American, 21% African American, 5% Asian American, and 5% biracial. Thirty-seven percent (n = 7) of the present sample endorsed having received therapy, with a mean time spent in therapy of 26 months (SD = 34.66). In addition, participants’ academic records indicated an overall mean undergraduate grade point average (GPA) of 3.57 (SD = 0.23), a mean quantitative Graduate Record Examination (GRE) score of 147.72 (SD = 7.63), mean verbal GRE score of 152.22 (SD = 5.85), and a mean analytic GRE score of 3.81 (SD = 0.57). Finally, at the time of this report, 16% (n = 3) of the sample had either withdrawn from, or been asked to leave, the program. For those participants failing to complete the program, three months was the average amount of time completed (SD = 1.73).

The undergraduate students serving as practice therapy clients are also recruited from the same university. The students who consent to participate in the sessions do so as one of the class project options in a course focused on personal growth and exploration. These students receive course credit for participating in the sessions and writing a reflection essay about their experience. Importantly, none of these practice clients know the researchers in this project and their professor is not provided any information about the therapy sessions other than that the students participated. Presently, the practice client group (N = 16) is 50% female, 50% European American, 25% African American, 12.5% Asian American, and 12.5% Hispanic. The mean age of the group is 20 years, (SD = 2.13). Moreover, 31% (n = 5) reported previous therapy experience, with the mean time spent in therapy of 15 months (SD = 58.38).

Multi-Method Assessment

At the beginning of their graduate training (2nd day of class of the 1st semester), all students enrolled in the Masters of Clinical Psychology program complete a multi-method personality assessment as part of the course work for their Personality Assessment class. The assessment battery is designed to assess the following individual characteristics: personal therapy experience, attachment and interpersonal style, empathy, and implicit dynamics related to self and others. A research assistant (RA) unaffiliated with the program administers, de-identifies, and scores the assessments. Importantly, all student responses are kept confidential and are not shared with anyone in the program (including the professor of the course). The following is a list of the measures used in this assessment:

Academic achievement. The participants’ college GPA and GRE data are acquired from the students’ applications to the program.

Personal therapy experience. Trainees complete a demographic questionnaire that also includes one yes/no question regarding whether they have been in personal therapy and, if so, for approximately how many months.

Experience in Close Relationships-Revised (ECR-R). The ECR-R (Fraley, Waller, & Brennan, 2000) is a five-point Likert-type self-report scale. It consists of 36 statements assessing attachment-related anxiety and avoidance in different types of relationships. The scale includes two subscales: Avoidance and Anxiety (Fraley et. al., 2000). The statements utilized in the questionnaire reflect worries about attachment-related concerns, as well as discomfort with intimacy. Research has confirmed the two factor structure of the measure as well as its temporal stability, with 86% of the variance shared between two administrations of the scale over a six week period (Sibley & Liu, 2004).

The Inventory of Interpersonal Problems-Short Circumplex (IIP-SC). The IIP-SC (Soldz, Budman, Demby, & Merry, 1995) is a 32-item four-point Likert-type self-report scale which assesses eight separate domains of interpersonal problems. The measured domains consist of four items each and yield eight scale scores: Domineering/Controlling, Vindictive/Self-Centered, Cold/Distant, Socially Inhibited, Nonassertive, Overly Accommodating, Self-Sacrificing, and Intrusive/Needy (Horowitz, Alden, Wiggins, & Pincus, 2003). The measure has been reported as having excellent overall reliability (r = .93) and moderate scale reliability, with Cronbach alpha coefficients for the eight scales ranging from .68 for Intrusive/Needy to .87 for Cold/Distant (Horowitz et al., 2003).

Interpersonal Reactivity Index (IRI). The IRI (Davis, 1980, 1983) is a 28–item, five-point Likert-type self-report scale. The measure evaluates four separate aspects of the global construct of empathy with items divided into the following four subscales: Perspective Taking; Fantasy; Empathic Concern; and Personal Distress (Davis, 1983). Research has provided evidence of convergent and discriminant validity of the IRI (Davis, 1983), as well as its four-factor structure (Pulos, Elison, & Lennon, 2004).

Thematic Apperception Test (TAT). The TAT (Murray, 1973) is a set of 32 black-and-white stimulus cards with stylized images depicting specific life scenes. Individuals are shown TAT cards and asked to make up a story in response to each respective card. These stories are then examined in an effort to draw conclusions regarding the respondent’s internal world. For this study, the following seven cards: 1, 2, 3BM, 4, 13MF, 12M, and 14, were administered in a group format in which the cards were projected on a screen and participants were asked to write their responses in a notebook. After the TAT was administrated, responses were transcribed, de-identified, and independently scored by two trained raters using the Social Cognition and Object Relations Scale-Global Rating Method (SCORS-G; Stein, Hilsenroth, Slavin-Mulford, & Pinsker, 2011; Westen, 1995). The SCORS-G is comprised of eight constructs which are rated using a seven-point Likert-type scale, where lower scores are indicative of more pathological aspects of object representations and higher scores are suggestive of more mature and adaptive functioning. The two expert raters used for this project had previously completed manualized training on the SCORS-G (Stein et al., 2011; Westen, 1995) and achieved “good” to “excellent” reliability on the SCORS-G in previous research (Stein et al., 2014).

Therapy Sessions

In the students’ second semester, all graduate trainees take an introductory therapy course. This four-credit course focuses on therapeutic technique with curriculum based on Hill’s three-stage model of helping as presented in Helping Skills: Facilitating Exploration, Insight, and Action, Third Edition (2009). As part of the requirements of the course, all students participate in a series of four practice therapy sessions with undergraduate student volunteers. The first session is a 1.5 hour intake and the remaining three sessions are 45 minutes and focus on whatever issues the client presents.

The practice therapy clients are told that they can use the sessions to work on whatever feels most important to them at the time. However, they are instructed not to share concerns which would necessitate an intervention by a licensed professional, such as suicidal or homicidal ideation, or child or elder abuse. Any clients presenting with these issues are directed to the campus counseling center. Common presenting problems have included difficulties in interpersonal relationships, anxiety related to school performance, and concern regarding choosing a career path.

All sessions are videotaped and the trainees receive supervision from the instructor who is a licensed psychologist. Following sessions one, two, and four, trainees receive 1.5 hours of group supervision (2-3 trainees per group). In addition, students receive 1.5 hours of individual supervision following session three. Supervision focuses heavily on the review of video-recorded case material with emphasis on conceptualization, process, interpretation, and clinical interventions.

Post Session Evaluation

At the end of the third session, clients fill out a measure to assess therapeutic alliance. In addition, the third session videotapes are rated for technique use by two trained raters. The following is a list of the measures used in this assessment:

Working Alliance Inventory, Client Form (WAI-C). The WAI-C (Horvath & Greenberg, 1989) is a 36-item seven-point Likert-type self-report scale designed to assess three facets of the therapeutic relationship: Task, Bond, and Goal (Horvath & Greenberg, 1989). Good reliability (Hanson, Curry, & Bandalos, 2002; Horvath & Greenberg, 1989) and construct validity of the WAI-C (Tichnor & Hill, 1989) have both been reported.

Helping Skill Measure (HSM). The first 13 items of the HSM (Hill & Kellems, 2002) capture basic exploration, insight, and action therapy skills using a five-point Likert format ranging from “strongly disagree” to “strongly agree.” For the purpose of this study, three additional items were added to assess the use of interventions intended to support the client, to employ immediacy in the session, and to utilize personal disclosure. In addition, all negative items (e.g., “In this session, the helper did not encourage the client to express what he/she was thinking or feeling”) were re-worded to have positive content (e.g., “In this session the helper encouraged the client to express what he/she was thinking or feeling”). Hill and Kellems (2002) found estimates of internal consistency to be adequate for the Exploration and Action scales, but less so for the Insight scale. The study also reported low to moderate intercorrelation among the scales, suggesting that the three scales were related, yet distinct.

The instructor and a master’s level research assistant underwent training on the HSM. The training consisted of reading Hill (2009) and Hill and Kellems (2002), practicing coding on 17 videotaped sessions, and discussing the rating categories as a team. After completing the training, the two raters will watch each trainee’s third session videotape in its entirety and then immediately rate the session independently using the HSM. Regular reliability meetings have been held during the coding process to prevent rater drift. The raters have demonstrated inter-rater reliability in the average (ICC(2,2) = .60 - .74; Shrout & Fleiss, 1979) to excellent range (≥ .75) for each of the items. In addition, their intraclass reliability coefficients, ICC(2,2), for the three scale scores Explore, Insight, and Action are in the good to excellent range (.87, .74, and .78).

Planned Statistical Analyses

Two multiple regression analyses will be conducted. The first regression will be used to predict client rated alliance as measured by the total score on the WAI-C. The second regression will be used to predict therapist technique as measured by the average external rater score on the HSM. For both analyses, the independent variables will be the following therapist characteristics: (a) College GPA; (b) Verbal GRE; (c) Performance GRE; (d) Avoidant Attachment scores on the ECR-R; (e) Anxious Attachment scores on the ECR-R; (f) IIP-SC total Interpersonal Problems score; (g) Empathic Concern subscale scores on the IRI; (h) Perspective Taking subscale scores on the IRI; (i) Number of months of personal therapy (none/0 – X); and (j) Average score on the SCORS-G. Additionally, we will examine incremental validity of the above listed variables using hierarchical, alternating, and block regression.

Anticipated Outcomes

Given that past research has shown therapists’ empathic ability, attachment, and interpersonal styles to be related to alliance (Ackerman & Hilsenroth, 2003; Diener & Monroe, 2011), we expect the ECR-R, IIP-SC, and IRI subscales to predict client-rated alliance. Relatedly, we may also expect the SCORS-G to be positively associated with alliance. However, given the lack of previous research on implicit measures of object relations as they relate to the process of therapy, the hypothesis about the SCORS-G is more tentative.

Finally, given that undergraduate GPA and GRE scores have been shown to predict graduate school performance such as graduate GPA, comprehensive exam scores, and faculty ratings (Kuncel, Hezlett, & Ones, 2001), we may expect that higher scores on these two domains will relate to the ability to learn and implement therapeutic techniques (i.e., higher scores on the HSM). However, similarly to the previous hypothesis, due to the paucity of existing research, this prediction is also tentative.

[1] In addition to several student and career awards, The Society for the Advancement of Psychotherapy regularly provides funding for research through two competitive grants—the Norine Johnson, Ph.D., Psychotherapy Research Grant and the Charles J. Gelso, Ph.D., Psychotherapy Research Grant. One Norine Johnson, Ph.D., Psychotherapy Research Grant of up to $10,000 is awarded each year for a project designed to study psychotherapist factors that may impact treatment effectiveness and outcomes. As many as three Charles J. Gelso, Ph.D., Research Grants of up to $5,000 are awarded each year for projects designed to study psychotherapy process and/or psychotherapy outcome. This year, the Psychotherapy Research feature articles will present brief reviews of some of the studies that have recently been funded through these grants.

Be the 1st to vote.
Cite This Article

Slavin-Mulford, J., Perkey H., Williams, C.,Verlaque, L., & Stein, M. (2015). Trainee therapist characteristics related to therapeutic alliance and technique: Project summary. Psychotherapy Bulletin, 50(2), 14-18.


Ackerman, S. J., & Hilsenroth, M. J. (2003). A review of therapist characteristics and techniques positively impacting the therapeutic alliance. Clinical Psychology Review, 23, 1-33.

Baldwin, S. A., & Imel, Z. E. (2013). Therapist effects, findings and methods. In M. J. Lambert  (Ed.), Bergin and Garfield’s handbook of psychotherapy and behavior change (6th ed., pp. 258-297). New York, NY: John Wiley & Sons.

Blatt, S. J., Sanislow, C. A., III, Zuroff, D. C., & Pilkonis, P. A. (1996). Characteristics of effective therapists: Further analyses of data from the National Institute of Mental Health Treatment of Depression Collaborative Research Program. Journal of Consulting and Clinical Psychology, 64, 1276–1284. doi: 10.1037/0022-006X.64.6.1276

Beutler, L. E., Malik, M. L., Alimohamed, S., Harwood, T. M., Talebi, H., & Noble, S. (2013). Therapist variables. In M. J. Lambert (Ed.), Bergin and Garfield’s handbook of psychotherapy and behavior change (6th ed., pp. 227-257). New York, NY: John Wiley & Sons.

Blow, A. J., Sprenkle, D. H., & Davis, S. D. (2007). Is who delivers the treatment more important than the treatment itself? The role of the therapist in common factors. Journal of Marital and Family Therapy, 33, 298-317. doi: 10.1111/j.1752-0606.2007.00029.x

Davis, M. H. (1980) A multidimensional approach to individual differences in empathy, JSAS Catalog of Selected Documents in Psychology, 10, 85-104.

Davis, M. H. (1983). Measuring individual differences in empathy: Evidence for a multidimensional approach. Journal of Personality and Social Psychology, 44, 113–126. doi: 10.1037/0022-3514.44.1.113

Diener, M. J., & Monroe, J. M. (2011). The relationship between adult attachment style and therapeutic alliance in individual psychotherapy: A meta-analytic review. Psychotherapy, 48, 237-248. doi: 10.1037/a0022425

Fraley, R. C., Waller, N. G., & Brennan, K. A. (2000). An item-response theory analysis of self-report measures of adult attachment. Journal of Personality and Social Psychology, 78, 350-365. doi: 10.1037/0022-3514.78.2.350

Gold, S. H., & Hilsenroth, M. J. (2009). Effects of graduate clinicians’ personal therapy on therapeutic alliance. Clinical Psychology and Psychotherapy, 16(3), 159-171. doi: 10.1002/cpp.612

Hanson, W. E., Curry, K. T., & Bandalos, D. L. (2002). Reliability generalization of Working Alliance Inventory scale scores. Educational and Psychological Measurement, 62, 659-673. doi: 10.1177/0013164402062004008

Heinonen, E., Lindfors, O., Laaksonen, M. A., & Knekt, P. (2012). Therapists’ professional and personal characteristics as predictors of outcome in short- and long-term psychotherapy. Journal of Affective Disorders, 138, 301–312. doi:10.1016/j.jad.2012.01.023

Hersoug, A. G., Høglend, P., Havik, O., von der Lippe, A., & Monsen J. (2009). Therapist characteristics influencing the quality of alliance in long-term psychotherapy. Clinical Psychology and Psychotherapy, 16, 100–110. doi: 10.1002/cpp.605

Hill, C. E. (2009). Helping skills: Facilitating exploration, insight, and action (5th ed.). Washington, DC: American Psychological Association.

Hill, C. E., & Kellems, I. S. (2002). Development and use of the helping skills measure to assess client perceptions of the effects of training and of helping skills in sessions. Journal of Counseling Psychology, 49, 264–272. doi:10.1037/0022-0167.49.2.264

Hill, C. E., Roffman, M., Stahl, J., Friedman, S., Hummel, A., & Wallace, C. (2008). Helping skills training for undergraduates: Outcomes and predictions of outcomes. Journal of Counseling Psychology, 55, 359–370. doi: 10.1037/0022-0167.55.3.359

Horowitz, L. M., Alden, L. E., Wiggins, J. S., & Pincus, A. L. (2003). Inventory of Interpersonal Problems, Manual. Menlo Park, CA: Mind Garden.

Horvath, A. O., & Greenberg, L. S. (1989). Development and validation of the Working Alliance Inventory. Journal of Counseling Psychology, 36, 223-233. doi: 10.1037/0022-0167.36.2.223

Kim, D., Wampold, B. E., & Bolt, D. M. (2006). Therapist effects in psychotherapy: A random-effects modeling of the National Institute of Mental Health Treatment of Depression Collaborative Research Program data. Psychotherapy Research, 16, 161–172. doi: 10.1080/10503300500264911

Kuncel, N. R., Hezlett, S. A., & Ones, D. S. (2001).  A comprehensive meta-analysis of the predictive validity of Graduate Record Examinations: Implications for graduate student selection and performance. Psychological Bulletin, 127, 162-181.

Lebow, J. (2006). Research for the psychotherapist: From science to practice. New York, NY: Routledge/Taylor & Francis.

Lindgren, O., Folkesson, P., Almqvist, K., & Mehler, S. (2010). On the importance of the therapist in psychotherapy: A summary of current research. International Forum of Psychoanalysis, 19, 224-229. doi: 10.1080/08037060903536047

Marmarosh, C. L., Markin, R. D., & Spiegel, E. B. (2013). Attachment in group psychotherapy. Washington, DC: American Psychological Association. doi: 10.1037/14186-000

Moyers, T. B., & Miller, W. R. (2013). Is low therapist empathy toxic? Psychology of Addictive Behaviors, 27, 878–884. doi: 10.1037/a0030274

Murray, H. (1973). The analysis of fantasy. Huntington, NY: Robert E. Krieger Publishing.

Okiishi, J., Lambert, M. J., Nielsen, S. L., & Ogles, B. M. (2003). Waiting for supershrink: An empirical analysis of therapist effects. Clinical Psychology and Psychotherapy, 10, 361-373. doi: 10.1002/cpp.383

Pulos, S., Elison, J., & Lennon, R. (2004). The hierarchical structure of the Interpersonal Reactivity Index. Social Behavior and Personality, 32, 355-360. doi: 10.1037/t01093-000

Rogers, C. R. (1957). The necessary and sufficient conditions of therapeutic personality change. Journal of Consulting and Clinical psychology, 21, 95–103. doi: 10.1037/h0045357

Shedler, J. (2006). Why the scientist–practitioner schism won’t go away. The General Psychologist, 41, 9–10.

Shrout, P. E., & Fleiss, J. L. (1979). Intraclass correlations: Uses in assessing rater reliability. Psychological Bulletin, 86, 420-428. doi: 10.1037/0033-2909.86.2.420

Sibley, C. G., & Liu, J. H. (2004). Short-term temporal stability and factor structure of the revised experiences in close relationships (ECR-R) measure of adult attachment. Personality and Individual Differences, 36, 969-975. doi:10.1016/S0191-8869(03)00165-X

Soldz, S., Budman, S., Demby, A., & Merry, J. (1995). A short form of the Inventory of Interpersonal Problems Circumplex scales. Assessment, 2, 53–63. doi: 10.1177/1073191195002001006

Stein, M., Hilsenroth, M., Slavin-Mulford, J., & Pinsker, J. (2011). Social Cognition and Object Relations Scale: Global Rating Method (SCORS-G, 4th ed.). Unpublished manuscript. Massachusetts General Hospital and Harvard Medical School, Boston. Retrieved from

Stein, M. B., Slavin-Mulford, J., Siefert, C. J., Sinclair, S. J., Malone, J. C., Renna, M., . . . &

Blais, M. A. (2014).  SCORS-G stimulus characteristics of select Thematic Apperception cards.  Journal of Personality Assessment, 96(3), 339-349. doi: 10.1080/00223891.2013.823440.

Stirman, S. W., & Crits-Cristoph, P. (2011). Psychotherapy research: Implications for optimal therapist personality, training, and development. In R. H. Klein, H. S. Bernard, & V. L. Schermer (Eds.), On becoming a psychotherapist: The personal and professional journey (pp. 245–268). New York, NY: Oxford University Press.

Tichnor, V., & Hill, C. E. (1989). A comparison of six measures of working alliance. Psychotherapy: Theory, Research, Practice, Training, 26,195-199. doi: 10.1037/h0085419

Wampold, B. E. (2001). The great psychotherapy debate: Models, methods, and findings. Mahwah, NJ: Lawrence Erlbaum Associates.

Westen, D. (1995). Social cognition and object relations scale: Q-sort for projective stories (SCORS-Q; Unpublished manuscript). Department of Psychiatry, The Cambridge Hospital and Harvard Medical School, Cambridge, MA.

Lauren Verlaque, M.S., & Michelle Stein, PhD.


Submit a Comment

Your email address will not be published. Required fields are marked *