“Evidence-Based” Training?

Research is important in the scientific field of psychotherapy, where we like to think of ourselves as “scientist-practitioners” who provide “evidence-based practice” (Overholser, 2012). However, when it concerns our professional training, this research emphasis appears to be conveniently forgotten. 

Therapists tend to spend many hours of their career in professional training, not only in graduate school but also post-licensure. Yet, there is a paucity of research on the effectiveness of training (Knox & Hill, 2021), especially training for licensed therapists (Aafjes-van Doorn & Barber, 2022). The impact of training might be especially pronounced at the beginning of a therapists’ career and level off, becoming harder to detect as individuals grow more individually toward the expertise level.

In the past 40 years, undergoing advanced credentialing (e.g., board certification through the American Board of Professional Psychology [ABPP]) and completing formal continuing education (CE) programs have become increasingly common requirements for license renewal. The principal purpose of CE programs is the maintenance of competence, the improvement of services, and the protection of the public. Therapists may get CE credits for attending trainings in varying formats such as conferences and in-person, online workshops, or expert-lead supervisions. The vast majority of psychologists (75%–85%) support the idea of mandated CE and since these mandates have been implemented, the intrinsic motivation of therapists to attend CE programs seems to have increased (Neimeyer et al., 2019). 

Yet, CE credits only measure attendance of events (hours), not the effect on the therapist or subsequent patient treatments. Despite the call for evaluation of continuing professional development in psychology several decades ago (Webster, 1971), it appears that relatively little research has focused on the effectiveness and efficacy of such professional training activities. When reviewing the literature, it becomes apparent that almost all studies that assessed the effect of training used the level of therapists’ satisfaction as a proxy of outcome. In all these studies, licensed therapists reported high levels of satisfaction, which likely reflects their choice-supportive bias (the tendency to retroactively ascribe positive attributes to an option one has selected), especially following their investment of time and money in attending these professional trainings (Aafjes-van Doorn & Barber, 2022). Indeed, when measured, therapists tend to report some therapy skill improvements immediately after brief skills training. However, these short, quick doses of training may not translate to therapist behavior change in practice. Multi-component training packages, especially those that include direct feedback and experiential components, seem to be somewhat beneficial in improving therapists’ skills, but only very few of these studies used standardized symptom measures to evaluate changes in patient outcome following therapist training. A noteworthy exception is provided by Weck and colleagues (2021), who showed that providing competence feedback to therapists may lead to greater change in patients’ alliance ratings and depression symptoms when compared to a control group. All in all, there is not enough evidence to suggest that attendance of professional training results in improved competence or better treatment outcomes for the respective patients. But if I need to place my bet, I would put my money on the type of experiential trainings that provide direct feedback to therapists on their level of performance. 

The Case for Deliberate Practice

Deliberate practice (DP), the explicit setting aside of private time to review one’s behavior and outcome feedback, developing plans for skills development based on repetition and successive refinement of individualized training activities, has been proposed as a means of enhancing individual therapists’ competence and expertise (Rousmaniere, 2019). Unlike traditional training workshops in which therapists receive little to no feedback and little opportunity to practice and to try something new, correct mistakes, and gradually develop a new skill (Ericsson & Pool, 2016), DP requires active involvement in experiential exercises and direct expert feedback (the two elements that appear to be effective in therapist training; Aafjes-van Doorn & Barber, 2022). When implemented in supervision, DP may enable the supervisee to address skill deficits highlighted in patient feedback or by the supervisor directly. DP can also be provided in the form of a skills workshop, in which therapists engage in DP of responses modeled by an expert (e.g., e.g., Shukla et al., 2021 ; Westra et al., 2021). Although the effectiveness of DP is by no means ‘proven,’ there is evidence that therapists who obtain better patient outcomes engage in more DP than colleagues whose patients demonstrate lower levels of change (Chow et al., 2015), and the time a therapist spends in DP activities might account for the effectiveness of top-performing therapists. 

The use of DP offers several benefits over the attendance of traditional professional development activities, such as conferences, webinars, and workshops. First, DP stimulates agency. Rather than passively attending a workshop someone else has organized, DP is guided by an active search for learning opportunities and for evidence of whether skills are being used effectively (McLeod, 2021). This active involvement and close observations highlight a therapist’s agency. Second, DP offers an individualized learning approach. Rather than following a standard agenda of the trainer, a therapist can focus on skills and competencies that they choose themselves, based on appraisal of their deficiencies. This kind of individualized learning goal enables a therapist to devise tasks that are at a level of emotional, interpersonal and intellectual difficulty that are neither too easy nor too demanding, in line with Vygotsky’s theory of ‘zone of proximal development.’ Third, DP activities can be flexible depending on the situation. DP activities can be done alone (reviewing video recordings of therapy sessions), but therapists may also choose to use one or multiple sources of expertise, including peers, experienced therapists (expert supervisors or videos of master therapists), or even patients (McLeod, 2021). This allows therapists to gain feedback from different people and consider multiple perspectives on what ‘skillful’ may look like.

Admittingly, DP is not an easy training option. It might take trial and error to identify the specific skills exercises that are challenging but not too difficult. And even when a suitable set of exercises is established, it likely requires a substantial time investment (e.g., 2 hours per week) to benefit from this learning process (Chow et al., 2015; McLeod, 2021). Above all, the effective use of DP requires a capacity for self-evaluation and an openness to be guided by results. A recent qualitative report of the DP experience of 42 students highlighted two particularly difficult aspects in the implementation of DP (McLeod, 2021). The first challenge is to identify what to work on. Some DP topics might arise from personal self-reflection on situations in therapy that were experienced as difficult. Therapists might also use implicit or explicit feedback from others to identify appropriate DP topics. For example, DP topics might arise from a patient’s confrontation rupture in a previous session, a more subtle communication of misattunement, or a supervisor’s feedback on a recorded therapy session. Once the DP topic is identified, the second challenge is to know when you have practiced a skill enough. It might be difficult to determine when you become proficient, or sufficient in a certain skill, or when it might require further work. 

Luckily, therapists don’t have to rely on the words of colleagues, a supervisor, or their own inner critic to self-evaluate their skills. Several technological advancements (i.e., clinical tools) may help the therapist to gain feedback on what DP topics to work on and when a skill has been sufficiently mastered. In particular, the use of progress monitoring (PM) and reviewing of videorecorded therapy sessions may provide useful information about the session-by-session change a patient experience of symptoms and alliance and the therapists’ interpersonal interactions that could potentially be targeted through DP exercises. A DP exercise often consists of watching a video of a challenging moment in a therapy session while tracking ones’ inner experiences and avoidance responses. By deliberately practicing with stimuli (e.g., video clips of therapy sessions) that mimic live therapy conditions, individual DP exercises might be an effective way to achieve changes in therapists’ interpersonal qualities, possibly improving psychological capacities to bear with intense emotions (Rousmaniere, 2019). For DP, it might be particularly useful to share these recorded sessions — in conjunction with patient-reported outcomes — with a supervisor. A video may be easier to translate into specific topics for DP if it can be augmented with continuous feedback about what the therapist is or is not doing, and a safe space to reflect on and analyze the patient feedback received. The supervisor doesn’t even need to review the whole session; it appears that even reviewing 5 or 15- minute segments of therapy sessions might be sufficient for the assessment of therapeutic qualities and potential skills deficiencies (Lewin & Berman, 2021). 

This careful review of treatment videos and outcome measures might be commonplace for therapists in training, but it is a lot less common among seasoned therapists. The unwarranted over-confidence of experienced therapists means that they are less likely to be motivated to take actions (e.g., obtain and use critical feedback) that would enhance their actual expertise (Pintrich, 2003). Many therapists don’t yet collect objective treatment information and do not know how to use the information that does exist to improve their performance over time (Tracey et al., 2014), but do attend conferences and workshops to fulfil their CE credits. 

That said, it is possible that for some therapists, the main purpose of attending these professional trainings may be to get some respite from busy caseloads outside the office, gain peer support, avoid burnout, or to build and maintain a professional network. Maybe this is why so many therapists are satisfied with their training experiences. Such potential benefits of professional trainings tend to be forgotten in the quest for ‘evidence-based training’ and have certainly not yet been empirically examined. 

This paucity of research on the effectiveness of therapist training doesn’t mean that it is ineffective per se, or that we should not engage in such professional development. It just means that we might need to look beyond the simple attendance of conferences, workshops, or webinars to continue to develop our skills and improve our patients’ outcomes. If we really are “scientist-practitioners” in “evidence-based practice,” we will need to develop openness to be guided by results, and to put in the time and effort to deliberately practice our skills. 

+4

0

Katie Aafjes-van Doorn, Psy.D.

Dr. Katie Aafjes-van Doorn is a clinical psychologist, psychotherapy researcher and assistant professor. She received a MSc in Clinical Psychology at the University of Amsterdam, and MSc in Psychological Research, followed by a doctorate in Clinical Psychology at University of Oxford, and gained licensure in the UK. She did her postdoctoral research fellowship with Dr. Jacques Barber and his lab at Adelphi University, New York. In 2017 she became a full-time tenure-track faculty at Ferkauf Graduate School of Psychology, Yeshiva University, New York. She teaches courses on psychotherapy process, and research design. She hopes to contribute to the "evidence-base" of psychodynamic psychotherapies by operationalizing psychoanalytic concepts such as defenses, affect experiencing, (counter) transference and reflective functioning, and to inspire the new generation of clinicians to continue to engage in psychotherapy research.