Internet Editor’s Note: Ms. Rebecca Janis and colleagues recently published an article titled “Developing a therapeutic relationship monitoring system for group treatment” in Psychotherapy.
If you’re a member of the Society for the Advancement of Psychotherapy you can access the Psychotherapy article via your APA member page.
Not a member? Purchase the Psychotherapy article for $11.95 here.
Treatment Feedback and Success
Monitoring Treatment Success
Measuring the success of treatment can involve many criteria, one being change on some outcome (e.g., psychiatric distress) to normal or improved levels (Kazdin, 2016; Lambert, 2015). Wampold (2015) noted that routine outcome monitoring (ROM) and its feedback to clinicians is now well-supported and should be adopted wherever possible. The recent special issue in Psychotherapy (see Marmarosh,  for an introduction and review of the series) deals with ROM in group psychotherapy. The Outcome Questionnaire-Analyst (OQ-A) system is one of two ROM systems that have shown evidence sufficient to be included in the Substance Abuse and Mental Health Services Administration’s (SAMHSA) National Registry of Evidence-based Programs and Practices (NREPP; https://nrepp.samhsa.gov/reviews_program.aspx). Of ROM structures, the OQ-A system has the most robust evidence basis and has produced the only randomized controlled trial (RCT) of ROM in group treatment. Our group’s (http://cgrp.byu.edu/) research on ROM in the OQ-A system began with a routine outcome for individual treatment.
The use of ROM in mental health treatment involves clinicians receiving progress feedback on their clients who complete an outcome assessment on a session by session basis (Shimokawa, Lambert, & Smart, 2010). The OQ-A's flagship routine outcome, a global measure of psychiatric distress (the Outcome Questionnaire [OQ-45]) came about after Lambert (2015) noticed a brief, clinically useful measure of feedback was needed to identify which clients were not on track (NOT) for treatment success. Following this, a clinical support tool (CST) was developed to measure reliable moderators of treatment success for NOT clients (Whipple et al., 2003). The measure, called the Assessment for Signal Clients (ASC; Witold, Lambert, Harris, Busath, & Vazquez, 2012) evaluates treatment moderators, like the working alliance, so the therapist can problem solve in those areas. Importantly, the ASC is only administered to individual therapy clients when they are shown to be NOT. A very important takeaway here is that progress feedback uses moderators of treatment success to guide the therapist in how best to rectify therapeutic relationship failure.
Group Outcome Monitoring
The OQ-A's history is rooted in individual therapy research, but has notable findings related to group treatment. The OQ-A ROM for group treatment came about from two findings: (1) Chapman and colleagues’ (2012) results showing group leaders could not accurately predict a single case of treatment deterioration, and (2) research showing clinicians find continual feedback during treatment preferable (Burlingame et al., 2018; Strauss, Burlingame, & Bormann, 2008). Given that group leaders cannot predict treatment failure and they want frequent feedback, the Group Questionnaire (GQ; Krogel et al., 2013) measures treatment moderators on a session-by-session basis to reverse relationship failure in therapy groups. The GQ shows promise as a way to receive clinically useful and immediate feedback about groups the therapist is running.
The Special Issue
Continuing research into the GQ is giving credence to its use in group treatment, and raising some questions. The RCT by Burlingame and colleagues (2018) addressed a critical lack of testing ROM’s feedback effect in group therapy. Their analysis came up with some conflicting findings: the major GQ subscales work to identify and reverse treatment failure, for the most part. One GQ subscale, measuring negative feelings toward the group, did not show evidence of a feedback effect following any treatment failure alert. Also in the special issue, Janis, Burlingame, and Olsen (2018) combined different six trials’ GQ data to verify the GQ’s factor structure with greater power and to create more specific norms (e.g., with European inpatients). One of their interesting findings is that items on the same negative relationship (NR) scale mentioned above related to the leader and other members’ genuineness, caring, and understanding all showed a poor association to the group’s general ratings. Hence, these items tapping perceptions of meaningful statements about other people in the group did not tend to agree.
The research above highlights some consistency in findings: that the NR subscale shows interesting trends in its use. In a recent secondary analysis of Burlingame and colleagues’ (2018) data, we instead asked if the number of co-occurring GQ alerts matters for the feedback effect (e.g., does it matter if the clinician only has to attend to one alert? What about more?). The results were even less clear than the original studies: NR originally showed no effect of feedback, while the secondary analyses showed no effect of feedback on all GQ subscales. Thus, we come to an impasse in dealing with NR alerts.
Because certain key NR items do not agree among members, is there a better way to analyze such data? Different analyses would allow future research to characterize ‘alerting styles’ for clients in group treatment. It might be that certain clients are much more likely to experience alliance ruptures with the leader, other members, or the group-as-a-whole. Perhaps these clients are more difficult to reverse ruptures with and account for the lack of change following an alert, as in Burlingame and colleagues’ (2018) trial.
Measuring treatment success can help the clinician keep track of individual clients and group members who might have bad outcomes. Our recent research in groups has validated using ROM in therapy groups and raised some questions. Measures like the GQ work and help mental health clinicians reverse treatment failure in key areas. Exciting new research is taking place in this area, particularly in characterizing what kinds of problems group members report.
Cite This Article
Svien, H., Janis, R. A., & Burlingame, G. M. (2019, June). Progress feedback in group therapy: The past and the future. [Web article]. Retrieved from http://www.societyforpsychotherapy.org/progress-feedback-in-group-therapy/
Burlingame, G. M., Whitcomb, K. E., Woodland, S. C., Olsen, J. A., Beecher, M., & Gleave, R. (2018). The effects of relationship and progress feedback in group psychotherapy using the group questionnaire and outcome questionnaire—45: A randomized clinical trial. Psychotherapy, 55, 116–131. http://dx.doi.org/10.1037/pst0000133
Chapman, C. L., Burlingame, G. M., Gleave, R., Rees, F., Beecher, M., & Porter, G. S. (2012). Clinical prediction in group psychotherapy. Psychotherapy Research, 22(6), 673-681. https://doi.org/10.1080/10503307.2012.702512
Janis, R. A., Burlingame, G. M., & Olsen, J. A. (2018). Developing a therapeutic relationship monitoring system for group treatment. Psychotherapy, 55(2), 105-115. http://dx.doi.org/10.1037/pst0000139
Kazdin, A. E. (2016). Research design in clinical psychology. London, UK: Pearson.
Krogel, J., Burlingame, G. M., Chapman, C., Renshaw, T., Gleave, R., Beecher, M., & Macnair-semands, R. (2013). The group questionnaire: A clinical and empirically derived measure of group relationship. Psychotherapy Research, 23(3), 344-354. https://doi.org/10.1080/10503307.2012.729868
Lambert, M. J. (2015). Progress feedback and the OQ-system: The past and the future. Psychotherapy, 52(4), 381-390.
Marmarosh, C. L. (2018). Introduction to special issue: Feedback in group psychotherapy. Psychotherapy, 55(2), 101-104. http://dx.doi.org/10.1037/pst0000178
Shimokawa, K., Lambert, M. J., Smart, D. W. (2010). Enhancing treatment outcome of patients at risk of treatment failure: Meta-analytic and mega-analytic review of a psychotherapy quality assurance system. Journal of Consulting and Clinical Psychology, 78(3), 298-311. doi: 10.1037/a0019247
Strauss, B., Burlingame, G. M., & Bormann, B. (2008). Using the CORE-R battery in group psychotherapy. Journal of Clinical Psychology, 64(11), 1225-1237. https://doi.org/10.1002/jclp.20535
Wampold, B. E. (2015). Routine outcome monitoring: Coming of age—with the usual developmental challenges. Psychotherapy, 52(4), 458-462. http://dx.doi.org/10.1037/pst0000037
Whipple, J. L., Lambert, M. J., Vermeersch, D. A., Smart, D. W., Nielsen, S. L., & Hawkins, E. J. (2003). Improving the effects of psychotherapy: The use of early identification of treatment failure and problem-solving strategies in routine practice. Journal of Counseling Psychology, 51(1), 59-68.
Witold, S., Lambert, M. J., Harris, M. W., Busath, G., & Vazquez, A. (2012). Providing patient progress information and clinical support tools to therapists: Effects on patients at risk of treatment failure. Psychotherapy Research, 22(6), 638-647. https://doi.org/10.1080/10503307.2012.698918