Internet Editor’s Note: Dr. Michael Lambert and colleagues recently published an article titled “Collecting and delivering progress feedback: A meta-analysis of routine outcome monitoring” in Psychotherapy.
If you’re a member of the Society for the Advancement of Psychotherapy you can access the Psychotherapy article via your APA member page.
Not a member? Purchase the Psychotherapy article for $11.95 here.
Thousands of clinical trials and naturalistic studies have now been conducted on the effects of psychotherapy. Reviews of this research have shown that about 75% of those who enter treatment in clinical trials show some benefit (Lambert, 2013). This finding generalizes across a wide range of disorders except for severe biologically based disturbances (e.g., bipolar disorder, schizophrenia).
An often ignored but critical consideration in psychotherapy is the degree to which it is not helpful or even harmful. An estimated 5%–10% of adult clients participating in clinical trials leave treatment worse off than they began (Lambert, 2013). In routine care, the situation is frequently more problematic. Outcomes for more than 6,000 clients treated in routine practice settings across the USA suggest that clients did not fare nearly as well as those in clinical trials, with only about one third showing improvement or recovery (Hansen, Lambert, & Forman, 2002).
Unfortunately, clinicians tend to hold overly optimistic views of their clients’ treatment progress in relation to measured change (Walfish et al., 2012). Clinicians frequently overlook negative changes and experience difficulty accurately gauging the benefit clients have received during treatment, particularly with clients who are failing to improve. For example, even when therapists were provided with the base rate of deterioration in their clinic (8%), and were asked to rate each client at the end of every session, they rated only 3 of 550 clients as predicted failures and seriously underestimated worse functioning for a significant portion of clients (Hannan et al., 2005).
In efforts to reduce negative outcomes, Routine Outcome Monitoring (ROM) has been proposed. ROM involves regularly measuring and monitoring client progress with standardized self-report scales throughout the course of treatment and providing clinicians with this information before psychotherapy ends (Lambert, Hansen & Finch, 2001; Newham, Hooke, & Page, 2010). Contextualized feedback theory suggests (Sapyta, Riemer, & Bickman, 2005) the value of ROM hinges on the degree to which the information provided to clinicians goes beyond what clinicians observe and understand about client progress without such information. ROM feedback provides clinicians with statistically-based predictive algorithms that compare a client’s expected and actual treatment response. The feedback helps clinicians recognize problematic treatment response and provides problem-solving tools that improve collaborative efforts in cases whose positive response to therapy is in doubt.
The American Psychological Association (APA, 2006) has recommended routine outcome monitoring be a part of effective psychological services because certain methods of monitoring have been shown to enhance client outcome. Two of the most widely studied monitor systems are the Outcome Questionnaire-System (OQ-System; Lambert et al., 2013) and the Partners for Change Outcome Management System (PCOMS; Duncan, & Miller, 2008; Prescott, Maeschalck & Miller, 2017). In the current Highlight we review an article by Lambert, Whipple, and Kleinstäuber which is published in a special issue of the journal Psychotherapy in December 2018 and which summarizes the results of a systematic and meta-analytic review of the effects of the OQ-System and the PCOMS.
A systematic search of electronic databases such as Medline, PsycINFO, PsycEXTRA, Google Scholar, past reviews, and hand searches identified 24 studies examined the impact of measuring, monitoring, and feeding back information on client progress to clinicians while they deliver psychotherapy. Of the included studies 15 addressed outcomes based on the Outcome Questionnaire System with a total sample size of 8,649 clients. The remaining nine studies reported effects of the PCOMS with a total sample size of 2272 clients. The studies were published between 2001 and 2017; 64% of the studies were conducted in the USA, the remaining studies in European countries (The Netherlands, Norway, Sweden, Denmark, Ireland, and Germany).
Two-thirds of the studies found that ROM-assisted psychotherapy was superior to treatment-without-feedback offered by the same practitioners. Mean standardized effect sizes indicated that the effects ranged from small to moderate. Feedback practices reduced deterioration rates substantially and nearly doubled clinically significant/reliable change rates in clients who were predicted to have a poor outcome. The best results were found when clinicians used problem-solving tools in addition to information that a client was not making expected progress. Such problem-solving tools were composed of a problem-solving decision tree that systematically directs a clinician’s attention to factors that have been shown to be consistently related to client outcome. They were driven be brief assessment of the therapeutic alliance or other problems such as diminished social supports. ROM practices were recommended across types of school-based psychotherapies or integrative practices, client diagnosis, treatment modalities (group, individual, couples), treatment settings (outpatient, inpatient), and countries. Differences in effectiveness were found across individual therapists, mainly due to therapist’s failure to access feedback information.
Summary and Conclusions
The research evidence supports the use of routinely and formally monitoring the mental health of clients as they undergo a course of psychotherapy using either of the ROM methods which were included to the systematic review presented in this highlight. Important recommendations can be drawn from this article for psychotherapists. First outcome monitoring systems such as the OQ-System or the PCOMS should be routinely applied with adults across treatment modalities (individual, couple, and group) and clinical settings. Second, it is recommended using electronic versions of ROM systems that expedite and ease practical difficulties. Third real-time ROM feedback with an alert that identifies at risk cases to compensate for the limited ability of clinicians to accurately detect a client worsening in psychotherapy should be employed. Fifth, feedback progress reports and alerts should be evaluated as vital signs of patient progress, not as a reflection of one’s ability as a mental health practitioner. Finally problem-solving tools combined with a brief assessment of factors which are associated with deterioration in psychotherapy should be applied. They can help by promoting discussion with patients and problem solving with at-risk cases to provide additional clinical benefits beyond progress feedback alone. As many as 30 new ROM methods have been developed in recent years, amongst others in the working group around Wolfgang Lutz at the University of Trier in Germany, but few have been tested in clinical trials at this time.
Cite This Article
Lambert, M. J. & Kleinstäuber, M. (2019, March). Why psychotherapists should measure and monitor client treatment response. [Web article]. Retrieved from http://www.societyforpsychotherapy.org/why-psychotherapists-should-measure-and-monitor-client-treatment-response
American Psychological Association Presidential Task Force on Evidence-Based Practice. (2006). Evidence based practice in psychology. American Psychologist, 61, 271-285. doi:10.1037/0003-066X.61.4.271
Duncan, B. L. & Reese, R. J. (2015). The Partners for Change Outcome Management System (PCOMS): Revisiting the client’s frame of reference. Psychotherapy, 52, 52, 391-401.
Hannan, C., Lambert, M. J., Harmon, C., Nielsen, S. L., Smart, D. W., Shimokawa, K., et al. (2005). A lab test and algorithms for identifying clients at risk for treatment failure. Journal of Clinical Psychology: In Session, 61, 155–63.
Hansen, N. B., Lambert, M. J., & Forman, E. V. (2002). The psychotherapy dose-response effect and its implications for treatment delivery services. Clinical Psychology: Science and Practice, 9, 329–43.
Lambert, M. J. (2013). The efficacy and effectiveness of psychotherapy. In M. J. Lambert (Editor) Bergin & Garfield’s handbook of psychotherapy and behavior change (6th Ed., Pp. 169-218). New York: Wiley.
Lambert, M. J., Hansen, N.B., & Finch, A.E. (2001). Patient-focused research: Using patient outcome data to enhance treatment effects. Journal of Consulting and Clinical Psychology, 69, 159-172.
Lambert, M. J., Kahler, M., Harmon, C., Burlingame, G. M., Shimokawa, White, M. M. (2013). Administration and Scoring Manual: Outcome Questionnaire OQ ® -45.2. Salt Lake City: OQMeasures.
Lambert, M. J., Whipple, J. L. & Kleinstäuber, M. (2018). Collecting and delivering client feedback: A meta-analytic review of routine outcome monitoring. Psychotherapy, 55, 520-537. doi: 10.1037/pst0000167
Newham, E., Hooke, G. R., & Page, A. C. (2010). Progress monitoring and feedback in psychiatric care reduces depressive symptoms. Journal of Affective Disorders, 117, 139- 146. D0I: 10.1016/jad.2010.05.003
Prescott, D. S., Maeschalck, C. L., Miller, S. D. (2017). Feedback-informed treatment in clinical practice: Reaching for excellence. Washington, DC; American Psychological Association.
Sapyta, J., Riemer, M, & Bickman, L. (2005). Feedback to clinicians: Theory, research, and practice. Journal of Clinical Psychology, 62, 145–53.
Walfish, S., McAlister, B., O’Donnell, P., & Lambert, M. J. (2012). An investigation of self- assessment bias in mental health providers. Psychological Reports, 110, (2), 639-644. doi: 10.2466/02.07.17.