Web-only Feature

Web-only Feature

Internet Editors Note: The authors recently published, “An Examination of Parent Sessions in Regulation Focused Psychotherapy for Children” which can be found in Division 29’s Psychotherapy. 

If you’re a member of the Society for the Advancement of Psychotherapy you can access the Psychotherapy article via your APA member page.

Not a member? Purchase the Psychotherapy article here. 

Professional practice is about making a difference with our clients—we want to help our clients reach their goals, and, we want our contribution to count. This is true in many professional settings, whether we offer treatment, counseling, training, education, or conduct applied research. But, how can we tell whether we actually have made a difference with our clients? This is not as easy as it may seem.

It Is Not About Change

In psychotherapy theory and practice, there has been a general tendency to focus on change as a sign that treatment is helpful. If clients experience change, we often tend to believe that we have successfully helped the client. On the other hand, if the client does not change, we may think we have failed to help the client.

In a recent paper (Langkaas, Wampold, & Hoffart, 2018), we argue that this may not be true at all, as making a difference could be completely unrelated of whatever change our clients exhibit. If treatment actually makes a difference, it is meant that the client would have experienced a different outcome without the treatment—treatment made a difference from what would otherwise have happened. Interestingly, the relationship between client change and treatment effect may be very different from the way we often expect it to be.

Consider a client who experiences positive change during treatment. But, it might have been the case that the client has a condition that probably would have led to even larger positive change even sooner without the treatment. This case would be an example of a client experiencing positive change while treatment likely made a negative difference. The example is not far-fetched. Several mental health problems tend to recover without treatment and treatment may inadvertently end up delaying recovery. Yet, the clinician and the client may be unaware of this, as they both still observe positive change.

Now, consider the opposite. A client experiences negative change during treatment. But, in this case it turns out that the client has a condition that would deteriorate even more even sooner without treatment. This would be an example of a client experiencing negative change while treatment likely made a positive difference. Again, this is not a far-fetched example either. Several mental health problems come with risk of recurring episodes of deterioration or relapse. Effective treatment would then seek to limit or prevent deterioration.

The main point here is that change is not always a sign that treatment helps—it could even be the opposite. Whether clients experience positive, negative or no change may be unrelated to whether treatment made a positive, negative or no difference. Knowing whether treatment made any difference at all requires knowledge about how various conditions are likely to develop without treatment—knowledge which is hard to come by.

Is Change Irrelevant?

Whether change is still relevant, depends. Clients want to achieve their goals, which often would mean experiencing change. In that case, change may still be relevant to our clients. But whether clients achieve change could still be unrelated to whether treatment made a difference. Clients may achieve change because of treatment or despite treatment. Thus, a natural question is: Under what circumstances does client change actually indicate that treatment helps?

Some psychotherapy researchers might respond that only randomized clinical trials could tell us if treatment makes a difference. But this seems to be at odds with common sense. Humans—and animals—can certainly learn how to make things happen in their environment without running large randomized trials, instead learning cause-effect relationships from their own experience. Construction workers can certainly learn from experience what actions they need to take to build a house.

However, there is an important difference between the art and science of psychotherapy and traditional craftsmanship. When working with inanimate objects, it is fairly certain that everything will remain unchanged without intervention—building materials do not tend to assemble themselves into a house. Thus, when a construction worker observes progress on a building project, this can reasonably be attributed to the actions of construction workers. When we attempt to help clients, this is quite different, as we generally do not expect humans to remain unchanged without psychotherapy. This is why, when clients experience change, it is difficult to know whether this was related to our professional contribution or not. This also means that if we take client change as sufficient evidence of helpful treatment, we effectively regard our client as inanimate—expected to remain unchanged without us. We know this to be a false premise when it comes to mental health, which is why client change alone is an unreliable sign of treatment effect.

Thus, the only circumstances where client change alone would indicate treatment effect are cases where clients will remain unchanged without us. This cannot be taken for granted in psychotherapy.

Does It Really Matter If Treatment Works?

From a pragmatic perspective, as long as clients reach their goals, does it really matter whether treatment made any difference? We could perhaps argue this point if a client has no expectation that our contribution would make much difference. But clients usually invest time, energy and money precisely because they expect psychotherapy to make a difference. As psychotherapists, we build our reputation on our ability to make a difference. We could then perhaps argue that research generally indicates that psychotherapy makes a difference. But, psychotherapy is never absolutely guaranteed to make a difference in a particular case. This means that there are several important opportunities that would be available to us if we were to know more precisely, in each particular case, when treatment works and when it does not:

  • We could discriminate between cases where our contribution made a difference and when it did not.
  • We could then learn from our successes and failures and improve our clinical skills.
  • We could create prediction models to predict whether ongoing cases were likely to benefit from continued treatment or not—informing clinical decisions about how to proceed with each case.
  • We could identify effective therapists and learn from them.
  • We could measure the performance of therapists under training and determine whether they had received sufficient training or not.

We present these opportunities as hypothetical, knowing that there already exist several attempts at realizing them:

  • The growing field of routine outcome monitoring regards client change as indication of treatment progress that can guide clinical decisions (e.g., Barkham, Mellor-Clark, & Stiles, 2015; Duncan & Reese, 2015; Lambert, 2015). But, client change does not indicate whether treatment has made a difference.
  • To guide clinical decisions, current research on psychotherapy feedback attempts to predict whether clients will experience change or not (e.g., Delgadillo et al., 2018; Lambert & Shimokawa, 2011). But, this is not the same as predicting whether treatment will make a difference.
  • Expertise research attempts to identify expert clinicians by their client outcomes (e.g., Brown, Lambert, Jones, & Minami 2005; Saxon & Barkham, 2012). But, client outcomes do not indicate how much difference clinicians made to their clients. As a consequence, we may fail to learn what distinguishes the real experts from other clinicians.

Such attempts to provide clinical guidance are generally based on the premise that client change is sufficient evidence that treatment works—a premise we have argued likely to be false. Thus, until we have found a way to solve the problem of how to actually monitor whether treatment works, we cannot truly depend on the practical guidance of any of these attempts.

Where To Go From Here

In our paper (Langkaas, Wampold, & Hoffart, 2018) we present the idea of monitoring induced difference—comparing actual case progress to likely progress without treatment—as a better indication of whether treatment is helpful, and as a possible direction for future development of routine outcome monitoring to guide professional practice. We believe that psychotherapy theory and practice in general would benefit from a joint effort to improve how we currently approach monitoring progress and outcome in practice. As of now, we have the following advice on what psychotherapists currently can do about the issues we have discussed:

  • Be aware that client change in itself does not indicate that treatment helped—client change and treatment effect could be unrelated or even inversely related.
  • Continue to monitor whether clients reach their goals, without regarding this as sufficient evidence of treatment being helpful.
  • Know that ranking of therapists according to their client outcomes may not reflect actual therapist performance, as client change is an unreliable indicator of treatment effect.
  • Be cautious of the practical guidance offered by current systems and approaches to routine outcome monitoring. They cannot indicate whether treatment works or whether it is likely to work as long as they mainly track client change.
  • Continue to seek guidance from experimental research, such as randomized clinical trials, as this is still our best source for guidance about what treatments or what factors that are likely to make a difference to a client.
Cite This Article

Langkaas, T. F., Hoffart, A., & Wampold, B. E. (2018, December). Did I make a difference with my client? [Web article]. Retrieved from http://www.societyforpsychotherapy.org/did-i-make-a-difference-with-my-client


Barkham, M., Mellor-Clark, J., & Stiles, W. B. (2015). A CORE approach to progress monitoring and feedback: Enhancing evidence and improving practice. Psychotherapy52(4), 402–411. https://doi.org/10.1037/pst0000030

Brown, G. S., Lambert, M. J., Jones, E. R., & Minami, T. (2005). Identifying highly effective psychotherapists in a managed care environment. American Journal of Managed Care11(8), 513–520.

Delgadillo, J., de Jong, K., Lucock, M., Lutz, W., Rubel, J., Gilbody, S., … & O’Hayon, H. (2018). Feedback-informed treatment versus usual psychological treatment for depression and anxiety: a multisite, open-label, cluster randomised controlled trial. The Lancet Psychiatry5(7), 564–572. https://doi.org/10.1016/S2215-0366(18)30162-7

Duncan, B. L., & Reese, R. J. (2015). The Partners for Change Outcome Management System (PCOMS) revisiting the client’s frame of reference. Psychotherapy52(4), 391–401.  https://doi.org/10.1037/pst0000026

Lambert, M. J. (2015). Progress feedback and the OQ-system: The past and the future. Psychotherapy52(4), 381–390. https://doi.org/10.1037/pst0000027

Lambert, M. J., & Shimokawa, K. (2011). Collecting client feedback. Psychotherapy, 48, 72–79. https://doi.org/10.1037/a0022238

Langkaas, T. F., Wampold, B. E., & Hoffart, A. (2018). Five types of clinical difference to monitor in practice. Psychotherapy, 55(3), 241–254. https://doi.org/10.1037/pst0000194

Saxon, D., & Barkham, M. (2012). Patterns of therapist variability: Therapist effects and the contribution of patient severity and risk. Journal of Consulting and Clinical Psychology80(4), 535–546. https://doi.org/10.1037/a0028898


1 Comment

  1. Elisabet Rosén

    This is really interesting and in my opinion good research, sins I know from before that Wampold is a “fan of” ROM (routine outcome monitoring), and here you’re rising a huge questionmark around what the outcomemeasures really means!


Submit a Comment

Your email address will not be published. Required fields are marked *