In 2007, the Institute of Medicine (IOM) recommended that health care consumers be granted access to provider performance data to inform treatment decisions. Theoretically, access to performance data would encourage patients to compare individual clinicians and preferentially choose the best performing clinician in a particular area of need or geographic location.

This recommendation relies on at least two basic premises:

1. there is measurable and clinically meaningful variability in health care provider outcomes, and
2. patients should be informed, active participants in their own health care decision making and service utilization.

Health care in the United States is extraordinarily complex. A “one size fits all” approach to health services research and clinical practice ignores the unique challenges faced by specific fields and disciplines.

Concretely, the metrics and standards for high quality care and relative provider performance necessarily depend on the level of analysis and care domain. For example, acute post-operation days of hospitalization may be a relevant outcome for cardiac surgeons, yet would be an odd metric to apply to psychotherapists.

Similarly, the relative importance of provider characteristics may differ across health service domains. For example, all other things being equal, a patient might tolerate an interpersonally cold cardiac surgeon if they have an impeccable outcome track record (e.g., 100% of their patients recover fully in short time and do not experience a second event in the next 5 years), yet the same patient may not tolerate an interpersonally cold psychotherapist, regardless of their performance track record.

Therefore, key questions for mental health care include:

• What are the relevant validated performance metrics?
• What metrics, provider and service characteristics do patients value mostly highly when making decisions regarding their care?

Measuring Therapist Performance

Psychotherapist performance can be assessed in a wide variety of domains. Research has identified significant variability in psychotherapists’ symptom and functioning outcomes (Baldwin & Imel, 2013), interpersonal skill (e.g., Anderson et al., 2016), ability to form a positive working alliance (e.g., Baldwin, Wampold, & Imel, 2007), and adherence and competence in implementing evidence-based treatments (e.g., Boswell et al., 2013). This research is based on both patient self-report and observational (i.e., external rater) methods, conducted in both controlled and naturalistic settings.

A significant source of our knowledge regarding variability in psychotherapy outcomes, both between psychotherapists and between individual patients treated by the same psychotherapist, has been derived from routine outcome monitoring (ROM) data.

Efforts to assess the quality of psychotherapy have focused significantly, although not exclusively, on symptom and functioning outcome monitoring (Brown, Scholle, & Azur, 2014). Our research team and collaborators have been working on the development and implementation of ROM in mental health care (Boswell, Kraus, Castonguay, & Youn, 2016; Boswell, Kraus, Miller, & Lambert, 2015), and the use of ROM-derived data to improve mental health care decision making (i.e., measurement-based care, MBC).

Consistent with other existing evidence, our research has demonstrated variability in mental health care provider outcomes. Concretely, some psychotherapists consistently produce positive symptom and functioning outcomes with most of their psychotherapy patients, other psychotherapists produce positive outcomes with certain types of patients and less positive outcomes with others, and a significant minority of psychotherapists consistently evidence poor symptom and functioning outcomes with their psychotherapy patients (Kraus, Castonguay, Boswell, Nordberg, & Hayes, 2011). Furthermore, psychotherapist effectiveness appears to be predictable and stable. In a recent study, we (Kraus et al., 2016) found that therapists who were particularly effective in reducing, for example, depressive symptoms in one wave of 30 patients in their caseload remained above average in their depressive symptom outcomes with a subsequent wave of 30 patients in their caseload.

The implications of these results are striking. Approximately 9 years after the IOM recommendation noted above, we can say to a given patient who is seeking psychotherapy – If you work with Dr. A, you have a high likelihood of experiencing improvements in your symptoms and functioning; If you work with Dr. B, you are unlikely to experience a benefit and may, in fact, experience a worsening in your condition(s) based on Dr. B’s track record with similar patients.

Despite the importance of this implication, psychotherapy consumers rarely have access to psychotherapist performance information and there remains a crucial knowledge gap regarding mental health care consumer perspectives on the value of such information.

Do Consumers Value Provider Performance, and is There More at Play?

Patient health care preferences are multidimensional. In addition to basic questions (e.g., Do patients value having access to provider performance data to inform their decisions?), little is known about patient preferences regarding mechanisms of provider performance information access or dissemination, or the relative value patients place on provider outcomes compared to other provider variables (e.g., demographic match, location, cost).

A meta-analysis of psychotherapy outcome studies found that patients who received their preferred treatment were significantly less likely to prematurely terminate treatment and experienced significantly better post-treatment outcomes (Swift, Callahan, & Vollmer, 2011).

Psychotherapist choice is a key component of patient-centered health care and shared-decision making. Results from a study conducted by Swift and Callahan (2010) demonstrated that patients may place a higher value on psychotherapist characteristics, such as empathy, experience, and non-directiveness, than the documented empirical support of a specific intervention. Mental health care systems (and federal funding agencies) have expended considerable effort and money to support the identification, dissemination, and implementation of empirically supported treatments (ESTs); however, empirically supported psychotherapists (ESPs) and scientifically informed patient-therapist matching has received little attention.

We conducted a pilot survey of mental health care stakeholders’ attitudes toward the use of provider track record information in treatment decision making (Boswell, Constantino, & Kraus, 2014). Results suggested that consumers generally value provider performance information and its use in decision making; yet, the relative value of this information in comparison to other provider characteristics, and the mechanisms through which consumers prefer to access and make use of this information remains unknown.

A more thorough understanding of consumer values and preferences, and the value they place on provider performance track records compared to other provider characteristics, may help consumers and other mental health care stakeholders make more personalized and informed treatment decisions, which, in turn, could lead to increased treatment engagement, retention, and better treatment outcomes.

We are in the process of conducting a mixed-method research study that is supported by a grant from the Robert Wood Johnson Foundation (#73049). In community mental health settings (CMHCs), we are examining diverse mental health care patients’:

1. attitudes and preferences regarding the use of provider outcome/performance information, and
2. the relative values placed on providers’ performance track records compared to other provider/treatment characteristics.

Specifically, we are utilizing semi-structured patient interviews and focus groups, as well as a survey methodology that integrates a delay-discounting paradigm (Critchfield & Kollins, 2001). Preliminary survey results were presented as part of Division 29 programming at the American Psychological Association Convention in Denver, Colorado (Boswell et al., 2016).

Summary and Conclusions

Psychotherapist choice is immensely important for patient-centered care; however, health care systems cannot effectively harness provider information (e.g., patient-provider matching algorithms) in the absence of relevant information on consumer valuing. We hope that our results will shed light on the actual values that consumers place on provider outcome information, and patients’ relative preferences for ESPs and other provider characteristics and treatment factors.

Acknowledgements

We would like to thank the Robert Wood Johnson Foundation for supporting this work, as well as Riverside Community Care, Family Continuity, Unity House, and Albany County Mental Health for partnering in this research.

+1

0

James F. Boswell, Ph.D.