Web-only Feature

Web-only Feature

Internet Editor’s Note: Dr. Derek Griner and colleagues recently published an article titled “Practice-based evidence can help! Using the Group Questionnaire to enhance clinical practice” in Psychotherapy.

If you’re a member of the Society for the Advancement of Psychotherapy you can access the Psychotherapy article via your APA member page.

Not a member? Purchase the Psychotherapy article for $11.95 here. 

Evidence-Based Practice (EBP) represents a standard of practice as clinicians seek increasingly effective ways to provide therapeutic services to their clients. Comprised of three main approaches, EBP seeks to apply a structured, systematic approach to the provision of therapy through the use of empirically supported treatments, practice guidelines, and practice based evidence (Burlingame & Beecher, 2008). While each represents an aspect of EBP, differing assumptions underlie these approaches. For example, empirically-supported treatments are typically focused on therapist activity in the therapy session, and as such are dependent on the theory that underlies the approach. Practice Based Evidence (PBE), on the other hand, involves using real time client data, gathered through standardized measures, to understand and adjust the therapeutic process in a theory neutral approach. With PBE, effectiveness is measured not through adherence to an empirically-driven protocol for therapy, but rather through meaningful client data gathered through the course of therapy (Burlingame & Beecher, 2008). For the purposes of this article we will focus on PBE specifically as used in a group therapy setting.

Hatfield and Ogles (2007) investigated the use of outcome measures, a type of PBE, in therapy. They found that a considerable number do not use outcome measures in practice. These authors identified a number of reasons why clinicians chose not to utilize such measures. The primary reason involved practical concerns such as administrative burden, logistics of administration, or cost, and these concerns cut across theoretical orientation (CBT, insight-oriented, or eclectic). A second reason involved utility concerns, or feeling as if the measures either were not helpful or might distort the effects of treatment. Endorsement of utility concerns was somewhat higher for insight-oriented and eclectic clinicians than for those who practice CBT. It should be noted that these findings were consistent across clinician setting (institutional, managed-care/insurance, and fee-for-service). The authors conclude by issuing a call for the development of outcome measures that are effective and efficient in order to surmount the barriers that practical and utility concerns present.

Valid and reliable measures that are both effective and efficient can augment clinical intuition in clinical practice and the information gathered through these measures can help identify clients who are not benefiting from therapy. For example, Hannan et al. (2005) showed that clinicians were inaccurate in their ability to predict treatment response in individual therapy, while Chapman et al. (2012) demonstrated much the same for clinicians who practice group therapy. Hatfield, McCullough, Frantz, and Krieger (2009) showed that clinicians had considerable difficulty recognizing client deterioration when these clinicians reviewed case notes. While clinical judgment in the therapy (or group) room is essential, it appears to be a necessary but not sufficient tool for creating optimal client outcomes.

The Group Questionnaire

Fortunately, there are PBE measures that can answer the call. The Group Questionnaire (GQ) is one measure that illustrates how PBE can provide meaningful, effective, and efficient data to group therapists. The GQ is based on findings that group outcomes improve as constructs such as climate, empathy, and alliance are increased (Law, et al., 2012; Gillaspy, Wright, Campbell, Stokes, & Adinoff, 2002; Norcross & Wampold, 2011; Summers & Barber, 2003; Budman et al., 1989). The GQ measures subsuming factors labeled Positive Bond (cohesion and engagement), Positive Work (agreement on the tasks and goals of therapy), and Negative Relationship (conflict and empathic failures) (Griner et al., 2018). While there are a number of published studies that more fully illustrate the characteristics of the GQ itself (e.g., Krogel et al., 2013; Thayer & Burlingame, 2014; Burlingame et al., 2018), we will briefly review how the GQ, as a PBE measure, speaks to the concerns raised by the Hatfield and Ogles (2007) study.

The GQ addresses practicality concerns, which were identified as the primary reason clinicians identified for not using outcome measures. Hatfield and Ogles (2007) showed that clinicians were reluctant to use measures that would add more paperwork, take too much time, increase costs (both money and personnel), or otherwise burden clients. These are valid concerns, particularly as clinicians must balance the information gained from using measures with the effort expended to use them. The GQ alleviates these concerns in a number of ways. First, the GQ has an online version that is scored instantly through the OQ-Analyst website, considerably reducing any time or personnel requirements that might be involved with scoring the measure (Griner et al., 2018). Burden on clients completing the GQ is minimal, as the 30 items typically take only five minutes to complete and can be taken from clients’ own smartphones or electronic devices. Time and effort spent interpreting the GQ are also minimal, as the online GQ report has numerical and graphical indicators for each of the three GQ factors for each individual and the group as a whole. Furthermore, the GQ has an alert system that conveys positive or negative change in group members’ scores on Positive Bond, Positive Work, and Negative Relationship, and these alerts are conveyed through changes in text color and through smiling or frowning face symbols. In short, a group therapist using the GQ can get a sense of members’ progress in group simply by glancing at the interpretive report. Therapists also have the option to review each individual’s score in detail for more information.

The GQ addresses many of the utility concerns identified by Hatfield and Ogles (2007). These concerns involved clinician perceptions that measures may interfere with clinician autonomy, could distort the effects of treatment, or would not be helpful. For example, clinicians may fear that autonomy is constrained when they use a measure that does not fully capture the intent or process of their therapeutic approach. To this end, the GQ emphasizes the key factors involved in group therapy outcome from an atheoretical framework. Additionally, clinicians are free to determine when and how they will utilize the GQ in their group therapy, with some clinicians choosing to use GQ data collaboratively with clients and others electing to review data outside of group on their own (Griner et al., 2018).

Fears of outcome measures distorting treatment or not being helpful should also be minimized by using the GQ. Partially due to the atheoretical underpinnings of the GQ and partially due to the focus on client activity (rather than clinician intervention) in group, clinicians who use the GQ can get a more accurate picture of client perceptions of group work. Given the difficulties group leaders have in predicting treatment response based on clinical judgment alone, a PBE approach provides more information to inform group work and can help clinicians adjust treatment to reduce early client deterioration and dropout. As noted in the Griner et al. (2018) article, the GQ is meant to supplement clinical judgment, not replace it.

In sum, PBE approaches represent an efficient and effective means to optimize the therapeutic services clinicians provide to clients. This article has illustrated how the GQ can minimize or eliminate common sources of resistance to using PBE approaches in group therapy. Clinicians would be wise to consider implementation of PBE approaches in their practice as these approaches have potential for considerable benefit in both practical and utility domains.

Klinton Hobbs is an Assistant Clinical Professor with Brigham Young University's Counseling and Psychological Services.

Cite This Article

Hobbs, K., Hansen, K., Griner, D., Beecher, M., Worthen, V., & Burlingame, G. (2019, January). The Group Questionnaire: A practical and useful practice-based evidence measure. [Web article]. Retrieved from http://www.societyforpsychotherapy.org/the-group-questionnaire


Budman, S.H., Soldz, S., Demby, A., Feldstein, M., Springer, T., & Davis, M.S. (1989). Cohesion, alliance, and outcome in group psychotherapy. Psychiatry, 52(3), 339-350.

Burlingame, G.M. & Beecher, M.E. (2008). New direction and resources in group psychotherapy: Introduction to the issue. Journal of Clinical Psychology, 64(11), 1197-1205.

Burlingame, G. M., Whitcomb, K. E., Woodland, S. C., Olsen, J. A., Beecher, M., & Gleave, R. (2018). The effects of relationship and progress feedback in group psychotherapy using the Group Questionnaire and Outcome Questionnaire–45: A randomized clinical trial. Psychotherapy, 55(2), 116-131.

Chapman, C.L., Burlingame, G.M., Gleave, R., Rees, F., Beecher, M., & Porter, G.S. (2012). Clinical prediction in group psychotherapy. Psychotherapy Research, 22(6), 673-681.

Gillaspy, J.A., Wright, A.R., Campbell, C., Stokes, S., & Adinoff, B. (2002) Group alliance and cohesion as predictors of drug and alcohol abuse treatment outcomes. Psychotherapy Research, 12(2), 213-229.

Griner, D., Beecher, M.E., Brown, L.B., Millet, A.J., Worthen, V., Boardman, R.D., Hansen, K., Cox, J.C., & Gleave, R.L. (2018). Practice-based evidence can help! Using the group questionnaire to enhance clinical practice. Psychotherapy, 55(2), 196-202.

Hannan, C., Lambert, M.J., Harmon, C., Nielsen, S.L., Smart, D.W., Shimokawa, K., & Sutton, S.W. (2005). A lab test and algorithms for identifying clients at risk for treatment failure.  Journal of Clinical Psychology, 61(2), 155-163.

Hatfield, D., McCullough, L., Frantz, S.H.B., & Krieger, K. (2009). Do we know when our clients get worse? An investigation of therapists’ ability to detect negative client change. Clinical Psychology and Psychotherapy 17(1), 25-32.

Hatfield, D.R. & Ogles, B.M. (2007). Why some clinicians use outcome measures and others do not. Administration and Policy in Mental Health and Mental Health Services Research, 34(3), 283-291.

Krogel, J., Burlingame, G., Chapman, C., Renshaw, T., Gleave, R., Beecher, M., & MacNair-Semands, R. (2013). The Group Questionnaire: A clinical and empirically derived measure of the group relationship. Psychotherapy Research, 23(3), 344-354.

Norcross, J.C. & Wampold, B.E. (2011). Evidence-based therapy relationships: Research conclusions and clinical practices. Psychotherapy, 48(1), 98-102.

Law, T., Lee, K.Y., Ho, F.N., Vlantis, A.C., van Hasselt, A.C., & Tong, M.C. (2012). The effectiveness of group voice therapy: A group climate perspective. Journal of Voice, 26(2), e41-8.

Summers, R.F. & Barber, J.P. (2003). Therapeutic alliance as a measurable psychotherapy skill. Academic Psychiatry, 27(3), 160-165.

Thayer, S. & Burlingame, G. (2014). The validity of the Group Questionnaire: Construct clarity or construct drift? Group Dynamics: Theory, Research and Practice, 18(4), 318-332.

1 Comment

  1. Marnin Romm

    Good day,
    My name is Marnin Romm. I am a current PhD student at the University of Miami specializing in Pain Management.

    I came across literature on group questionnaire that has been developed. I have a very keen interest in group-based intervention and hence my interest in your article on the questionnaire.

    I am doing research on group-based intervention and was wondering if I could somehow get hold of the actual GQ instrument?


Submit a Comment

Your email address will not be published. Required fields are marked *