Walk the Walk, Talk the Talk, Do the Work
Recommendations for Conducting Research With Diverse Populations
Todd Ryser-Oatman is enrolled in the University of Kentucky Counseling Doctoral Program. His research interests focus on the well-being of the LGBTQ community. His dissertation focuses on help-seeking experiences of sexual minority men who have experienced intimate partner violence in their same-sex relationships.
Alyssa Clements-Hickman is also enrolled in the University of Kentucky Counseling Doctoral Program, where she primarily focuses on applied psychotherapy research. Her research interests broadly include psychotherapy process and outcome.
Their doctoral training program has a strong emphasis on social justice practice and has afforded them many opportunities to engage in applied research studies that focus on diverse populations. In the following article, the authors share their experiences and recommendations for conducting research with diverse samples, primarily focusing on sexual minority individuals.
Doing research with diverse populations is often championed in psychotherapy training programs. However, there is a tendency to ignore social identity and diverse identities in both research and practice (Sue, 1999). This may be in part due to using convenience sampling, pressure to complete projects in a timely manner, or because researchers simply do not know how to recruit from these groups in effective ways. Below we discuss methods for engaging in research with a desire to disrupt research that reflects the experiences of majority group individuals.
Take a Culturally Humble Stance
There has been a recent shift in psychotherapy from one of cultural competence to one of cultural humility. Cultural humility involves remaining open to learning from clients, while competence focuses on acquiring a set amount of skill (Gelso, Nutt Williams, & Fretz, 2014). Humility involves maintaining a “stance that is other-oriented (or open to the other) in relation to aspects of cultural identity” (Davis et al., 2018, p. 91). Many articles have been published on the advantages of maintaining a culturally humble stance in the therapeutic context (e.g., Owen et al., 2016). These writings provide therapists with guidance regarding concrete ways to embody cultural humility with clients. Unfortunately, little has been written on the importance of taking a culturally humble stance when conducting research with diverse populations. Although our methodologies are scientific and seemingly objective in nature, our work is always infused with our own biases and worldviews as we conduct research (Fassinger & Morrow, 2013). It is therefore necessary to continuously reflect on how we attempt to represent marginalized voices in our work.
We have found that it is important to take a culturally humble stance throughout the research process (e.g., recruitment, development of interview/protocol questions, executing interviews, presenting data, and choosing how to present findings about diverse groups). Regarding recruitment, researchers have begun to describe the importance of developing connections with the community you wish to recruit from, such as developing partnerships where researchers actively contribute outside of trying to gather participants (e.g., Kano, Sawyer, & Willging, 2016). This is especially critical given the positionality of researchers being in positions of power relative to participants from diverse communities (Chavez, Duran, Baker, Avila, & Wallerstein, 2008). For instance, as students we often try to collect data from diverse groups in order to publish and receive their degree without taking their work and finding ways to apply it to working with the groups they collected data from. In this sense, a power imbalance is created by taking resources from their sample without finding a way to make their findings matter to the group who they want represented in their data. For instance, in my (Todd) dissertation, I plan not only to send my completed manuscript to participants, but also hope to disseminate my completed manuscript to local organizations that work with sexual minority individuals as well as domestic violence survivors to raise awareness about how these two identities intersect. In this sense, I plan on using my participants’ willingness to share vulnerable stories about a traumatic experience with intimate partner violence in such a way that it could benefit future service providers working with these groups.
Locate Resources and Faculty That Can Help You
With many projects, it is critical that you have a knowledgeable, accessible faculty member who will support you throughout the research process. This may be particularly critical if you are seeking participants with diverse identities from communities with whom you have never worked. By engaging with a faculty advisor with social consultation experience, for example, my colleagues and I (Todd, who lacked such experience) were able to learn about theoretically grounded models of consultation, the process of developing collaborative goals with stakeholders, and how to navigate barriers to our project throughout the process (Clements et al., 2018).
It is also important to consider where it is best to collect one’s sample. For instance, researchers often collect data from educational settings, particularly from college students. This, however, may not be an effective way to collect data from marginalized groups (Hooven, Walsh, Willgerodt, & Salazar, 2011). And, furthermore, some research has identified that traditional research methods, such as random or probability sampling, may not be as effective as social networking recruitment (Bull, Levine, Schmiege, & Santelli, 2013). For this reason, it is helpful to turn to faculty who do research with marginalized groups, to seek literature about sampling, and to even engage with community leaders who are associated with a population of interest.
Utilize a Strengths-based Framework
Despite researchers’ frequent focus on barriers of diverse groups, it is critical that findings are disseminated so as to advocate for marginalized groups (Fassinger & Morrow, 2013). It is important to understand what barriers these groups face; however, it is critical that as psychologists we also ask what helps diverse groups thrive and locate their strengths. For instance, community-based research (CBR) is a framework used to build on strengths of a community, sharing social power, promoting co-learning between participants and researchers, and other principles of empowerment that develop a dynamic that emphasizes uplifting diverse communities (Chavez et al., 2008). Indeed, such frameworks can help shape how we as researchers tailor our research approach while honoring historical contexts between research institutions and marginalized groups. For instance, Kano and colleagues (2016) note that their sexual and gender minority participants using CBR wished that researchers knew more about the context of health services and the LGBT community, as it promotes transparency and authenticity of researchers hoping to improve outcomes for the community being studied. Understanding the cultural contexts and history of a studied population is critical to social justice research, as it can allow researchers to focus on social change as a goal and create ease with engaging with communities of interest during the research process by communicating knowledge about that group and their historical context in relation to psychology (Fassinger & Morrow, 2013).
Using a strengths-based framework proved particularly useful in a consultation project the co-authors recently participated in for a midwestern city’s homelessness prevention and intervention office. The goal of the organization was to not only prevent homelessness but also to help people exit homelessness quickly so as to reduce their chances of staying there. A quick exit from homelessness is critical as many homeless individuals can get caught in a cycle. For instance, in our consultation work, we met several individuals who were incarcerated, which led to outcomes such as loss of housing and relapse into substance abuse. Our qualitative and quantitative findings revealed that this population faced several challenges, including mental health concerns, feeling unsafe, substance abuse, involvement with correctional facilities, and housing instability (Clements et al., 2018). In response to literature that is deficit-focused, our consultation team focused on identifying the strengths of the individuals who are homeless. For instance, our consultation goals not only focused on barriers to leaving homelessness, but also identifying strengths. Based on responses to our focus groups, we learned that individuals who are homeless are resourceful, build strong communities, and have a strong drive to create better futures for themselves. By focusing on strengths as well as growth areas, we were able to provide formative feedback to our stakeholders that framed individuals who are homeless as resilient, resourceful, full of potential that can be tapped into, while still leaving room for feedback about how services could be improved to end homelessness.
Developing Research Questions for Qualitative Research: Recommendations From Todd
Part of understanding what kinds of research questions to ask is understanding the literature about a population of interest and having a desire to highlight strengths. This is critical when marginalized groups typically have research that highlights deficits. For instance, while I was developing my dissertation proposal, the population I focused on was men who have sex with men (MSM). When reading literature about this population, nearly every study I read focused on risk factors and negative health outcomes. Although my topic area is intimate partner violence, part of my identity as a counseling psychologist is to take a strengths-based approach, not only with my therapy but with my research as well. This branding led to my dissertation project to find out what strengths of sexual minority men facilitate help-seeking for intimate partner violence. In this sense, rather than focus on the risk factors that sexual minority men face in response to intimate partner violence, such as increased sexually transmitted infection risk, substance abuse potential, and mental health concerns (Stall, Friedman, & Catania, 2008), I tried to frame sexual minority men as having strengths in the face of even a difficult issue such as interpersonal violence. I wrote my dissertation questions in a way to not only elicit the challenges sexual minority men faced with intimate partner violence, such as their experiences with abuse and barriers to seeking out psychological help, but, in addition, to focus on what strengths facilitate seeking help, what promotes leaving abusive relationships, and what meaning they were able to make of their experiences. In this way, I hope to frame an experience that is fraught with risk for negative outcomes in a way that strengthens a marginalized group.
Recommendations for Recruitment
Recruitment of underrepresented research participants presents both challenges and opportunities to use innovative sampling techniques. For example, I (Todd) use a variety of methods to reach sexual minority individuals. For those in metropolitan areas, it is helpful put to flyers up in LGBT organizations, particularly in community centers and bars. If you are not sure about what organizations are in your area, consider using Google to help identify LGBT organizations, as some restaurants and hotels are known within the community and could be useful for study advertisement. Regardless of where you post, always ask for permission before posting a flyer. Posting more flyers, while requiring my time, can help recruit participants who may not use social media. I have used research funding to send flyers to LGBT bars and organizations in different states, which might be useful if you do not have access to locations to hang flyers locally. For my dissertation, I am attempting to interview sexual minority men in the southeastern United States. Living in Kentucky, I am only able to access a small portion of my desired sample, so it has been helpful to mail flyers with a brief note explaining my study and asking if organization owners would be willing to display them. Lastly, conducting recruitment in person is an effective way to find participants. For my dissertation project, I have engaged in distributing flyers both in person and by mailing flyers to collect data. In-person flyer distribution can be helpful, as you can engage individuals directly and talk about your study. This can be effectively done at festivals and events in your area that might draw a diverse group of individuals. It is important to consider the topic of your study when passing out flyers; for instance, my dissertation focuses on intimate partner violence in same-sex relationships. Due to the sensitive subject matter, I opted to place flyers at vendor stands as opposed to approaching men to participate when passing out flyers at my local Pride Festival.
Social media is an increasingly common way researchers are recruiting underrepresented groups (e.g., Guillory et al., 2018). This is largely due to the massive number of users present and the ability to search for diverse groups by markers such as race, gender, sexual orientation, and other social identities. While it is more convenient to reach large groups of individuals at once, it can be overwhelming to sift through Facebook or find other websites from which to recruit. As I work on my dissertation project, I have used several online resources to help me recruit participants. Initially, I develop an Excel table with tabs for the various social media websites from which I plan to recruit, with columns noting specific group name, hashtags used, the date I message website moderators, the date I post my study information, and a few columns for additional postings in the future. Furthermore, I create a tab for participants who have reached out to me to participate, with space to denote when I followed up with them, when we scheduled interviews, when transcription was done, all the way through the coding process. For quantitative research this may be more difficult, but it still highly recommended to have some sort of organizational framework. It is so easy to get overwhelmed by how many places you can post your research information, so it is necessary to stay organized.
Typically, I recruit using the following platforms: Facebook, Twitter, Instagram, Tumblr, and Reddit. Depending on the website you use to recruit and the subgroup you are targeting (e.g., sexual minority, racial minority) you will likely need to develop different forms of ads. For instance, Guillory and colleagues (2018) developed study ads that were targeted at specific subgroups, such that ads targeting sexual minority men were designed differently than those targeting sexual minority women; examples of their flyers can be seen within their study, and they serve as a helpful template when developing recruitment materials. Each of these websites function differently, but several ground rules should be recognized. First, it is important to contact page administrators (Vincent, 2018) to request permission to post a study on their social media page. I usually let them know who I am and my credentials, what the purpose of my study is, the blurb I will be posting, and the image of my flyer. When posting, I also found more participants when my posted blurbs are succinct, providing a link to my survey and a flyer with more information. Particularly, if you identify similarly to a group you wish to recruit, identifying yourself that way can make you appear more relatable to participants. For instance, when posting on LGBT-related social media pages I often try to identify myself as a member of the community; this feels particularly important given the historical context of health services harming the LGBT community in several instances (e.g., Berger, 1994). Lastly, it is important to consistently check on your social media posts for comments and questions. Frequently, I have had participants ask for more study details, for me to contact them directly, or for clarification about the study.
Conducting research with diverse groups is a challenging endeavor. It is easy to collect convenience samples and ignore the voices of diverse groups. The set of recommendations we have provided is by no means exhaustive, but we encourage others in the field to reflect on the ways we can strive to respectfully collect data from marginalized populations.
Cite This Article
Ryser-Oatman, T., & Clements-Hickman, A. (2019). Walk the walk, talk the talk, do the work: Recommendations for conducting research with diverse populations. Psychotherapy Bulletin, 54(3), 41-48.
Berger, J. (1994). The psychotherapeutic treatment of male homosexuality. American Journal ofPsychotherapy, 48(2), 251-261. https://doi.org/10.1176/appi.psychotherapy.
Bull, S. S., Levine, D., Schmiege, S., & Santelli, J. (2013). Recruitment and retention of youth for research using social media: Experiences from the Just/Us study. Vulnerable Children and Youth Studies, 8(2), 171-181. https://doi.org/10.1080/17450128.2012.748238
Chavez, V., Duran, B., Baker, Q. E., Avila, M. M., & Wallerstein, N. (2008). The dance of race and privilege in community-based participatory research. In M. Minkler & N. Wallerstein (Eds.), Community-based participatory research for health: From process to outcomes (pp. 91-105). San Francisco, CA: Jossey-Bass.
Clements, A., Dschaak, Z., Hargons, C. N., Kwok, C., Meiller, C., Ryser-Oatman, T., & Spiker (2018). Humanity in homelessness: A social justice consultation course for counseling psychology students. Journal for Social Action in Counseling and Psychology, 10(2), 34-48. Retrieved from https://openjournals.bsu.edu/jsacp/article/view/1061
Davis, D. E., DeBlaere, C., Owen, J., Hook, J. N., Rivera, D. P., Choe, E., … Placeres, V. (2018). The multicultural orientation framework: A narrative review. Psychotherapy, 55(1), 89-100. http://dx.doi.org/10.1037/pst0000160
Fassinger, R., & Morrow, S. L. (2013). Toward best practices in quantitative, qualitative, and mixed-method research: A social justice perspective. Journal for Social Action in Counseling and Psychology, 5(2), 69-83.
Gelso, C. J., Nutt Williams, E., & Fretz, B. R. (2014). Counseling Psychology (3rd ed.). Washington, DC: American Psychological Association.
Guillory, J., Wiant, K. F., Farrelly, M., Fiacco, L., Alam, I., Hoffman, L., … & Alexander, T. N. (2018). Recruiting hard-to-reach populations for survey research: Using Facebook and Instagram advertisements and in-person intercept in LGBT bars and nightclubs to recruit LGBT young adults. Journal of medical Internet research, 20(6), e197.
Hooven, C., Walsh, E., Willgerodt, M., & Salazar, A. (2011). Increasing participation in prevention research: Strategies for youths, parents, and schools. Journal of Child and Adolescent Psychiatric Nursing, 24(3), 137-149. https://doi.org/10.1111/j.1744- 6171.2011.00288.x
Kano M, Sawyer KP, Willging CE. “Guidelines for Conducting Research WITH LGBTQ+ Individuals and Communities in New Mexico.” Albuquerque, NM. NM CARES Health Disparities Center, Community Engagement Core, University of New Mexico. 2016.
Owen, J., Tao, K. W., Drinane, J. M., Hook, J., Davis, D. E., & Kune, N. F. (2016). Client perceptions of therapists’ multicultural orientation: Cultural (missed) opportunities and cultural humility. Professional Psychology: Research and Practice, 47(1), 30-37.http://dx.doi.org/10.1037/pro0000046
Stall, R., Friedman, M., & Catania, J. A. (2007). Interacting epidemics and gay men’s health: A theory of syndemic production among urban gay men. In R. J. Wolitski, R. Stall, & R. O. Valdiserri (Eds.), Unequal opportunity: Health disparities affecting gay and bisexual men in the United States (pp. 251-274). New York, NY: Oxford University Press.
Sue, S. (1999). Science, ethnicity, and bias: Where have we gone wrong? American Psychologist, 54(12), 1070-1077. http://dx.doi.org/10.1037/0003-066X.54.12.1070
Vincent, B. W. (2018). Studying trans: Recommendations for ethical recruitment and collaboration with transgender participants in academic research. Psychology & Sexuality, 9(2), 102-116. https://doi.org/10.1080/19419899.2018.1434558