Throughout 2017, the Psychotherapy Research Committee and the Scholarship Domain have been providing Psychotherapy Bulletin articles with recommendations for sharing our research with others. In the first Bulletin issue of the year, we included suggestions for sharing our research with policy makers. In the second issue, we focused on sharing our research with psychotherapy clients. In the third issue, we discussed sharing research with the media. In this issue, we would like to provide recommendations for sharing research with the professional field. Specifically, the focus of this article is on preparing manuscripts for publication in peer-reviewed psychotherapy research journals.

In preparing our comments, we sought input from the editors and associate editors of three journals that publish psychotherapy research—our society’s journal (Psychotherapy), the Society for Psychotherapy Research’s journal (Psychotherapy Research), and the Society for the Exploration of Psychotherapy Integration’s journal (Journal of Psychotherapy Integration). These editors were asked to provide their opinions about how to write a high quality manuscript. They were asked to not focus on the actual quality and contribution of the research study, but rather, aspects of the written manuscript itself. Here we provide a summary of their suggestions, split by sections of the article, including the abstract, introduction, method, results, and discussion.

Before we get to those sections though, one of the editors strongly recommended researchers start their manuscript preparation by taking a few minutes to read the instructions to authors on journal websites. The journal websites often have information about the scope and aims of the journal that can guide authors in how to tailor their manuscript to the audience of the journal. In addition, the instructions to authors often include information about page lengths, abstract word count and format, style, and additional items that should be submitted with the manuscript (e.g., clinical significance statement, ethics statements that should be included in the cover letter). When some of the basic requirements of the journal are not addressed, the peer-review process will likely not start off on the best foot.

Second, the Journal Article Reporting Standards (APA Publications and Communications Board Working Group on Journal Article Reporting Standards, 2008) provide detailed guidance on what specific information should be included in each section of the manuscript. These recommendations cover everything from what words to include in the title to the proper description of the recruitment method. Authors should familiarize themselves with these standards for the field prior to beginning the manuscript.

Abstract

The editors and associate editors all emphasized that clarity is a priority when writing an abstract. Having an opening sentence describing the value of the study in a straightforward but attention-grabbing way is essential. The abstract should not cover the background literature extensively; a brief statement showing the need for the current study is sufficient. That way, the abstract can focus on the main aims of the manuscript, the methods used to study those research questions, and the primary results. Authors should also pay careful attention to abstract word limits. It may not be possible to cover all of the findings in the abstract. Rather than highlighting only the statistically significant results, an abstract should present the results most relevant to the main objectives of the study.

Introduction

When it comes to the introduction of the manuscript, the editors and associate editors that were surveyed discuss the importance of “hooking” readers. One editor suggested that authors should attempt to answer the question, “Why is the paper worth reading?” in this section of the manuscript. In addition, the introduction should have a clear focus and remain on topic throughout.

The introduction should be organized like a funnel, starting with the general problem and moving to a more detailed review of the research directly related to the study on which the manuscript is based. Several editors and associate editors talked about giving sufficient information in the introduction so that the study makes sense in the context of the background literature. The introduction should provide enough evidence to support why the study needs to be done. Describing what has been done and the gaps in the literature the current study fills is necessary, but not sufficient, in building a case for the study. For example, just because no one has studied what color shoes psychotherapists wear when providing treatment doesn’t mean that a study on that is needed. Instead, focus on the actual contribution and generation of knowledge from the study. One editor suggested that, in providing a convincing rationale for a study, “the strongest introductions make an argument, present a counter-argument, and then counter the counter-argument.”

Several of the editors and associate editors stated the introduction section should always end with the hypotheses—be specific and don’t leave the readers guessing. The hypotheses should not be a surprise to readers; rather, the literature reviewed throughout the introduction should provide a rationale for the hypotheses made. There should also be sufficient information so the introduction makes sense in the context of the method section—reviewers should also not be surprised once they get to the methods.

In preparing the introduction it is also important to pay attention to the style formatting required by the journal. This is particularly important in the introduction, as it provides the first impression for the manuscript. One associate editor stated, “The more mistakes, the more it detracts from understanding the substance.” Another editor explained, “Reviewers notice little errors (regardless of whether they take time to comment on each individual error or not) and have a subjective tally running in their heads as they read the manuscript. When too many errors add up, reviewers form an unfavorable impression that often appears (to me in the editor role) as generalizing about the entire study.” This is true not just for grammatical errors, but also for the general organization and flow of the introduction.

Method

The surveyed editors and associate editors emphasized that the methods section of a manuscript is not the place to be creative. Although creativity may have played an important role in the design of the study, the description of the Method should be straightforward, detailed, and follow the standards of the field. Remember, the goal of the Method section is to allow readers to know exactly what was done with enough precision that, if they would like to reproduce the study, they would know exactly how to do so. Given this goal, details are essential.

When describing the measures, it is important to include an assessment of internal consistency with the present sample, as well as clinical and nonclinical benchmarks; however, one associate editor commented that authors “usually do a pretty good job of describing measures, but the procedures are where information is lacking.” This statement was echoed by several others. The exact information needed in the procedures section differs somewhat depending on the design of the study and likely includes details about the assignment of participants to conditions, the treatment/experimental conditions themselves (e.g., manualized or not, treatment length, format, prohibited content), an assessment of fidelity, and the setting in which the study was conducted. It is important to remember the procedures not only cover how the study was conducted, but they also cover the methods for data collection. Given this, information needs to be provided about how observational coding or coding of qualitative data were performed. Whenever these types of coding take place, statistics regarding interrater reliability should be provided. In addition, information about the timing of data collection is critical. Along these lines, authors should provide a rationale for the procedural decisions that were made. For example, if the researchers chose to analyze data from the third and eighth treatment sessions, the authors should provide a rationale for why those sessions were chosen over the fifth and tenth.

In most psychotherapy research studies, the participants include both clients and therapists, and an adequate description should be provided for each. The client description should include demographics as well as information about diagnosis, symptoms, level of distress at the start of the study, and previous experience with treatment. Inclusionary and exclusionary criteria, if present, should be carefully spelled out. The therapist description should also include demographics, experience level, type of degree, theoretical orientation, a description of the training and ongoing supervision received while providing the intervention. Given the nested nature of the data in most psychotherapy research, appropriate statistical techniques should be chosen.

One associate editor also suggested that the methods section should include a subsection on the data analyses. This section should clearly tie the planned statistical techniques to the hypotheses presented at the end of the introduction. As with the procedures section, a rationale for the specific data analytic procedures should be given and enough detail should be provided so it is reproducible. In this section, the authors should also state which analyses were planned versus exploratory.

It seems in the methods section, reproducibility is of paramount importance. This was perhaps best reflected in the comments of one editor who stated, “When you think you are done, read this section again but with the question looming in your mind….could I replicate this study on the basis of the information provided?” It may even be helpful to have colleagues outside your particular area of study read through the methods section and see if they believe they could replicate the design. Related, the editors and associate editors emphasized if the manuscript provides a secondary analysis from a previously published study, that fact should be explicitly stated, though the methods of the initial study should still be described in enough details so readers will not have to look up the initial study to see what was done. Many readers will not have access to the initial publication and most won’t take the extra time to find it.

Results

In keeping with the theme of clarity, the editors and associate editors surveyed recommended that the results be presented in a straightforward and clear manner. Several of them suggested the results should be organized by hypotheses. First, remind readers what the hypothesis was, present the descriptive data (e.g., means, standard deviations, correlations between measures), then state what statistical tests were used to test the hypothesis, and present the results. Then move on to the second hypothesis, the third hypothesis, and so on.

When writing the results section, authors should take an impartial stance. One editor recommended authors “write the results section as if you were writing up someone else’s findings. Would you be inclined to interpret someone else’s findings as ‘approaching significance’ as you would your own?” Similarly, another editor suggested authors “present only findings which are there, and not what you wish for.”

Several of those interviewed stated specific statistics that should be included in the results section, such as confidence intervals (particularly necessary when plotting means), raw means and standard deviations, and common effect sizes (d, g, or r). Those interviewed also highly recommended using tables and figures to display detailed information, then reviewing the aggregate data and the results of the statistical tests in the text. While many researchers do not use enough tables and figures, some include too many. Tables and figures may not be needed when the results can be presented simply in written form.

Discussion

Some of the editors and associate editors shared that reviewers often make up their minds regarding the quality of the manuscript before making it to the discussion section. If the introduction does not build a case for the study or the method and results sections are written poorly, even the best discussion section cannot change the reviewer’s perception. However, the opposite may be true—a manuscript with an adequate introduction, methods, and results section may be rejected if the discussion section is not well thought through.

The primary concern emphasized was that the discussion stick to the results of the study. That is, the discussion should not make statements or conclusions that over-reach the results found. This may be seen when authors make causal claims that do not fit the design or discuss clinical implications that were not actually tested in the study. At the same time, authors should not simply restate the study’s findings. One editor commented “remember, you wrote an introduction with relevant context. Wrap the manuscript in that context by revisiting how the study enriches that literature.” While not overreaching the findings, this may be the place to, as one associate editor said, “Take some risks with the explanations/implications.” This same associate editor mentioned getting bored when reading the obvious in the discussion section, such as, “alliance is related to outcome, therefore pay attention to the alliance.”

The editors and associate editors also emphasized the need for a thorougAPA Publications and Communications Board Working Group on Journal Article Reporting Standards (2008). Reporting standards for research in psychology: Why do we need them? What might they be? American Psychologist, 63, 842-845. doi:10.1037/0003-066X.63.9.839h section covering the limitations, future research, and clinical implications. Authors should spend a good amount of time thinking about the limitations of their work. One associate editor stated, “No study could cover the whole territory,” emphasizing a study can still have merit even if it is not perfect. However, if major limitations are not recognized by the authors, reviewers may question the objectivity and expertise with which the study was conducted. Also, the limitations discussed should be meaningful. For example, most manuscripts discuss the limits in generalizability based on the demographics of the participants; but, is there some reason to believe different results would be observed in different samples? The future research section should also not simply state the obvious. Of course, every study could be replicated in a different setting or with a different sample, but what are the important next steps? One associate editor said, “A discussion should leave the person feeling inspired to drop everything and follow up your research.” In concluding the manuscript, authors should help the readers see the contribution of the study to the field. For psychotherapy research, this contribution should have direct clinical applications to the practice of psychotherapy. These applications should be stated explicitly. However, authors should still remember not to overstate. In the end, as one editor put it, “We ought to be more invested in discerning what is true than in proving our original ideas to be right.”

+2

0

Joshua K. Swift, Ph.D.

Joshua K. Swift, Ph.D. is an Assistant Professor in the Department of Psychology at Idaho State University. He conducts psychotherapy process and outcome research, with a special focus on client factors (preferences and expectations) and premature termination. He is also a licensed psychologist.