Part 1 - 3 Ways to Ensure Your Research Data Add Up Right: Tips from JWHPT Statistician

In 2016, members of the Section on Women’s Health rated the Journal of Women’s Health Physical Therapy as their most used and valued benefit. Their allegiance to the peer-reviewed publication prompted the SOWH Board of Directors to invest in a major upgrade, including development of a revamped strategic plan, now underway.

Key to the process has been engaging fresh volunteers and contracting with top physical therapy research professionals. Newest to the editorial team, led by Editor-in-Chief Cynthia Chiarello, PT, PhD, of Columbia University, is Mark Bishop, PT, PhD. An associate professor at the University of Florida Department of Physical Therapy, Bishop had not worked with a publisher before he started as statistician for JWHPT. He had, however, performed experimental design and statistical analyses for both funded and unfunded projects.

In this conversation with SOWH, Bishop advises researchers how best to ensure their evidence-based manuscripts pass the rigor of at least one aspect of peer review—his close inspection of their data.

SOWH: Your role as the new JWHPT statistician should further strengthen the data integrity presented in the journal’s articles. What will you be looking for when a manuscript arrives for your review?

Bishop: “Once a paper is submitted for review, my primary focus the first read-through is to ensure that (1) the primary question or purpose matches the design (including the outcomes chosen), (2) the analysis presented matches the design of the study and form of the data, and (3) the conclusions presented match the results of the analysis.”

What are the most common errors you spot?

“There are a few common slips that can happen. Probably the most important of these is overreaching on the conclusions, especially about causation; for example, the analysis tests association using correlations and the discussion talks of relationship of one predicting the other.

“A second example is attempting to draw broad conclusions from a case study or case series, and a third example is that the analysis performed doesn’t match the question being asked. Many papers include tests of differences in means (t-tests, F-tests, etc.) when the question might be much broader about general differences between or among groups rather than specifically about mean differences–or maybe the question is about specific ordering or if the frequency of an event occurring differs from the expected frequency. Related to the last example, sometimes the analysis doesn’t match the form of data as well as it could, particularly when the samples are small.”

What are the top three actions an author can take to ensure his or her statistical data hold up to careful scrutiny? 

“I think the important first step relates to the issues mentioned above. For example, check that the design and analysis match. For investigators working deductively, this should be done before beginning–that is, start with the question, then design the data collection and analysis to answer that question.

“Of course, this is not always the case, and sometimes questions arise after data are collected. In that case, this should be clearly stated.

“Second, investigators should make sure they have used the correct test for the type of data they have collected. Involving a biostatistician or someone with experience in experimental design to assist in developing the experiment can really help if there are questions about this.

“And third, check that the conclusions can be supported by the results.”

Watch for Part 2 of this interview in July, when Bishop identifies the two key statistical skills needed by PTs today.