Scientific papers should have two Discussion sections - one written by the authors, and the other by an independent researcher. According to a new paper from Michael S. Avidan, John P. A. Ioannidis and George A. Mashour, this "second discussant" system could help ensure more balanced and objective inference in science.
The authors begin by noting that while the reproducibility crisis has focussed attention on the Methods and Results sections of papers, Discussion sections are not free of problems of their own:
Some of the ongoing deficiencies of Discussion sections in medical journals include lack of contextualisation of results using systematic review, selective citation of prior studies that agree with the results, overinterpretation or misinterpretation of findings, and incomplete acknowledgement, analysis, and assessment of study limitations.
To address this issue, Avidan et al. suggest the concept of an additional expert who would write their own discussion, to be published alongside the paper, without seeing the authors' discussion. The second discussant (I would prefer the term 'discussionist') would be chosen by the journal editors, and they would need to be familiar with the general field of research, but wouldn't necessarily require detailed topic-specific expertise. Avidan et al. then give an example of independent discussion writing in action. Avidan read the Introduction, Methods, and Results of this paper, about a recent clinical trial of the drug haloperidol. He then wrote a Discussion of the trial results, without having seen the author's version. It turned out that, in this case, both Discussions were pretty similar, but there were a couple of differences. In particular, Avidan's discussion noted that the whole trial was based on a seemingly shaky assumption, one which the authors never questioned. To me, independent discussion sections is a great idea... but I do have doubts that it would work in practice. Were every paper to need a second discussion section, scientists would be even more over-worked than they already are. Writing, revising and reviewing papers is already a huge amount of work. Avidan et al. suggest that researchers would be motivated to contribute discussions because they would get a publication out of it:
Much like an editorialist, an independent discussant would fulfill the requirements for authorship as stipulated by the International Committee of Medical Journal Editors, so there should be also an academic incentive for becoming a good discussant and writing balanced, carefully thought Discussions.
(It's not quite clear if they envisage the discussant being added as an author on the paper, or getting a separate publication for the discussion, but I assume the latter.) But even if the second discussant was motivated to help, this doesn't change the fact that there's only so much time in the day, and that the proposed system would increase the amount of work that goes into each paper. Perhaps the system could work if it were reserved for especially important papers, but these already tend to get independent discussions (of a kind) anyway, in the form of editorials and (ahem) blog posts...
