While peer-reviewing a paper this week, I found myself in a quandry.
The study in question is an open-label, observational trial of a psychoactive medication. There's no control group and no randomization. Personally, I think such studies are of little scientific value, that they more often mislead than enlighten, and that they should only be published when exceptionally informative. But these studies do get published - in fact, rather similar articles have been published recently in the (mid-ranked) journal I'm reviewing for. Apart from having no comparison group, there is nothing wrong with this study - the sample size is decent, the paper is well-written, etc. In other words, by the standards of its genre, this manuscript is perfectly good. But I don't like the genre. So what should I do? Recommend acceptance, against my convictions? Wouldn't that be betraying my scientific integrity? Or do I recommend rejection, and be unfair to the authors? To reject would be unfair because many other reviewers, for this journal, evidently do accept manuscripts like this. If I rejected it, the authors might curse the capricious nature of peer review, and complain about being unlucky enough to get a reviewer with an axe to grind. They'd probably be right. I'm inclined to compromise, by accepting the paper, but only after demanding changes, to make the manuscript more open about the limitations of the design. But I have previously written that
The reviewers are often the ones who write the Limitations Section, but I think this is often a way of passing the buck... rather than bite the bullet and reject a paper, you may feel – well, being realistic, everyone’s in the same boat, who hasn’t cut corners, and we’ve all got to earn a living… etc… – so you recommend the paper is accepted, but in order to avoid feeling like a walkover, you make the authors include your concerns as Limitations.
Which is a stinging indictment of exactly what I'm proposing to do. Ooops. I wonder how other peer-reviewers resolve this dilemma?
