What's the News: Scientific publishing is how researchers get the word out about what they've learned. It's how people outside the lab learn whether a drug was safe or not, or whether a treatment had any effect. But a team of scientists looking at a sample of clinical trials funded by the US government found that 30 months after the trials were completed, more than half had not yet been published. And that means that other studies trying to assess whether those treatments are safe and effective are working with incomplete information, while the relevant trials, already paid for by the public, are gathering dust on a lab bench. How the Heck:
The team, whose study appears in a special report in the British Medical Journal, looked at clinical trials listed as having finished in December 2008 on ClinicalTrials.gov.
Of the 635 they examined, 341 (54%) were not published in a peer-reviewed journal within 30 months of the end of the trials. 51 months after completion---that's over four years---a third of the trials remained unpublished.
That could have a serious effect on research and health. Another team whose work appears in the BMJ special issue took information the FDA made available from unpublished trials on drugs it approved about ten years ago. They then reran meta-analyses looking at how effective drugs were, now with the previously unpublished results included.
They found, chillingly, that including the unpublished data could change the conclusions reached by meta-analyses. The researchers write: "Overall, addition of unpublished FDA trial data caused 46% (19/41) of the summary estimates from the meta-analyses to show lower efficacy of the drug, 7% (3/41) to show identical efficacy, and 46% (19/41) to show greater efficacy." That means that when scientists try to study those FDA-approved drugs, they may not realize that they work less well than published papers indicate (or better, as the case may be).
What's the Context:
This is far from the first time it's been shown that trials go years and years without being published: previousstudies have found that 25% to 50% of clinical trials are unpublished years after completion. But the first study mentioned above is the only one since 1979 looking at studies funded by the US National Institutes of Health, which doles out taxpayer dollars to support important research projects related to human health.
Part of the agreement you make when you receive a grant from the NIH is that your work must be listed on the PubMedCentral public database within twelve months of having been published. But there aren't hard and fast requirements on when it has to get published.
There are a variety of reasons that trial results might take a long time to publish: maybe it's not the biggest priority in the lab (or as a grad student once jokingly said to me, "I know the answer now. Who cares if anyone else does?"), maybe the researchers are having trouble finding a (sufficiently prestigious) journal to publish it in, or maybe the team is reluctant to share less than stellar results. And as far as releasing the information before it's published is concerned, in biology such a thing is almost never done: A journal's peer reviewers might spot some flaw in the analysis that the researchers hadn't noticed, of course, but there are also numerous taboos against letting the cat out of the bag, related to problems of intellectual property and the climate of competition within the field.
The Future Holds:
It's understandable that it can take some time to get scientific results together for publication. But these numbers are frankly frightening. Given that clinical trials are explicitly about advancing people's health, it seems like there should be a premium placed on getting that information out there.
The arXiv, a database of yet-to-be-published physics and math papers maintained by Cornell, has become a useful resource for researchers, journalists, and the interested public. Several years ago, Nature set up Nature Preceedings, which acts as an arXiv for biology, and though it hasn't really taken off, probably for the reasons mentioned above, maybe that model is what's required for clinical trials. Perhaps a new requirement for accepting NIH funding should be publication in such a preprint database, if not a regular journal, before a certain number of years are up.
References: J. S. Ross, T. Tse, D. A. Zarin, H. Xu, L. Zhou, H. M. Krumholz. Publication of NIH funded trials registered in ClinicalTrials.gov: cross sectional analysis. BMJ, 2012; 344 (jan03 1): d7292 DOI: 10.1136/bmj.d7292 B. Hart, A. Lundh, L. Bero. Effect of reporting bias on meta-analyses of drug trials: reanalysis of meta-analyses. BMJ, 2012; 344 doi: 10.1136/bmj.d7202. Update: The headline on this story has been changed to make the central message clearer.
