Fixing Science's Chinese Wall

Neuroskeptic iconNeuroskepticBy NeuroskepticDec 21, 2013 4:59 PM


Sign up for our email newsletter for the latest science news

In science we very often want certain things to be true. Sometimes, this desire comes from noble reasons: we want a vaccine to work, because that would save many lives. Other times our motivations are less altruistic: we want our vaccine to work because then we will get lots of citations, a promotion, and a raise.

Scientists wanting things to be true is a fact, and a problem. Which is why science has evolved methods to prevent our desires from distorting the findings. For instance, we use double blind trials to ensure that the researcher's preferences and expectations can't influence the outcome measurements. We use statistical inference with an agreed standard (p = 0.05) to determine the significance of findings, rather than leaving it up to individuals to pick what they want to consider signal and what noise. These systems are designed to protect observations from the whims of the observers. As Feynman famously said, "Science is a way of trying not to fool yourself... and you are the easiest person to fool." It is rather like how in business, banks and other firms are required to have a 'Chinese Wall' - a cordon around certain information to ensure that it can't reach certain people. This is intended to prevent conflicts of interest influencing those branches of the company whose job is to provide impartial advice to the public or other departments. However, there is a gap in science's Wall between data and desire - the publication process. Whether the results of a particular experiment get published in a peer-reviewed journal depends upon at least three people: - the researcher(s) themselves, who decides which results to submit, in what form - the journal editor(s), who decides whether submissions are 'interesting' - the peer reviewer(s), who advises the editor on whether submissions are 'good' All of these people are scientists, with their own agenda and desires. They get to decide the fate of scientific results - knowing what those results are. So the desires of researchers get to influence what's published, and the result is publication bias. This influence would be considered unscientific if it were allowed to happen within the conduct of a given experiment - but once the results are in, we allow it. There is an alternative. Journals ought to peer review and accept articles before the work has been begun (preregistration with pre-peer review.) This ensures that science is published or not on the basis of the strength and originality of the methods, independent of what the results happen to be. This would do for the later stages of the scientific process what double-blinding and randomization have done for the earlier, experimental stages. The Chinese Wall would be complete.

1 free article left
Want More? Get unlimited access for as low as $1.99/month

Already a subscriber?

Register or Log In

1 free articleSubscribe
Discover Magazine Logo
Want more?

Keep reading for as low as $1.99!


Already a subscriber?

Register or Log In

More From Discover
Recommendations From Our Store
Shop Now
Stay Curious
Our List

Sign up for our weekly science updates.

To The Magazine

Save up to 70% off the cover price when you subscribe to Discover magazine.

Copyright © 2023 Kalmbach Media Co.