On Tuesday I'll be speaking at a debate in University College London (UCL) on the topic of "Is Science Broken?" I'll be arguing that it is.
One of the other people on the panel is UCL neuroscientist Sam Schwarzkopf, who on his (alter ego) Devil's Neuroscientist blog (DNS) recently argued that science is not broken. He makes several points but here's the nub:
“Is Science broken?” - Attentive readers of my blog will probably guess that my answer to this question is a clear No. In fact, I would question whether the question even makes any sense. Science cannot be broken. Science is just a method - the best method there is - to understand the Cosmos. It is inherently self-correcting, no matter how much Crusaders
[he's referring to advocates of reform of science like me]
like to ramble against this notion. Science is self-correction. To say that science is broken is essentially stating that science cannot converge on the truth. If that were true, we should all just pack up and go home. What the organizers of the event mean in reality is that the human endeavor of scientific research is somehow broken. But is that even true?
DNS says that Science is the best method for understanding the cosmos. I agree. And I suspect that everyone at the debate will agree. No-one is saying that we should go back to trying to find truth by scrutinizing Aristotle. Science is the use of observation to guide thinking about the world to understand it. This grand, idealistic, with-a-big-SScience is not broken. However, much of the actual, concrete with-a-small-s science, i.e. the activity of scientists today, is not good Science. Some aspects of how modern science works go against the principles of Science. For instance, one of the key theories of how Science ought to work is Karl Popper's notion of falsifiability. Popper argued that for a theory to be considered scientific, it had to be falsifiable. That is, a theory should make predictions that could be tested and, potentially, proven wrong. An unfalsifiable theory is just not science. A falsifiable theory might be right or wrong - to find out, we should design experiments that would produce contrary evidence if it is wrong. If it survives our best attempts at disproof, a theory can be considered probably true. So that's the ideal. Popper's vision is of Science as a self-correcting enterprise. But the way science is performed today is often at odds with this view. I'll outline three problems:
Popper says that rather than trying to find evidence for hypotheses, we should actively seek to find evidence against them. Yet in science today, scientists are incentivized not to do this. Negative results are discriminated against, and find it hard to get published. The language we use betrays this bias: scientists often use "results" to mean positive results (consistent with theory), with negative results being referred to as "finding nothing"!
Popper says we must reject hypotheses if they're inconsistent with the evidence. But for this to work, evidence contrary to hypotheses has to be published. Yet we know that p-hacking, selective reporting, publication bias etc. means that scientists can (unconsciously) ignore contrary evidence, or allow it morph into positive conclusions.
Popper says that hypotheses should make predictions, and then the evidence should be collected to test those predictions. But in science today, hypotheses are often formed after the results of the experiment are known but before publication (post-hoc storytelling.) As a result, the hypotheses can't be tested. All scientists will know how this works: you get old hypotheses that never go away, because they adapt to explain away every single piece of contrary evidence. Of course, hypotheses should always be formed in the light of the known facts. But that's not enough: to be scientific, as Popper said, a hypothesis must also predict unknown facts. Post-hoc storytelling means that hypotheses never make real predictions.
I'm not saying that these problems mean that science today is totally compromised. There is lots of excellent science in all fields. But, in some fields, it has become difficult to do good science and, unless action is taken, things might get even worse. The reason these problems arise is not because the ideal of Science is flawed; it's because the processes by which we run science are imperfect (or outdated). In particular, I believe that the system by which scientists publish their results is broken; and this creates perverse incentives that lead us astray. As to how to fix this, that's a separate story.
