X

X

# Website access code

### Enter your access code into the form field below.

If you are a Zinio, Nook, Kindle, Apple, or Google Play subscriber, you can enter your website access code to gain subscriber access. Your website access code is located in the upper right corner of the Table of Contents page of your digital edition.

Mind

# The F Problem With The P-Value Sciences

October 17, 2013 2:00 AM

There is a problem in science today. I've written a lot about how to cure it, but in this post I want to outline the nature of the disease as I see it. The problem goes by many names:

So I'm going to call it the f problem for short. I like to visualize f as a forking path. Given any particular set of raw data, a researcher faces a series of choices about how to turn it into a 'result'. There are choices over which statistical tests to run, on which variables, after excluding which outliers, and applying which preprocessing... and so on:

The f problem is that researchers can try multiple approaches in private, and select for publication the most desirable ones. Most often, it's statistically significant effects, that match with prior hypotheses, that are desired. Even if there are no real effects of interest in the data, some comparisons will be 'positive' just by chance. Researchers today face pressure to publish 'good results' and are rewarded for doing so - this is what turns f from a theoretical concern into a real one. Yet f is not a problem for all of science. Broadly, f only affects research in which the results take the form of p-values. Thus fields like mathematics, where results are proofs, are immune. But even some p-value sciences manage to escape f. This is because in these fields, the nature of the enterprise means that 'everyone knows' what experiments are being carried out, and how the data ought to be analyzed - in advance. In particle physics, for example, it was public knowledge that CERN was looking for the Higgs Boson, and it was also known exactly how they planned to look for it in the data. Openness meant that was no room for f, because f is all about the scope for hidden flexibility.

Recommendations From Our Store
Stay Curious
Join
Our List

Subscribe
To The Magazine

Save up to 70% off the cover price when you subscribe to Discover magazine.

## 2 Free ArticlesLeft

Want it all? Get unlimited access when you subscribe.

Subscribe

## Want unlimited access?

Subscribe today and save 70%