Register for an account

X

Enter your name and email address below.

Your email address is used to log in and will not be shared or sold. Read our privacy policy.

X

Website access code

Enter your access code into the form field below.

If you are a Zinio, Nook, Kindle, Apple, or Google Play subscriber, you can enter your website access code to gain subscriber access. Your website access code is located in the upper right corner of the Table of Contents page of your digital edition.

Mind

Psychology’s Inner Demons

Results from the field’s biggestreplication study yet make it clearpsychology has room for improvement.

By Christie AschwandenNovember 30, 2015 6:00 AM
inner-demons.jpg
Brian Nosek and his team replicated 100 high-profile psychology studies. The findings highlight the field’s need for more rigorous experiments. | Stephen Voss

Newsletter

Sign up for our email newsletter for the latest science news

It was a group project of staggering proportions. University of Virginia psychology professor Brian Nosek and his colleagues at the nonprofit Center for Open Science got help from over 350 scientists to repeat 100 high-profile psychology experiments published in 2008 — the largest replication study to date. In August, they announced the results in Science, forcing psychologists to face some hard truths about the reliability of their field’s studies.

The project is a response to the so-called “reproducibility crisis,” a growing concern that many published studies may not hold up to repeat testing. The problem affects other sciences, but several high-profile failures to reproduce important psychology findings have made it a particular concern.

Researchers not involved in the initial studies contacted the original authors to get feedback on their protocols; in most cases, the original researchers helped with study designs and strategies. Despite this thoroughness, while 97 of the original studies reported significant results, only 35 of the replications reported the same. And even then, the effect size (a measurement of how strong a finding is) was smaller — on average, less than half the original size. So even when Nosek’s team ended up with significant results, those results weren’t as strong as the initial reports claimed they were.

The replications are not definitive, Nosek says: “No study provides perfect evidence for or against any finding, and that also applies to the original [studies].” The replication simply adds information about the phenomenon under study, he says.

But it does mean that psychologists are starting to learn more about how to make their experiments more rigorous. “What this paper can help us do is accept that this is a problem so we can go about finding ways to fix it,” says Dorothy Bishop, a psychologist at the University of Oxford and a leading voice on psychology’s reproducibility issues. The problem won’t be solved overnight, but this study is a step in the right direction.

2 Free Articles Left

Want it all? Get unlimited access when you subscribe.

Subscribe

Already a subscriber? Register or Log In

Want unlimited access?

Subscribe today and save 70%

Subscribe

Already a subscriber? Register or Log In