We have completed maintenance on DiscoverMagazine.com and action may be required on your account. Learn More

Psychology’s Inner Demons

Results from the field’s biggestreplication study yet make it clearpsychology has room for improvement.

By Christie Aschwanden
Nov 30, 2015 6:00 AMNov 12, 2019 5:19 AM
Brian Nosek and his team replicated 100 high-profile psychology studies. The findings highlight the field’s need for more rigorous experiments. | Stephen Voss


Sign up for our email newsletter for the latest science news

It was a group project of staggering proportions. University of Virginia psychology professor Brian Nosek and his colleagues at the nonprofit Center for Open Science got help from over 350 scientists to repeat 100 high-profile psychology experiments published in 2008 — the largest replication study to date. In August, they announced the results in Science, forcing psychologists to face some hard truths about the reliability of their field’s studies.

The project is a response to the so-called “reproducibility crisis,” a growing concern that many published studies may not hold up to repeat testing. The problem affects other sciences, but several high-profile failures to reproduce important psychology findings have made it a particular concern.

Researchers not involved in the initial studies contacted the original authors to get feedback on their protocols; in most cases, the original researchers helped with study designs and strategies. Despite this thoroughness, while 97 of the original studies reported significant results, only 35 of the replications reported the same. And even then, the effect size (a measurement of how strong a finding is) was smaller — on average, less than half the original size. So even when Nosek’s team ended up with significant results, those results weren’t as strong as the initial reports claimed they were.

The replications are not definitive, Nosek says: “No study provides perfect evidence for or against any finding, and that also applies to the original [studies].” The replication simply adds information about the phenomenon under study, he says.

But it does mean that psychologists are starting to learn more about how to make their experiments more rigorous. “What this paper can help us do is accept that this is a problem so we can go about finding ways to fix it,” says Dorothy Bishop, a psychologist at the University of Oxford and a leading voice on psychology’s reproducibility issues. The problem won’t be solved overnight, but this study is a step in the right direction.

1 free article left
Want More? Get unlimited access for as low as $1.99/month

Already a subscriber?

Register or Log In

1 free articleSubscribe
Discover Magazine Logo
Want more?

Keep reading for as low as $1.99!


Already a subscriber?

Register or Log In

More From Discover
Recommendations From Our Store
Shop Now
Stay Curious
Our List

Sign up for our weekly science updates.

To The Magazine

Save up to 40% off the cover price when you subscribe to Discover magazine.

Copyright © 2024 Kalmbach Media Co.