We have completed maintenance on DiscoverMagazine.com and action may be required on your account. Learn More

The Replication Crisis: Response to Lieberman

Neuroskeptic iconNeuroskeptic
By Neuroskeptic
Sep 1, 2014 12:57 AMNov 19, 2019 10:50 PM


Sign up for our email newsletter for the latest science news

In a long and interesting article over at Edge, social neuroscientist Matthew Lieberman discusses (amongst other things) the 'replication crisis' in his field. Much of what he says will be of interest to regular readers of this blog. Lieberman notes that there has been a lot of controversy over 'embodied cognition' and social priming research. For instance,

There are studies suggesting that washing your hands can affect your sense of being moral or immoral, and so on. These studies are very interesting. They're very counter-intuitive, which I think leads lots of people to wonder whether or not they're legitimate.

Lately there was a particular, well-publicized case of a non-replication of one of these counter-intuitive effects, and Lieberman discusses this, but I think the issue is a general one. Here's what Lieberman says (emphasis mine) about the effort to try and replicate these findings:

I do have some issues with the process of selecting who's going to do the replications — what their qualifications are for doing those things, have they done successful work in that area previously — because if they haven't shown that they can successfully get other priming effects, or other embodied cognition effects, how do I know that they can do this? I wouldn't go and try to do chemistry. I don't know anything about doing chemistry. There are issues like that.

This argument - which Lieberman is by no means alone in making - might be called the Harry Potter Theory of social psychology. On this model, some effects are real but are difficult to get to work in an experiment ('spells'). Some people ('wizards') have the knack of getting spells to work. Other researchers ('muggles') just can't do it. So if a muggle fails to cast a spell, that's not evidence against the spell working. What else would you expect? They're a muggle! Only if a wizard fails to replicate a spell, should we be worried about the reliability of that particular piece of magic. Accordingly, muggles should not even be trying to test whether any spells work. Wizards can safely ignore muggles.

Lieberman would probably object at this point that he's not saying that some researchers should be banned from the replication process. Rather, he might say, he is only emphasizing the fact that some scientists are more qualified than others for particular tasks. If so, fair enough, but all I'm saying is that there's something odd about the idea that ones qualifications should include a track record in finding positive results in the field in question. That seems to be putting the cart before the horse. I agree that replicators should have the necessary technical skills, but I question whether generating positive (as opposed to negative) results can be used as a proxy for being skilled. That would make sense if we assume that our basic psychological theory (e.g. of social priming) is valid, and therefore that at least some of our effects are real and replicable. If we grant that, then yes, we could assume that people who fail to find effects, must be doing it wrong. (If magic exists, then non-wizards are muggles.) But can we assume that? Isn't that, in fact, the issue under debate in many cases?

1 free article left
Want More? Get unlimited access for as low as $1.99/month

Already a subscriber?

Register or Log In

1 free articleSubscribe
Discover Magazine Logo
Want more?

Keep reading for as low as $1.99!


Already a subscriber?

Register or Log In

More From Discover
Recommendations From Our Store
Shop Now
Stay Curious
Our List

Sign up for our weekly science updates.

To The Magazine

Save up to 40% off the cover price when you subscribe to Discover magazine.

Copyright © 2024 Kalmbach Media Co.