Register for an account

X

Enter your name and email address below.

Your email address is used to log in and will not be shared or sold. Read our privacy policy.

X

Website access code

Enter your access code into the form field below.

If you are a Zinio, Nook, Kindle, Apple, or Google Play subscriber, you can enter your website access code to gain subscriber access. Your website access code is located in the upper right corner of the Table of Contents page of your digital edition.

Mind

The Replication Crisis: Response to Lieberman

NeuroskepticBy NeuroskepticSeptember 1, 2014 12:57 AM

Newsletter

Sign up for our email newsletter for the latest science news

In a long and interesting article over at Edge, social neuroscientist Matthew Lieberman discusses (amongst other things) the 'replication crisis' in his field. Much of what he says will be of interest to regular readers of this blog. Lieberman notes that there has been a lot of controversy over 'embodied cognition' and social priming research. For instance,

There are studies suggesting that washing your hands can affect your sense of being moral or immoral, and so on. These studies are very interesting. They're very counter-intuitive, which I think leads lots of people to wonder whether or not they're legitimate.

Lately there was a particular, well-publicized case of a non-replication of one of these counter-intuitive effects, and Lieberman discusses this, but I think the issue is a general one. Here's what Lieberman says (emphasis mine) about the effort to try and replicate these findings:

I do have some issues with the process of selecting who's going to do the replications — what their qualifications are for doing those things, have they done successful work in that area previously — because if they haven't shown that they can successfully get other priming effects, or other embodied cognition effects, how do I know that they can do this? I wouldn't go and try to do chemistry. I don't know anything about doing chemistry. There are issues like that.

This argument - which Lieberman is by no means alone in making - might be called the Harry Potter Theory of social psychology. On this model, some effects are real but are difficult to get to work in an experiment ('spells'). Some people ('wizards') have the knack of getting spells to work. Other researchers ('muggles') just can't do it. So if a muggle fails to cast a spell, that's not evidence against the spell working. What else would you expect? They're a muggle! Only if a wizard fails to replicate a spell, should we be worried about the reliability of that particular piece of magic. Accordingly, muggles should not even be trying to test whether any spells work. Wizards can safely ignore muggles.

potter_psychology.png

Lieberman would probably object at this point that he's not saying that some researchers should be banned from the replication process. Rather, he might say, he is only emphasizing the fact that some scientists are more qualified than others for particular tasks. If so, fair enough, but all I'm saying is that there's something odd about the idea that ones qualifications should include a track record in finding positive results in the field in question. That seems to be putting the cart before the horse. I agree that replicators should have the necessary technical skills, but I question whether generating positive (as opposed to negative) results can be used as a proxy for being skilled. That would make sense if we assume that our basic psychological theory (e.g. of social priming) is valid, and therefore that at least some of our effects are real and replicable. If we grant that, then yes, we could assume that people who fail to find effects, must be doing it wrong. (If magic exists, then non-wizards are muggles.) But can we assume that? Isn't that, in fact, the issue under debate in many cases?

    2 Free Articles Left

    Want it all? Get unlimited access when you subscribe.

    Subscribe

    Already a subscriber? Register or Log In

    Want unlimited access?

    Subscribe today and save 70%

    Subscribe

    Already a subscriber? Register or Log In