Last Wednesday, Nobel laureate Daniel Kahneman sent an email to a group of a dozen or so psychologists telling them that the credibility of their field was in danger. The recipients all worked on social priming – the study of how subtle unconscious cues can influence our behaviour. It’s an area that has attracted controversy of late, due to failed replications of classic results, the outing offraudulentresearchers, and a more general concern among psychologists about the validity of their field’s results. Kahneman meant the email as helpful advice, but his wording couldn’t have been stronger: “Your field is now the poster child for doubts about the integrity of psychological research... I believe that you should collectively do something about this mess.” His solution: a “daisy chain” of replications, where laboratories collaborate to check the results of their neighbours, in an open, transparent, and pre-established way. Kahneman requested that the email be sent to anyone relevant and, presumably because I have written several pieces on this topic, a copy landed in my Inbox on Wednesday. I interviewed Kahneman about it, and my story about his challenge (my description, not his) is now up at Nature News. Here's some overmatter that didn’t make it into the piece because of length.
I quoted Norbert Schwarz in the piece, but he sent over a far lengthier set of comments, which are online in a Google doc.
John Bargh, whose name will be familiar to readers of this blog, sent me comments that were cut from the piece during editing. They echo Schwarz’s views in emphasising the “extensive” support for priming effects—“If people continue to read just critiques and calls to action, the perception of a replication problem will probably continue to grow. I would urge instead a careful reading of the literature as a whole.” But he is also broadly supportive of Kahneman’s suggestion: “I do not see any harm that could be done if social cognition researchers began to replicate each other's new findings, which is what Danny called for... I support Danny's suggestion or some similar mechanism that helps to accomplish this important goal of further establishing the reliability of new effects in social psychology.”
David Funder, the President-Elect of the SPSP (Society for Personality and Social Psychology) thinks that Kahneman’s email carries a lot of weight with it. “He must be the most prestigious living psychologist. He’s our one and only living Nobel laureate and has a huge amount of respect from just about everybody. He’s just trying to play referee, and telling people to dial down the anger and defensiveness. One of the things that has made me sad about how this has played out is that this should be a scientific discussion and not an argument. For scientists, it should be that the data is the data, and let the chips fall where they may.”
In his email, and in a phone interview with me, Kahneman said that priming effects are very subtle, and could be undermined by small changes to experimental protocols at the hands of unskilled experimenters. “Replication is intended as cloning but there are mutations and they can be lethal,” he said. This is why his “daisy chain” idea involves people within the priming field replicating each others’ studies. But social psychologist Brian Nosek makes this good point: “People say there is nuance here, but that’s a hypothesis. That’s not true prima facie. I’d love to see collaborations across laboratories and to see if [expertise] really matters.”We’d then learn not just if results are true, but why they turn up in some labs and not others.
Kahneman mentions recently outed fraudsters in his email – these would include Diederik Stapel from last year, and Dirk Smeesters and Lawrence Sanna this year, all of whom used priming techniques in their experiments. I asked him about this, and the effect on the priming field. “It is a feature of priming studies that they’re surprising; some of them have been quite exuberant in the kind of things they have shown,” he says. “It doesn’t follow that people who are doing priming are committing fraud; it’s more likely that people who are committing fraud are drawn to that field. It’s important not to invert the logic of that.”