Register for an account


Enter your name and email address below.

Your email address is used to log in and will not be shared or sold. Read our privacy policy.


Website access code

Enter your access code into the form field below.

If you are a Zinio, Nook, Kindle, Apple, or Google Play subscriber, you can enter your website access code to gain subscriber access. Your website access code is located in the upper right corner of the Table of Contents page of your digital edition.


The Reliability of fMRI Revisited

Neuroskeptic iconNeuroskepticBy NeuroskepticJanuary 10, 2014 10:57 PM


Sign up for our email newsletter for the latest science news

A new paper brings worrying news for neuroscientists using fMRI to study memory:

Across-subject reliabilities were only poor to fair... for novelty encoding paradigms, the interpretation of fMRI results on a single subject level is hampered by its low reliability. More studies are needed to optimize the retest reliability of fMRI activation for memory tasks.

The researchers, David Brandt and colleagues from Marburg, Germany, scanned 15 healthy volunteers twice each. In order to measure the neural activation associated with memory formation, the subjects were shown word and picture stimuli that they'd never seen before.


In a group-level analysis, when all the volunteer's data was averaged together, several brain areas were activated by the memory task (and these were pretty much the areas one would expect.) However, the degree of activitation in different brain areas was not very stable within individuals. Comparing the two activity patterns from the scans a month apart, the ICC  for each voxel of the brain was poor (the median ICC was at best 0.35, which is low, and that was in the most favorable of the several conditions of the task.) In other words if an individual shows a huge activation of the hippocampus in one session, it doesn't mean that will happen whenever they get scanned. Which implies that we shouldn't 'read too much into' a given individual's degree of activation - as it might be quite different next time around. You might remember that four years ago, I blogged about a review finding that the test-retest reliability of fMRI was modest: Can We Rely On fMRI? That paper reported that the mean test-retest ICC for fMRI is 0.50 which means that memory activations are well below average. An interesting discussion followed my tweeting about the new paper. Simon W Davis pointed out that, four years ago, Miller et al (2009) reported comparable results. The two studies are very similar in some ways, both involving fMRI of volunteers scanned twice, several weeks apart, with memory tasks, although the paradigms were different (novelty encoding in Brandt, vs episodic retrieval, semantic retrieval, and working memory tasks in Miller.) However, as Kirstie Whitaker observed, there was a difference in interpretation:

Miller et al span it as "brains are different to each other", Brandt et al as "neuroimaging is noisy".

To which Davis commented that

This is what 3 years of neurobashing gets you: same study, more conservative interpretation.


Brandt DJ, Sommer J, Krach S, Bedenbender J, Kircher T, Paulus FM, & Jansen A (2013). Test-Retest Reliability of fMRI Brain Activity during Memory Encoding. Frontiers in Psychiatry, 4 PMID: 24367338

    3 Free Articles Left

    Want it all? Get unlimited access when you subscribe.


    Already a subscriber? Register or Log In

    Want unlimited access?

    Subscribe today and save 70%


    Already a subscriber? Register or Log In