Our Facial Recognition Prowess

Most of us have a powerful ability to recognize faces, and yet we hardly ever take note of it. We can commit a face to memory with a single glance, and even if we see that face only once its memory can stay fresh for years. The faces we remember may differ only in subtle tweaks of geometry: the ratio of distances between the eyes and the mouth, for example. A small fraction of people, however, cannot recognize faces—even the faces of their parents, spouses, and children. Prosopagnosia, as this condition is known, can affect people from birth or be triggered later in life by injuries to the brain. It strikes an estimated 2 percent of Americans and is often accompanied by other types of recognition impairments, including difficulty recognizing places and objects, such as cars. Despite the millions of people who suffer from prosopagnosia, it remains an obscure disorder, probably due to the skill with which face-blind people quietly compensate for their condition. In his recent book The Mind’s Eye, neurologist and writer Oliver Sacks makes the surprising disclosure that he has prosopagnosia. “I have had difficulty remembering faces for as long as I can remember,” he writes. Sacks, who turns 79 this year, has been publishing books for four decades. Despite his many years in the public eye, this is the first time most of his fans have learned of his condition. The Multidimensional 'Face Space' Doctors can’t do much for people with prosopagnosia, in part because neuroscientists still have only a rough idea of how normal face recognition works. Recently, cognitive neuroscientist Marlene Behrmann at Carnegie Mellon University and her colleagues gathered some important clues to this puzzle by comparing the brains of individuals who are face-blind to those who are face-sighted. Their results hint at how we recognize faces: not in a flash of insight, as it may seem, but by building up recognition on a neurological assembly line. Behrmann has been testing a model for face recognition that was first proposed 25 years ago by Tim Valentine and Vicki Bruce, two psychologists at the University of Nottingham in England. Valentine and Bruce argued that our brains do not store a photographic image of every face we see. Instead, they carry out a mathematical transformation of each face, encoding it as a point in a multidimensional “face space.” On a map of face space, imagine replacing the east-west axis with a small-mouth-to-wide-mouth axis. But instead of three different dimensions, like the space we’re familiar with, face space may have many dimensions, each representing some important feature of the human face. Just as the ancient cosmos was centered on Earth, Valentine and Bruce argued that the facial universe is centered on the perfectly average face. The further a face is from this average center, the more extreme it becomes. Over the past quarter century, the face space model has gained the support of a number of neuroscientists. For one thing, it offers an elegant explanation for how we can store so many images of faces in our heads. By reducing a face to a point—creating a compact code for representing an infinite number of faces—our brains need to store only the distance and direction of that point from the center of face space. Face space also sheds light on the fact that we are more likely to correctly identify distinctive faces than typical ones. In the center of face space, there are lots of fairly average faces. Distinctive faces dwell far away from the crowd, in much lonelier neighborhoods. Face space also explains why the favorite trick of editorial cartoonists works so well. By exaggerating features on a politician’s face—Bush’s eyebrows, Obama’s ears—cartoonists push it farther away from the center of face space, to places where it has less competition from other faces we have stored in our memory. As a result, we recognize people from hand-drawn caricatures as quickly as from photographs, and sometimes even more quickly. In addition to explaining the experiences we are familiar with, face space also explains a particularly bizarre illusion called the facial aftereffect. It’s similar to what happens when you stare for a long time at a picture of, say, an American flag. If you then look at a blank wall, you’ll see a ghostly afterimage of the flag in reversed colors for a few seconds. Much the same thing happens if you stare at a face for a long time. When you look at another face, it will look a little like the opposite version of the face you just stared at. Behrmann and her colleagues documented the facial aftereffect with an experiment using two faces they nicknamed Dan and Jim. For each face, they created an “anti-face.” Dan and Anti-Dan differed from an average face in opposite directions, reflecting opposite ends on the face space axis. Dan had an unusually long face, for instance, so Anti-Dan’s was unusually short. Once the scientists had crafted Anti-Dan and Anti-Jim, they began showing them on a computer monitor to volunteers without prosopagnosia. They would show the volunteers one of the anti-faces for five seconds. Next, they would flash a second face on the screen for just a fifth of a second. These fleeting faces were either “Dan-like” or “Jim-like.” That is, they were computer-generated faces that were somewhere between the average face and either Dan or Jim. The volunteers had to guess whether the faces looked more like one or the other. Behrmann and her colleagues found that staring at anti-faces skewed how volunteers perceived the faces that followed. If the volunteers looked at Anti-Dan first, they did a better job of guessing the identity of Dan-like faces. On the other hand, Anti-Dan made them do a worse job of recognizing Jim-like faces. The reverse happened if they saw Anti-Jim first: They did better at recognizing Jim-like faces and worse at recognizing Dan-like ones. Neuroscientists aren’t sure exactly why this happens. Behrmann thinks it has to do with aftereffect, which triggers neurons sensitive to the opposite of what we just saw. Staring at Anti-Dan could therefore make us especially attuned to the faces along the continuum closer to Dan. Likewise we become worse at recognizing faces that do not fall along that line. If face space is crucial for recognizing faces, Behrmann and her colleagues speculated that it might play a role in face blindness: Perhaps prosopagnosia is the result of warped face space. The scientists carried out a battery of face space tests on normal individuals and on seven people with prosopagnosia. Six had been face-blind since childhood. (They could name famous faces correctly only about half the time.) The seventh, a man the researchers referred to as SM, lost his ability to recognize faces after a head injury suffered when he was 18. In one experiment, the scientists taught SM to name Dan and Jim from their faces. He could learn to do the task, but the memory didn’t last long. If Behrmann and her colleagues then showed SM Dan-like and Jim-like examples and asked which one they looked like, his answers were never better than chance. The scientists then tried to trigger a facial aftereffect by first having him look at Anti-Dan and Anti-Jim. The experience had no effect on SM’s answers. Looking at Anti-Dan, for example, he was no more likely to guess correctly that Dan-like images looked like Dan. Revelations All in all, the experiments revealed that SM’s experience with face space was drastically different from that of normal people. But when Behrmann and her team tested the six people who had been face-blind from birth, they got an unexpected result: The members of that group did about as well as normal people at guessing whether faces were more like Dan’s or Jim’s. What’s more, they fell prey to the facial aftereffect. Anti-faces changed their performance in the same way they changed the performance of normal people. Additional tests yielded the same kinds of results. SM failed to show any sign of perceiving face space, while the six people born face-blind showed normal face space effects. People with the normal ability to recognize others will say that caricatures are more distinct than “anti-caricatures,” which are closer to an average face. So will people who were born face-blind. These people also found the caricatures to be more realistic than the anti-caricatures—just like people with normal brains. These results suggest that it is possible to be face-blind and yet still retain face space. They also provide clues to where in the brain this capacity for recognition unfolds. A number of scientists study facial recognition by putting people in brain scanners and looking for regions that are especially active when the subjects look at faces. One region that consistently lights up in these studies is a small patch of neurons known as the facial fusiform area—the part of SM’s brain that was damaged in his accident. People who are born face-blind, on the other hand, typically have a normal facial fusiform area. It even lights up in brain scans when they look at faces. On the basis of these results, Behrmann argues that this area is where we put faces into face space. But her experiments show that it alone is not enough to let us recognize faces. In another study on people born face-blind, Behrmann discovered a potential clue to the rest of the story. She and her colleagues found that people born face-blind have a smaller-than-normal bundle of nerve fibers linking the facial fusiform area to other regions toward the front of the brain. It is possible that once we encode faces in face space, our brains have to forward that information to other regions of the brain for more processing. If the connections along this recognition network are too weak, then people will be born face-blind. It will take more research to decipher how this remarkable ability develops, but there are practical reasons to keep up the search. By unlocking the secrets of how brains encode faces, scientists may be able to make computers better at recognizing us. And once scientists understand how most of us recognize faces, they may find a way to help the millions of people who can’t. [This article originally appeared in print as "About Face."]

Our Facial Recognition Prowess

Don't I know you? Humans have an extraordinary talent for recognizing and remembering subtle details that distinguish one mug from another. The trick may be in the way our brains encode "face space."

Newsletter

Carl Zimmer

Published in:

Mind + Machine

Turns Out Some Smells May Be Processed in the Same Brain Area as Taste

When It Comes to Thinking, Our Brains Are Surprisingly Slow

Sensory Deprivation Alters Our Experience, Both of Body and Time

What Is Emotional Contagion: Can We Actually Get Secondhand Stress?

What Is Tinnitus: It's Causes, Effects and Brain Connections

What's Actually Happening When We Have An Out-of-Body Experience?

How to Grow a Phantom Finger

Why Do Smells Trigger Such Powerful Memories?

How Language Shapes Our Understanding of Reality

Why The Paranormal And Supernatural Continue To Fascinate Us

Your Navigation Skills Depend on Where You Grew Up

You May Be Hallucinating Right Now

Are People More Prone to Supernatural Thinking if They Believe in the Afterlife?

How Super Are Super Senses?

From Clowns to Buttons, These Weird Phobias Afflict Many People

Stay Curious

JoinOur List

SubscribeTo The Magazine