Imagine that an eccentric psychologist accosts you. In his hand is a piece of paper with 20 pictures of roses. One of the pictures shows a rose in the flower bed you just passed, he says, and he asks you to pick its picture out from his lineup. The challenge would seem absurd—but if you were to change the roses to faces, nearly everyone could meet it.
Most of us have a powerful ability to recognize faces, and yet we hardly ever take note of it. We can commit a face to memory with a single viewing, and even if we see that face only once its memory can stay fresh for years. The faces we remember so easily may differ only in subtle tweaks of geometry: the ratio of distances between different landmarks such as the eyes and the mouth, for example.
A small fraction of people, however, cannot recognize faces—even the faces of their parents, spouses, and children. Prosopagnosia, as this condition is known, can affect people from birth or be triggered later in life by injuries to the brain. It strikes an estimated 2 percent of Americans and is often accompanied by other types of recognition impairments, including difficulty recognizing places and objects, such as cars.
Despite the millions of people who suffer from prosopagnosia, it remains an obscure disorder, probably due to the skill with which face-blind people quietly compensate for their condition. In his new book, The Mind’s Eye (Amazon; book review in The New York Times), neurologist and writer Oliver Sacks makes the surprising disclosure that he has prosopagnosia. “I have had difficulty remembering faces for as long as I can remember,” he writes. Sacks, who turns 78 this year, has been publishing books for four decades. Despite his many years in the public eye, this is the first time most of his fans have learned of his condition.
Doctors can’t do much for people with prosopagnosia, in part because neuroscientists still have only a rough idea of how normal face recognition works. Recently, cognitive neuroscientist Marlene Behrmann at Carnegie Mellon University and her colleagues gathered some important clues to this puzzle by comparing the brains of individuals who are face-blind to those who are face-sighted. Their results (pdf) hint at how we recognize faces: not in a flash of insight, as it may seem, but by building up recognition on a neurological assembly line.
Behrmann has been testing a model for face recognition that was first proposed 25 years ago by Tim Valentine and Vicki Bruce, two psychologists at the University of Nottingham in England. Valentine and Bruce argued that our brains do not store a photographic image of every face we see. Instead, they carry out a mathematical transformation of each face, encoding it as a point in a multidimensional “face space.”
On a map of face space, you might imagine the north-south axis being replace with a small-mouth-to-wide-mouth axis. But instead of three different dimensions, like the space we’re familiar with, face space may have many dimensions, each representing some important feature of the human face. Just as the ancient cosmos was centered on Earth, Valentine and Bruce argued that the facial universe is centered on the perfectly average face. The farther a face is from this average center, the more extreme it becomes.
Over the past quarter century, the face space model has gained the support of a number of neuroscientists. For one thing, it offers an elegant explanation for how we can store so many images of faces in our heads. By reducing a face to a point—creating a compact code for representing an infinite number of faces—our brains need to store only the distance and direction of that point from the center of face space. Face space also sheds light on the fact that we are more likely to correctly identify distinctive faces than typical ones. In the center of face space, there are lots of fairly average faces. Distinctive faces dwell far away from the crowd, in much lonelier neighborhoods.
Face space also explains why the favorite trick of editorial cartoonists works so well. By exaggerating features on a politician’s face—Bush’s eyebrows, Obama’s ears—cartoonists push it farther away from the center of face space, to places where it has less competition from other faces we have stored in our memory. As a result, we recognize people from hand-drawn caricatures as quickly as from photographs—and sometimes even more quickly.
In addition to explaining the experiences we are familiar with, face space also explains a particularly bizarre illusion called the facial aftereffect. It’s similar to what happens when you stare for a long time at a picture of, say, an American flag. If you then look at a blank wall, you’ll see a ghostly afterimage of the flag in reversed colors for a few seconds. Much the same thing happens if you stare at a face for a long time. When you look at another face, it will look a little like the opposite version of the face you just stared at.
Next page: Now, where's that face coming from?
Behrmann and her colleagues documented the facial aftereffect with an experiment involving two faces they nicknamed Dan and Jim (pdf). For each face, they created an “anti-face.” Dan and Anti-Dan differed from an average face in opposite directions, reflecting opposite ends on the face space axis. Dan had an unusually long face, for instance, so Anti-Dan had an unusually short one. Dan and Anti-Dan were at two ends of one of the lines that pass straight through the center of face space.
Once the scientists had crafted Anti-Dan and Anti-Jim, they began showing them on a computer monitor to volunteers without prosopagnosia. They would show the volunteers one of the anti-faces for five seconds. Next, they would flash a second face on the screen for just a fifth of a second. These fleeting faces were either “Dan-like” or “Jim-like.” That is, they were computer-generated faces that were somewhere between the average face and either Dan or Jim. The volunteers had to guess whether the faces looked more like one or the other.
Our brains do not store a photographic image of every face we see. Instead, they carry out a mathematical transformation of each face, encoding it in “face space.”
Behrmann and her colleagues found that staring at anti-faces skewed how the volunteers perceived the faces that followed. If the volunteers looked at Anti-Dan first, they did a better job of guessing the identity of Dan-like faces. On the other hand, Anti-Dan made them do a worse job of recognizing Jim-like faces. The reverse happened if they saw Anti-Jim first: They did better at recognizing Jim-like faces and worse at recognizing Dan-like ones.
Neuroscientists aren’t sure exactly how staring at faces triggers this aftereffect, but they generally agree it has something to do with a change in face space. Staring at Anti-Dan makes us more sensitive to the faces along the line from Anti-Dan to Dan. Likewise we become worse at recognizing faces that do not fall along that line.
If face space is indeed so crucial for recognizing faces, Behrmann and her colleagues speculated that it might play a role in face blindness: Perhaps prosopagnosia is the result of warped face space. The scientists carried out a battery of face space tests on normal individuals and on seven people with prosopagnosia. Six had been face-blind since childhood. (They could name famous faces correctly only about half the time.) The seventh, a man the researchers referred to as SM, had lost his ability to recognize faces after a head injury suffered when he was 18.
In one experiment, the scientists taught SM to name Dan and Jim from their faces. He could learn to do the task, but the memory didn’t last long. If Behrmann and her colleagues then showed SM Dan-like and Jim-like faces and asked which one they looked like, his answers were never better than chance. The scientists then tried to trigger a facial aftereffect by first having him look at Anti-Dan and Anti-Jim. The experience had no effect on SM’s answers. Looking at Anti-Dan, for example, he was no more likely to guess correctly that Dan-like faces looked like Dan.
All in all, the experiments revealed that SM’s experience with face space was drastically different from that of normal people. But when Behrmann and her team tested the six people who had been face-blind from birth, they got a completely different—and unexpected—result: The members of that group actually did about as well as normal people at guessing whether faces were more like Dan’s or Jim’s. What’s more, they fell prey to the facial aftereffect. Anti-faces changed their performance in the same way they changed the performance of normal people. (To test your own face recognition, visit the Prosopagnosia Research Center.)
Additional tests yielded the same kinds of results. SM failed to show any sign of perceiving face space, while the six people born face-blind showed normal face space effects. People with normal face recognition will say that caricatures are more distinct than “anti-caricatures,” which are closer to an average face. So will people who were born face-blind. These people also found the caricatures to be more realistic than the anti-caricatures—just like people with normal brains.
These results suggest that it is possible to be face-blind and yet still retain face space. They also provide clues to where in the brain face recognition unfolds. A number of scientists study facial recognition by putting people in brain scanners and looking for regions that are especially active when the subjects look at faces. One region that consistently lights up in these studies is a small patch of neurons on the underside of the brain known as the facial fusiform area. That is precisely the part of SM’s brain that was damaged in his accident.
People who are born face-blind, on the other hand, typically have a normal facial fusiform area. It even lights up in brain scans when they look at faces. On the basis of these results, Behrmann argues that the facial fusiform area is where we put faces into face space. But her anti-face experiments show that face space is not enough to let us recognize faces. Something is also going on elsewhere in the brain.
In another study on people born face-blind, Behrmann discovered a potential clue to the rest of the story. She and her colleagues found that people born face-blind have a smaller-than-normal bundle of nerve fibers linking the facial fusiform area to other regions toward the front of the brain. It is possible that once we encode faces in face space, our brains have to forward that information to other regions of the brain for more processing. If the connections along this face-recognizing network are too weak (the result, perhaps, of a genetic disorder), then people will be born face-blind.
It will take more research to fully decipher how we recognize faces, but there are some practical reasons, beyond sheer fascination, to keep up the search. By unlocking the secrets of how our brains encode faces, scientists may be able to make computers better able to recognize us. And once scientists understand how most of us recognize faces, they may find a way to help the millions of people who can’t.
