The scientists who unraveled the neural puzzle of face recognition

We are a fundamentally visual and social species, and being able to almost instantly recognise particular faces, as well as facial cues, are crucial tools for interacting with our conspecifics and navigating our complex social world.

But how does the brain achieve the feat of face recognition? How do we humans pick out a unique face in a sea of unfamiliar faces? And how do we recognize a virtually unlimited number of familiar faces? The laureates of the António Champalimaud Vision Award 2024 – Margaret Livingstone, Nancy Kanwisher, Doris Tsao, and Winrich Freiwald – have greatly contributed to the discovery of the brain structures and neural mechanisms that make this possible.

In 1997, American neuroscientist Kanwisher, at MIT, used fMRI (neuroimaging) to localize a region in the human brain, called the “fusiform area”, that responded to faces. It was located in nearly the same place in all the subjects she tested and was activated when people were shown pictures of human faces, but not any of a number of other objects. 

She and her team would later show, in 2020, that this brain region shows a response even in blind people when they run their fingers over models of human faces! We are so tuned to faces that vision might not even be required for the effect…

“The way the brain normally gets this information is through the eyes, but when that's blocked, it can use other information to fill the gap”, said co-laureate Winrich Freiwald in an interview a few days ago, when in Lisbon to receive the award. “That doesn’t mean blind people would use this as a means of recognition on a daily basis, but it is likely that there is a non-visual process for building a model of people’s faces from other inputs, in order to function in a social world.”

Back to the story: a few years later, Kanwisher’s discovery of the fusiform face area inspired the work of Livingstone, Tsao and Freiwald – the three other 2024 awardees. Tsao, Chinese-born, Maryland-raised neuroscientist, who at the time was doing her PhD at Harvard with Margaret Livingstone, teamed with German neuroscientist Freiwald, who was completing a postdoctoral fellowship under Kanwisher’s mentorship.
 
Together, they replicated Kanwisher’s fMRI analyses, and also found a “face area.” They then used the fMRI results as a guide to target recording electrodes in animal models, and found that over 95% of the neurons in this small area responded selectively to faces. In groundbreaking work, they described these cells and showed that individual cells in the face area responded to distinct constellations of face parts. This strongly suggested that the brain was building faces by assembling them piece by piece, as if they were puzzles.

More to the point, Freiwald and Tsao showed that face sensitivity resides in a set of six neighboring, interconnected brain areas, each playing different roles in face recognition. They called these areas “face patches”. There are six of them in each hemisphere, in the inferior temporal cortex of the brain. And since perturbing neural activity in these areas degrades face recognition, it was clear that they are indeed critical for face perception.

“Faces are very complex”, said Freiwald. “We have 23 facial muscles that change the configuration of the face. There are many other visual objects that are much simpler. Yet, as to the question of how something as complex as the brain processes something as complex as the face, there's a simple answer.”

“This doesn't mean that all brain functions will be localised and organised in this particular way”, he continued. “But it makes me hopeful that we can understand the human brain despite its complexity.”

Later, at Caltech, Tsao discovered how neurons in our brain encode faces, by untangling the connections of the “face patch” system. With her team, she effectively “cracked the neural code” – the Rosetta Stone for faces, as she calls it (http://magazine.ar.fchampalimaud.org/doris-tsao-the-woman-who-cracked-t…).

Freiwald, in 2021, already at Rockefeller University, found that a separate brain region – the temporal pole, at the tip of the brain's temporal lobe – accelerates our recognition of familiar faces. “We believe that there are cells in the temporal pole which are selective specifically for individuals that we know very well”, said Freiwald. “One way to put it is: [we don’t know if] there is a special place in our heart for our loved ones, but there is definitely a special place for our loved ones in this part of our brain”, he added good-humouredly.

“At Harvard, before Doris and Winrich started looking at faces”, said American neuroscientist Livingstone, who was also in Lisbon for the Award Ceremony, in a separate interview, “David Hubel [1981 Nobel Prize in Medicine for pioneering work on vision processing in the brain] and I had done a huge amount of work on subdivisions of the primary and secondary visual cortices into these modules. It was just one step further to find modules in the infratemporal cortex”.

Livingstone was instrumental in giving Tsao and Freiwald the space and freedom they needed to pursue their work on faces. “I sort of think of my way of running a lab as ‘benign neglect’” she said. “I just let everybody do what they want to do because I want to do what I want to do. And from time to time, we talk about what we're doing”.

Livingstone believes that the modular organisation of the face patches applies to other parts of the brain: “I think [this research] tells us something about how the brain wires itself up, as a self-organizing system, with universal simple rules”, she pointed out during her interview.

The laureates’ contributions have opened the way to further studying the general principles of the brain’s neural organization.

Working independently, “Livingstone and Kanwisher have used the face areas to gain new insight into the age-old nature/nurture issue”, writes the Champalimaud Foundation (CF) in a statement announcing the Prize, “by exploring whether face recognition is an innate feature of the primate brain or whether it is learned, presumably by exposure of an infant to its caregiver’s face.”

Tsao’s team is now exploring how the brain might perceive other objects in a similar way and ultimately create comprehensive visual scenes. As to Freiwald, he has started to unravel the complex brain circuitry underlying social interactions. 

Freiwald pointed out during his interview in Lisbon that he is particularly interested in diseases such as autism, a condition that makes it difficult to recognise facial expressions and other social cues. He believes that the understanding of facial processing by the brain may have implications for alleviating the symptoms of people with autism.

The laureates’ collective work “provides a window into brain mechanisms crucial to human development and social interactions”, the CF’s statement sums up.

The Award Ceremony took place at the Champalimaud Foundation’s outdoor amphitheater on the evening of September 11th, counting with the presence of illustrious guests and State representatives, including the President of Portugal, who presided the ceremony. On this occasion, the President bestowed an honor to the chairman of the award’s Jury, Alfred Sommer, a distinguished ophthalmologist and epidemiologist, dean emeritus of the Johns Hopkins Bloomberg School of Public Health, referring to it as “a testimony of gratitude in the scientific domain on behalf of Portugal”.
 

Text by Ana Gerschenfeld, Health & Science Writer of the Champalimaud Foundation.