An odd, but not all that memorable photo, according to MIT researchers' deep-learning algorithm that rates a photo's memorability. (Credit: AnastasiaNess/Shutterstock (photo), scored via LaMem Dataset) A picture may be worth a thousand words, but that doesn’t mean it's memorable. So what exactly makes one image more memorable than another? Researchers at MIT drew on the collective knowledge of the masses to identify memorable photos, and they’ve incorporated their findings into an algorithm that predicts a photo’s memorability as accurately as humans. Based on their findings, a “selfie” snapped by the beach is far more memorable than a photo capturing the majesty of Yosemite’s Half Dome.
(Credit: LaMem Dataset)
Deep-Learning
To build their algorithm, researchers relied on the crowdsourcing muscle of Amazon’s Mechanical Turk. Human participants viewed a series of 35 to 150 images and were told to identify the photo that was repeated in a given set. Correctly identified duplicates were deemed more memorable. Researchers used results from human trials to assign memorability scores to 60,000 images, and then fed them into their algorithm. They borrowed their computing technique from a field of artificial intelligence called deep-learning, which allows computers to utilize artificial neural networks to analyze mountains of data and discover patterns all on their own. It’s how Facebook recognizes your friends’ smiles in photos and suggests you tag them. As a demonstration of their algorithm's prowess, researchers told it to predict how memorable a group of people would find a never-before-seen image. The algorithm predicted its memorability score within a few percentage points of humans. The computer, in other words, simulated the capabilities of human visual memory. The published their findings in a paper that's available on the LaMem Dataset website. You can also put researchers’ algorithm to the test online and upload photos from your collection to see if you're a memory-making photographer.
Remember Me
As the algorithm does its work, it builds a heat map over images to highlight the more memorable or forgettable elements of each photo. Memorable images tended to have specific focal points — a face or an object — and were often counterintuitive or just plain odd. A tentacle-like couch, a snowboarder in a sumo wrestling outfit and a man with an orange juice mustache ranked among the highest in memorability.
The most memorable images from researchers' collection. (Credit: LaMem Dataset) Landscapes, cityscapes and photos of your house were most forgettable. Interestingly, images of angry and fearful faces were far more memorable than pictures of awe or contentment, which gels with past research that indicates our brains evolved to remember negative stimuli for self-preservation. Remembering the colors of a deadly snake, for example, is far more important than remembering a scenic view during a hike.
The most forgettable images in researchers' dataset. (Credit: LaMem Dataset) The algorithm isn’t perfect, however. Some images seared into our collective consciousness aren’t all that memorable based on the algorithm’s correlations. Although the model is good at catching memorable visual elements, it can't grasp the historical context that makes some images iconic.
(Credit: Screenshot via LaMem Databset) Sure, there are some wrinkles, but researchers are already planning to spin their model off into an app that subtlety tweaks images to make them more memorable — a potential plus for advertisers. Their findings could also help enhance visual materials in the classroom to help students remember their lessons.
