Beauty is in the eye of the beholder, but memorability can be universal -TGN

Imagine you a weekend afternoon with friends at an art museum: nodding with arms crossed, desperate for something enlightening to say. The vast majority of the paintings you walk past are immediately forgotten, but some remain with you. It turns out that the paintings you remember are probably the same as everyone else.

There’s a scientific term for that: image memorability. “It’s the idea that there are essentially some intrinsic patterns that make some content more memorable than others,” said Camilo Fosco, a doctoral student studying computer science at MIT and the CTO of MIT. Memorable AI, a startup that uses machine learning to test how engaging content will be for advertisers and creators. In other words, certain works of art have that je ne sais quoi – and now a team of scientists are using AI to figure out what it is.

In a study published earlier this month in the Proceedings of the National Academy of Sciences, Researchers Trent Davis and Wilma Bainbridge of the University of Chicago show that the memorability of artworks is not only consistent across all humans, but also predictable by AI. In an online experiment, they pulled about 4,000 paintings from the Art Institute of Chicago’s database, excluding anything the institute labeled as “boosted” or especially famous. More than 3,200 people viewed hundreds of images, so that each painting was seen by about 40 people. Next, the volunteers were shown the paintings they had seen mixed in with paintings they had not seen, and were asked if they remembered them. People were very consistent – everyone tended to remember (or forget) the same images.

Using a deep learning neural network called ResMemdesigned by data scientist Coen Needell as part of his master’s thesis in Bainbridge’s psychology lab, the research team was can predict how likely it was that each painting would be memorable. ResMem roughly mimics how the human visual system passes information from the retina to the cortex, first processing basic information such as edges, textures and patterns, then scaling up to more abstract information such as the meaning of objects. The memorability scores correlated very strongly with those of humans in the online experiment, even though the AI ​​knew nothing about the cultural context, popularity, or significance of each piece of art.

Counterintuitively, these findings suggest that our memory of art is less about subjective experiences of beauty and personal meaning, and more about the artwork itself — which could have major implications for artists, advertisers, educators, and anyone else who hopes their content will stick in your brain. “You might think that art is something very subjective,” says Bainbridge, “but people are surprisingly consistent in what they remember and forget.”

While the online experiment was an intriguing start, she continues, “it’s more interesting if we can predict real-world memory.” So Bainbridge along with Davis, then a dual major in neuroscience and fine arts, recruited 19 more people to actually roam the museum’s American Art wing as if exploring with friends. The only requirement was that they had seen each piece at least once. “Especially as an artist, I wanted the results to apply to the real world,” says Davis, who is now the lab’s manager. “We wanted it to be a natural and enjoyable museum experience.”