While we can readily distinguish between more memorable and less memorable images, it's difficult to pin down exactly what affects this perception. Researchers at MIT have devised GANalyze to investigate what changes in “memorability" look like?

This senior co-authors of this research are Aude Oliva  and Phillip Isola whose earlier work on MemNet we reported back in 2015 in Are Your Pictures Memorable? Now they have extended their research into the visual features and properties of images that underlie high-level cognitive attributes like memorability, aesthetics, and emotional valence. 

Their new framework uses Generative Adversarial Networks (GANs) which we have repeatedly encountered in applications such as style transfer, repairing/restoring images and adding speech to avatars. In this instance GANs are applied to the problem of understanding high-level, cognitive image properties, such as memorability.

To obtain the results reported by their study MIT researchers Lore Goetschalckx, Alex Andonian, Aude Oliva, Phillip Isola used the Generator of BigGAN which was pretrained on ImageNet. Its Assessor was MemNet, a CNN predicting image memorability. 

In addition to using the two popular automatic measures, the Frechet Inception Distance (FID) and the Inception Score (IS) the researchers conducted an experiment to collect human scores. They found that the images people remembered best had bright colors, simple backgrounds, and subjects centered prominently in the frame. They noted that a cheeseburger shifted to the far end of the memorability scale not only looks fatter and brighter but also “tastier,” while a panda stands out from its background and its black eyes, ears, and paws contrast sharply and with its white face.

Having verified that their method successfully discovers image manipulations that causally affect human memory performance, they went on to apply the same framework to analyze image aesthetics and emotional valence. to do this they reconfigured GANanalyze to generate images of varying aesthetic and emotional appeal. They found that images rated higher on aesthetic and emotional grounds were brighter, more colorful, and had a shallow depth of field that blurred the background, much like the most memorable pictures. However, the most aesthetic images were not always memorable. 

If you want to know more about GANalze visit its website. If you want to experiement with it, it has a project repo on GitHub.

More Information

GANalyze: Toward Visual Definitions of Cognitive Image Properties Lore Goetschalckx, Alex Andonian, Aude Oliva, Phillip Isola 

Related Articles

Are Your Pictures Memorable?

A Neural Net Colorizes Photos 

SparkleVision - Seeing Through The Glitter 

GANs Create Talking Avatars From One Photo

GANPaint: Using AI For Art

AI Makes Deep Fake News

Inventor Of GANs Joins Apple

The Flaw Lurking In Every Deep Neural Net 

To be informed about new articles on I Programmer, sign up for our weekly newsletter, subscribe to the RSS feed and follow us on Twitter, Facebook or Linkedin.

Comments

Make a Comment or View Existing Comments Using Disqus

or email your comment to: comments@i-programmer.info