Generative AI Unravels the Secrets of Memory and Imagination

A recent study by UCL researchers reveals that breakthroughs in generative AI shed light on the mechanisms through which memories facilitate learning about the world, revisiting past experiences, and crafting entirely novel scenarios for imagination and planning.

Published in Nature Human Behavior, the research employs a generative neural network, an AI computational model, to simulate the learning and memory processes of neural networks in the brain. This simulation involves representing each event with a simple scene.

Incorporating networks symbolizing the hippocampus and neocortex, the model examined their interactions. These two components of the brain are recognized for their collaborative role in processes such as memory, imagination, and planning.

Lead author, Ph.D. student Eleanor Spens (UCL Institute of Cognitive Neuroscience), said, “Recent advances in the generative networks used in AI show how information can be extracted from experience so that we can both recollect a specific experience and also flexibly imagine what new experiences might be like. We think of remembering as imagining the past based on concepts, combining some stored details with our expectations about what might have happened.”

To ensure survival, humans rely on making predictions, such as avoiding danger or locating food. According to the AI networks, when we revisit memories during periods of rest, our brains can discern patterns from past experiences, aiding in the formulation of these predictions.

In an experiment, researchers exposed the model to 10,000 images of basic scenes. The hippocampal network swiftly encoded each scene upon its occurrence, subsequently replaying the scenes iteratively to train the generative neural network in the neocortex.

The neocortical network acquired the skill of transmitting the activity of numerous input neurons (which receive visual information) corresponding to each scene through smaller intermediary layers of neurons. These layers, with the smallest containing only 20 neurons, reconstructed the scenes as patterns of activity in their extensive array of output neurons (which predict visual information).

As a result, the neocortical network developed highly efficient “conceptual” representations of the scenes, encapsulating their meaning, such as the spatial arrangements of walls and objects. This proficiency enabled both the recreation of past scenes and the generation of entirely new ones.

As a result, the hippocampus could encode the essence of newly presented scenes without the necessity of encoding every intricate detail. This ability allowed the allocation of resources to capturing distinctive features that the neocortex couldn’t replicate, such as novel types of objects.

The model elucidates the gradual acquisition of conceptual knowledge by the neocortex and how, together with the hippocampus, it facilitates the “re-experiencing” of events through mental reconstruction.

Furthermore, the model clarifies the process by which new events can be generated during imaginative endeavors and future planning. It sheds light on the reasons behind the prevalent occurrence of “gist-like” distortions in existing memories, wherein distinctive features are often generalized and recollected as resembling the features present in prior events.

Senior author Professor Neil Burgess (UCL Institute of Cognitive Neuroscience and UCL Queen Square Institute of Neurology) explained, “The way that memories are re-constructed, rather than being veridical records of the past, shows us how the meaning or gist of an experience is recombined with unique details, and how this can result in biases in how we remember things.”

More information: A Generative Model of Memory Construction and Consolidation, Nature Human Behaviour (2024). DOI: 10.1038/s41562-023-01799-z

Source: University College London