Memory systems for reinforcement learning

August 5, 2017

Richards BA, Frankland PW, “The Persistence and Transience of Memory”, Neuron, 94 (6), 1071-1074

The predominant focus in the neurobiological study of memory has been on remembering (persistence). However, recent studies have considered the neurobiology of forgetting (transience). Here we draw parallels between neurobiological and computational mechanisms underlying transience. We propose that it is the interaction between persistence and transience that allows for intelligent decision-making in dynamic, noisy environments. Specifically, we argue that transience (1) enhances flexibility, by reducing the influence of outdated information on memory-guided decision-making, and (2) prevents overfitting to specific past events, thereby promoting generalization. According to this view, the goal of memory is not the transmission of information through time, […]
January 31, 2017

Santoro A, Frankland PW, Richards BA, “Memory Transformation Enhances Reinforcement Learning in Dynamic Environments”, Journal of Neuroscience, 36 (48), 12228-12242

Over the course of systems consolidation, there is a switch from a reliance on detailed episodic memories to generalized schematic memories. This switch is sometimes referred to as “memory transformation.” Here we demonstrate a previously unappreciated benefit of memory transformation, namely, its ability to enhance reinforcement learning in a dynamic environment. We developed a neural network that is trained to find rewards in a foraging task where reward locations are continuously changing. The network can use memories for specific locations (episodic memories) and statistical patterns of locations (schematic memories) to guide its search. We find that switching from an episodic […]
June 24, 2015

Multiple memory systems for enhanced reinforcement learning

Not all of our memories are stored in the same way. We have multiple memory systems that can provide different types of information [1,2]. Some of our memories are very detailed, giving us the ability to recall specific events and remember what it felt like when we were living it [3]. Our recent memories are stored in this way, as are some of the more emotionally salient events from our life. For example, you might be able to recall exactly what you had for breakfast yesterday or the song that was playing during your first kiss. In contrast, many of […]
June 24, 2015

Richards BA, Xia F, Santoro A, Husse J, Woodin MA, Josselyn SA, Frankland PW, “Patterns across multiple memories are identified over time”, Nature Neuroscience, 17(7), 981-986

Memories are not static but continue to be processed after encoding. This is thought to allow the integration of related episodes via the identification of patterns. Although this idea lies at the heart of contemporary theories of systems consolidation, it has yet to be demonstrated experimentally. Using a modified water-maze paradigm in which platforms are drawn stochastically from a spatial distribution, we found that mice were better at matching platform distributions 30 d compared to 1 d after training. Post-training time-dependent improvements in pattern matching were associated with increased sensitivity to new platforms that conflicted with the pattern. Increased sensitivity […]