Learning and Organization of Memory for Evolving Patterns

Abstract

Storing memory for molecular recognition is an efficient strategy for responding to external stimuli. Biological processes use different strategies to store memory. In the olfactory cortex, synaptic connections form when stimulated by an odor and establish an associative distributed memory that can be retrieved upon reexposure to the same odors. In contrast, the immune system encodes specialized memory by diverse receptors that can recognize a multitude of evolving pathogens. Despite the mechanistic differences between memory storage in the olfactory system and the immune system, these processes can still be viewed as different information encoding strategies. Here, we develop analytical and numerical techniques for a generalized Hopfield network to probe the utility of distinct memory strategies against both static and dynamic (evolving) patterns. We find that while classical Hopfield networks with distributed memory can efficiently encode a memory of static patterns, they are inadequate against evolving patterns. To follow an evolving pattern, we show that a Hopfield network should use a higher learning rate, which can in turn distort the energy landscape associated with the stored memory attractors. Specifically, we observe the emergence of narrow connecting paths between memory attractors that lead to misclassification of evolving patterns. We demonstrate that compartmentalized networks with specialized subnetworks are the optimal solutions to memory storage for evolving patterns. We postulate that evolution of pathogens may be the reason for the immune system to be encoded in a focused memory, in contrast to the distributed memory used in the olfactory cortex that interacts with mixtures of static odors. Our approach offers a principled framework to study learning and memory retrieval in out-of-equilibrium dynamical systems.

Alternate abstract:

Plain Language Summary

Biological systems store memories of molecular interactions to efficiently recognize and respond to stimuli. However, the strategies for encoding a memory vary largely. For example, in the olfactory cortex, an odor can stimulate synaptic connections and establish an associative distributed memory that can be retrieved upon reexposure to the same smell. In contrast, the immune system memory uses diverse receptors that can recognize a multitude of evolving pathogens. Despite the differences between these two systems, these processes can still be viewed as information encoding strategies. Here, we present a theoretical framework with artificial neural networks to characterize optimal memory strategies for both static and evolving patterns.

Our approach is a generalization of the energy-based Hopfield-like neural networks, in which memory is stored as the network’s energy minima. We show that while classical Hopfield networks with distributed memory can efficiently encode and retrieve a memory of static patterns, they consistently misclassify evolving patterns. We demonstrate that compartmentalized networks with specialized subnetworks are the optimal solutions to memory storage for evolving patterns.

The contrast between these memory strategies is reflective of the distinct molecular mechanisms used for memory storage in the immune system and in the olfactory cortex. The memory of odor complexes, which can be assumed as static, is stored in a distributed fashion. On the other hand, the immune system, which encounters evolving pathogens, allocates distinct immune cells to store a memory for different types of pathogens.

Our results suggest that evolution of pathogens may be the reason for the immune system to encode a focused memory, in contrast to the distributed memory used in the olfactory cortex that interacts with mixtures of static odors. Our approach also offers a framework to study learning and memory retrieval in out-of-equilibrium dynamical systems, with broad implications for artificial intelligence and deep learning.