Neural Networks All the Way Down

The brain is a remarkably intricate and multifaceted organ that controls all aspects of human behavior and cognition. It is composed of billions of interconnected neurons that communicate with each other through complex electrochemical signals. Scientists have made significant progress in mapping the connections between these neurons, but we still have a very limited understanding of how the brain processes information and generates behavior.

One of the key challenges in studying the brain is the sheer complexity of its structure and function. The brain is a highly dynamic system that constantly adapts to changing circumstances and stimuli. This complexity makes it difficult to isolate specific neural circuits and understand their role in generating behavior.

Another challenge is the lack of a clear relationship between brain activity and behavior. While scientists have identified many brain regions that are involved in specific tasks, such as language or memory, we still do not fully understand how these regions work together to generate complex behaviors.

The recent work of a trio of researchers at the Brain Mind Institute & Neuro X Institute in Switzerland promises to help us break through some of these barriers and help us better understand the function of the brain, especially as it relates to behavior. They have developed a machine learning algorithm called CEBRA that is capable of learning complex neural representations and associating them with behavioral activities. And once these relationships have been learned, CEBRA can then be used to decode neural activity in some very impressive ways.

At its core, CEBRA is based on a contrastive learning technique that remaps the complex brain signals into a lower-dimensional space in which related data points are clustered together. This has the effect of both reducing the computational workload associated with processing the data, and also of revealing hidden relationships and structure in the data.

CEBRA is flexible in the type of input data that it can accept. It was tested using measurements from electrodes directly inserted into the visual cortex area of a mouse’s brain, and also with data from optical probes inserted into genetically modified mice with neurons that glow when activated. Suffice it to say that the data collection process is highly invasive, and would only be suitable for human use in a relative handful of special cases.

In a stunning demonstration, a dataset from the Allen Brain Observatory in which mice watched videos while their brain activity was recorded with Neuropixels probes or calcium imaging was used to train the model. In this case, video frames were treated as the “behavior” features to see if it would be possible for the system to recognize how brain states correlate with visual observations.

With this trained model, it was shown that CEBRA could predict, with incredible accuracy, unseen frames from the movie based solely on measurements of neural activity. A side-by-side video showed both the original movie, and the movie reconstructed from brain signals, and the two were surprisingly similar, aside from some small timing errors.

CEBRA may not actually be able to read minds — if the model had not been trained on the specific video the mice had watched, it would not have been able to reconstruct what they had seen — but it is a promising tool that will likely help researchers to better understand the brain’s mysteries in the future. And because the methods are general, they can be applied to many types of data in many different domains, which means that CEBRA may be able to offer insights into many biological processes.