Progress Report
Development of a Neuroscientific Basis for Visualization and Manipulation of the Mind[1] Establishment of a VR system to visualize brain function network dynamics
Progress until FY2024
1. Outline of the project
This research aims to visualize how the brain works by observing mice as they move freely in a virtual reality (VR) environment. By linking two VR systems, we have created a “mouse metaverse”—a shared virtual space where two mice can interact socially. Through this setup, we analyze how brain activity changes when mice engage in social behaviors, helping us understand the brain's connection to internal mental states—what we call “the mind.”
2. Outcome so far
We successfully combined calcium imaging, a method to visualize brain activity using light, with the VR system. This allows real-time observation of how the brain responds to visual stimuli while the mouse moves freely in a virtual space. Using AI-based machine learning, we developed a system that can predict a mouse's behavior based on its brain activity patterns. In autism model mice, we observed specific changes in the brain areas related to movement. These findings may help explain the motor difficulties often seen in autism and could eventually contribute to new diagnostic technologies.
We created a mouse avatar in the VR space, combining animated movements with scent cues (such as urine) to examine whether test mice show social responses. By combining various sensory inputs—sight, touch, smell—we are analyzing which types of stimulation trigger social interaction.
We also developed an interactive social VR system (iSVR) where two mice can engage with each other in real time. By synchronizing two VR setups, we created a more natural and lifelike virtual environment for studying social behavior.
3. Future plans
With the iSVR system, we can now record both brain activity and behavior from two mice at the same time as they interact in a virtual social space. This makes it possible to study the neural dynamics of social interaction in ways that were previously impossible. We expect this research to bring new insights into the connection between the brain and the mind, ultimately contributing to the scientific understanding of social behavior and mental functions.
(TAKUMI Toru: Kobe University)