The goal of this theme is to develop technology to extract thoughts and mental states from brain activity using various devices and to implement it in society as an application, thereby promoting the use of brain-machine interface (BMI) technology. Specifically, we are developing algorithms that can extract thoughts and mental states in daily environments for short periods of time by combining gadget-type electroencephalography (EEG) sensors such as headphones and camera images from cell phones. Using these, we aim to disseminate BMI technology to society by creating applications that enable self-regulation by visualizing daily physical changes of which one is not aware, and applications that support the communication of intentions of users who are in a state or situation where they cannot externally express their intentions.

Controlling Avatars

Junichi Ushiba group has further developed a technology that uses headphone-type EEG sensors to read the user's motor intentions and “control” avatars in the metaverse space game Fortnite through the power of thought. Brain waves contain a variety of noise for various reasons, and the group has developed “noise cleansing technology” that removes these noises from the measured signals by preparing “noise recognition AI” specialized for each type of noise. In addition, we have developed an “AI assist function” that identifies obstacles in the Fortnite space and assists avatar operation to avoid collisions, and a "brain function enhancement AI" generates sound and visual effects to naturally strengthen the brain waves needed for avatar operation has been developed. Such a "composite AI”that combines these multiple AIs has created a technology that allows the avatar to move smoothly as desired in a Fortnite environment with various obstacles. The group is also working on the development of a new interface for the IoB interface by Katsumi Watanabe group.
Katsumi Watanabe group is analyzing the conditions that give avatars their own physicality and sense of ownership. They are also working on the development of testing methods to quantitatively and objectively quantify these ever-changing senses.

Brain Monitoring and Training through Avatar

The difficulties caused by brain limitations are widely shared among musicians and music lovers. The “ceiling effect” is when skills acquired as a result of repeated training reach a plateau, but we have found that even when this state is reached, there is still room for growth in the brain. Specifically, Shin-ichi Furuya group found that when high-speed finger movements are input into the brain by using an exoskeleton robot, the brain's finger movement skills improve even after the robot is detached from the body. The results of this study have been well-received both in Japan and abroad, and were featured on the cover of Science Robotics, the journal in which the paper was published.

Ai Koizumi group has also been testing a “self”-reflecting avatar application that normalizes memory consolidation for people suffering from traumatic memories, and have promoted an understanding of the mechanisms in the brain through EEG and MRI measurements.

Social implementation

This series of efforts has been steadily returned to society through trials at the Music Excellence Project Academy (in Japan), trials at the Hannover College of Music and Munich College of Music (overseas), and commercialization through the university start-up LIFESCAPES, Inc.

While publishing world-class academic papers, conducting international joint research activities, and promoting technological standardization, we aim to realize “a society where everyone can pursue their dreams” and “a society where people can enjoy their lives until age 100 without health concerns” by promoting radical innovation that harmonizes academic inquiry and social implementation.