Progress Report
The Realization of an Avatar-Symbiotic Society where Everyone can Perform Active Roles without Constraint4. Cooperative control of multiple CAs
Progress until FY2022
1. Outline of the project
This R&D theme is responsible for basic research and development of technology for simultaneous remote control and coordinated control of multiple CAs (flexible CA control). Achievement of this R&D theme will enable a single operator to operate multiple CAs simultaneously, contributing to the realization of an avatar symbiotic society, which is the goal of the project, and to the liberation from space and time constraints, which is the goal of the Moonshot target.
To achieve this, the challenge is to develop technology to use multiple CAs that work while understanding the operator's intentions according to the task and environment. In this R&D theme, first, the living and hospital room environments in which CAs work together are actually reproduced, and the semi-autonomous CAs that work in these environments are prototyped based on autonomous CA technology that can be made autonomous from remote operation. Then, a semi-autonomous cooperative control system consisting of multiple CAs will be developed, including BMI and the operation interface.
2. Outcome so far
An integrated system was implemented and a demonstration experiment was conducted.
(1) An integrated system was implemented in which five physically operated CAs could be operated by a single person. A total of five physical CAs can now be operated by a single person in Tokyo x 1 (elderly persons' homes) and Osaka x 2 (elderly persons' facilities and hospitals).
(2) Experiment of a CA system for taking care of elderly people at elderly people's facilities/hospitals were conducted. We interviewed the staff to find out what kind of CA services were needed, and clarified that it would be good to be able to share information about the activities of the residents with the CA system. An automatic information sharing system using SNS was then implemented.
In addition, the following are representative technologies developed as the basis for flexible CA control.
- (1) Robot world model realization using Transformer
- (2) Development of a cooperative method for multiple CAs
- (3) Implementation of a computational model in CA that integrates on-the-spot learning based on a place concept model and common sense knowledge based on symbolic logic
- (4) Developed HLSM-MAT, a method for solving object manipulation instructions, and achieved the world's highest performance in the ALFRED task
- (5) Realized smooth and stable grasping using proximity sensors to cope with communication delays
- (6) Achieved BMI voluntary control by decoding intracranial EEG and achieved 95% accuracy in writing operation
3. Future plans
In the future, we will take on the challenge of realizing even more CA simultaneous remote operations. We believe this can be achieved by incorporating the elemental technologies we have developed so far into an integrated demonstration system and building a mechanism for CAs to work together autonomously. In addition, we will promote further development of elemental technologies by continuing field studies.