Progress Report

The Realization of an Avatar-Symbiotic Society where Everyone can Perform Active Roles without Constraint1. Development of a CA which has a humanlike presence and a lifelikeness

12345678Next»

Progress until FY2022

1. Outline of the project

In this R&D theme, our goal is to develop cybernetic avatars (CAs) with humanlike appearance, functionality, and lifelike characteristics. Additionally, we will create mobile CAs with childlike features that can extend their presence and lifelikeness by being able to move freely in the environment. Furthermore, an automatic motion generation system will express the teleoperator's intentions, emotions, and personality through CAs. An immersive interface will provide operators with free control over the CAs. Through these R&D efforts, CAs will serve as substitutes or avatars for teleoperators in social situations, overcoming spatial and temporal constraints. It will enable us to effectively accomplish and excel in tasks that require simultaneous presence at multiple locations.

2. Outcome so far

1. We designed and implemented expressive gestures into the CA to enable it to interact with users through appropriate body movements and facial expressions based on the dialogue partner and content. This implementation allows the CA to perform fluent and beautiful gestures, particularly in situations where it is crucial to appeal to the dialogue partner, such as during presentations. As a result, the CA now exhibits more expressive and hospitable behavior than the operator themselves (Figure 1).

Figure 1: CA which has a humanlike presence, exhibiting more expressive gestures than the operator themselves
Figure 1: CA which has a humanlike presence, exhibiting more expressive gestures than the operator themselves

2. We developed a CA for the Minister of the Digital Agency (Figure 2) and conducted a field experiment in which the Minister provided official services by remotely operating the CA. The Minister controlled the CA installed at a commercial facility and offered official services, such as explaining the initiatives of the Digital Agency to passers-by. During this field experiment, we conducted a survey on the social acceptability of CA use by individuals with significant influence.

Figure 2: Digital Agency Minister's CA and the Minister himself
Figure 2: Digital Agency Minister's CA and the Minister himself

3. We have developed CAs with lifelikeness by making their moving mechanism silent. It has been confirmed that these silent CAs are perceived as having more agency compared to CAs with motor sound. Additionally, we have created a life-like CA equipped with an organic EL display on its face. This CA can freely alter its facial expressions, allowing for a more diverse expression of personality.

4. By employing inverse reinforcement learning, the avatar has acquired the skill to recognize the optimal moments to request disinfection, surpassing the perceptual abilities of typical individuals. It contributed to the overall goal of expanding the capabilities of the operator.

Figure 3: Mobile life-like CA
Figure 3: Mobile life-like CA
Figure 4: CA asking for disinfection at a shopping mall
Figure 4: CA asking for disinfection at a shopping mall

3. Future plans

The CA to be developed under this R&D theme will free the operator from physical, spatial, and temporal constraints by taking their place. Furthermore, we aim to develop CAs with functions and capabilities that go beyond mere substitution for humans. Through such CAs, we will not only be able to work in different places and at different times, but we will also be able to demonstrate abilities and behaviors that we would not normally be able to do.

12345678Next»