Progress Report
Collaborative AIrobots for adaptation of diverse environments and innovation of infrastructure construction3. Sensor pod system to Obtain Environmental Information
Progress until FY2022
1. Outline of the project
「To achieve "infrastructure construction adapted to diverse environments by multi-construction robots," it is crucial to acquire environmental information and perform evaluations and predictions of the environment. However, technologies for remotely acquiring real-time information in evolving environments and predicting future environmental changes have not been realized thus far. Therefore, in this R&D theme, our primary goal is to develop technologies for acquiring various environmental information, including soil information and the situational status of each robot. We aim to achieve this by deploying a stationary sensor system called the "Sensor Pod" within the environment. Furthermore, we will build a system that aggregates and provides the acquired information to each robot. Additionally, we will develop an "Environmental Evaluation AI" that analyzes this information to predict future ground movements in scenarios such as river channel blockages and landslides.
In addition, developed technologies can be applicable to the earth such as natural disasters and etc.
2. Outcome so far
In 2022, as part of developing technologies for the stationary sensor system called the "Sensor Pod," field experiments were conducted by Kyushu University to validate a method for estimating ground strength using deviations in vibration waveforms, which was developed in 2021. The results of this experiment showed that the proposed method significantly measured an indicator that converges with an increase in the strength of the ground, compared to the conventional method that used vibration sensors attached to a vibratory roller. Furthermore, we conducted development and operational verification experiments for a technology that utilizes multiple LiDAR-equipped Sensor Pods to estimate the real-time positions of multiple construction robots. The results of the experiments confirmed that it is possible to track the real-time positions of multiple construction robots. (Please refer to Figure 2.)
Meanwhile, the Nara Institute of Science and Technology has been developing a technology that estimates the moisture content within the soil by utilizing temperature changes on the ground surface. They have successfully developed a moisture estimation technique that can adapt to unstable sunlight conditions, such as cloudy weather.
Additionally, research and development efforts have been devoted to assessing the environment using the environmental data obtained from the Sensor Pods. One of these efforts focuses on estimating the condition of dams using displacement data obtained from the Sensor Pods during river channel blockage. In 2022, we developed the "virtual site model" of river channel blockages that we constructed using the finite difference method. With this model, we developed forward analysis PINN (Physics Informed Neural Network) for deformation analysis considering groundwater level fluctuations and collapse risk analysis, followed by PINN for continuum elastic analysis this year.
The second development is Environmental Evaluation AI, which assesses disasters based on image data from disaster environments. In this study, we have developed a V&L (Vision and Language) model capable of using figurative expressions in image descriptions and a method for evaluating its performance. Currently, we are verifying the validity of this AI framework by applying it to past disasters and constructing an image-language dataset specifically for landslide collapse sites. Furthermore, we are working on integrating the framework with a Large Language Model (LLM) and advancing rule-based language generation.
3. Future plans
This R&D project for 2025 aims to utilize the Sensor Pods to obtain information from disaster environments, perform environmental assessments, and provide information to collaborative AI robots. To achieve this, by 2023, we will integrate various sensing technologies developed throughout the R&D process into the Sensor Pod. Additionally, we will continue to realize a prototype of the Environmental Evaluation AI for conducting environmental assessments and future predictions.