TOP > Publications > Sensor Fusion Technologies in IoT era - Multimodal Sensing and Data Processing for Creating New Value -/CRDS-FY2019-SP-10
Mar. /2020
(Strategic Proposals)
Sensor Fusion Technologies in IoT era - Multimodal Sensing and Data Processing for Creating New Value -/CRDS-FY2019-SP-10
Executive Summary

The IoT is required to acquire a wide range of data through sensors installed at a variety of places to derive sophisticatd, valued information from such data, i.e., it is required to add high values to sensing information. To realize this, it is essential to acquire a wide range of sensing data and process it in an integrated manner, which requires the buildup of the base technology that enables this. This proposal focuses particular attention on the sensing system used on the data creation side (edge side). Based on this, we propose that in order to build the base technology required for adding high values to sensing information, we should work on the research and development challenges at three different levels (system, sensor terminal, and sensor levels) collaboratively. It should be noted that the term "sensor" used in this document refers to a device that detects desired physical/chemical quantities by converting them into electric signals, and the term "sensor terminal" means a device consisting of components such as sensors, analog-digital converter circuits, battery units, and communication circuits, installed near the object to be measured. As such, this proposal differentiates between the sensor and sensor terminals in terms of strategy structure.

The buildup of a sophisticated IoT system capable of adding high values to sensing information requires a high-performance cloud server and network along with an edge-side sensing system that acquires a variety of useful data, and processes such data in order to send it to the cloud. Traditional sensing systems are used in limited places, such as factories, for a specific application and only acquire and process limited types and amounts of data. On the other hand, the IoT must collect a much wider range of data---information such as on the environment, device operation, and human health---without human intervention. To this end, it is important to use the sensing system at the edge side to automatically collect a variety of information in order to make a determination on the spot and take necessary action, or, in order to process information, send it to the cloud for determination at the cloud side.

For big data and AI-related technologies and services in the upper layers of the IoT system such as the cloud, GAFA in the U.S.A. and other platformers have already dominated the market, and thus many players around the world start to take interest in the lower layers, i.e., the edge side. With strength in sensing technology, which is positioned in the lowest layer of the IoT system, Japan is expected to lead the IoT industry from the lower layer by delivering an excellent edge-side sensing system.

This edge-side sensing system (hereafter simply referred to as the sensing system) consists of a sensor terminal installed near the object to be measured for the purpose of acquiring data, and a device (for processing edge-side information) that processes data acquired by the sensor terminal to send it to the cloud. Although the capabilities required for the sensing system depend on the application, it is recognized as a common direction of the sensing system to acquire a variety of information and add high values to sensing information through integrated processing.

Under this situation, this proposal proposes that toward a sensing system capable of acquiring a variety of information and processing it in an integrated manner, we should work on the following research and development challenges:

Research and development challenge 1: Integrated processing of sensing information (system-level challenge) With consideration given to the role of the cloud, located in the upper layers of the IoT system, we must design the architecture of an edge-side information processing device and sensing system consisting of sensor terminals and a network. The specific architecture highly depends on each application and thus must be designed so as to reflect the requirements of the users. The research and development challenges also include those associated with sensor fusion, which fuses multiple sensing data items to derive high-value added information; event-driven sensing, which detects temporal changes in data to make changes of commands to be communicated to upper/lower layers for causing actuation; and active sensing, which works together with an actuator to change the way of next data acquisition based on the data acquired.

Research and development challenge 2: Optimization and sophistication of sensor terminals (sensor-terminal-level challenge) It is required to develop base technology for creating sensor terminals that provide functions and capabilities best suited for each of a variety of applications. Sensor terminals involve individual blocks of functions such as those related to sensor control, analog front end (AFE), temporary storage of data, data conversion/feature amount extraction, communication, and power-supply control/energy harvesting. Further research and development activities are required for each of these blocks. In addition, as the technologies that integrate the function blocks, the integration and mounting technologies are vital in creating sensor terminals with optimal functions and capabilities.

Research and development challenge 3: Improvement in sensor performance (sensor-level challenge) In order to add high values to sensing information, it is required to increase the types and range of sensing data, and it is important to improve the performance of sensors and apply new detection principles to sensing. For biological information sensing based on a chemical sensor or biosensor, it is required to increase the sensitivity of the sensor and improve selectivity in order to detect infinitesimal biomarker molecules. Physical sensors, many of which are already in practical use, are required, for example, to be further sensitized and downsized depending on the application. It is also an important challenge to improve the sensor capabilities by arraying multiple sensors. In addition, it is also important to apply to the sensing principles, new physical phenomena/chemical reactions and/or new materials that have never been applied, with the object of making it possible to sense physical/chemical quantities that have been undetectable.

In addressing these research and development challenges, it is required to design the system, sensor terminal, and sensor based on the entire system architecture, with information exchanged among different technology layers. To this end, it is effective to build a research and development platform with a capability for developing necessary base technology to promote cooperation among experts in different areas. In addition, it is important to respond to a variety of needs of users based on the base technology; to do so, it is essential that (1) a wide range of base technologies that realize the sensing system should be always maintained in the most advanced way and (2) coordinators should exist, who strongly promote needs and seeds matching between sensing system users and base technology providers. A physical base must be formed to realize these functions.

Under this situation, where time is required for the research and development of the sensing system, in particular, the development of hardware such as new sensors and sensor terminals, efforts are required from a long-term perspective based on a government policy and with cooperation between academia and industry. It is important to build the research and development platform mentioned above at an early stage, as well as to work steadily on the research and development challenges. In working on the challenges, it is assumed that research activities will be conducted by academia with focus placed on seeds research, and that research and development activities will be carried out based on needs from industry. In both cases, the key is cooperation between academia and industry. The research and development platform is expected to provide a basis that plays a role of collecting and integrating excellent base technologies individually owned by universities to promote their utilization in the society.

Related Reports