Ryosuke Isono

Ultra-robust satellite image synthesis based on mathematical and learning fusion models

Grant No.：JPMJAX24C1

Researcher

Ryosuke Isono

Outline

Detailed and accurate Earth observation requires satellite imagery with high spatial, temporal, and spectral resolution. However, there is a trade-off between these resolutions due to differences in the satellites and sensors used. In addition, data degradation such as noise, outliers, clouds and fog are inevitable in the process of satellite measurements. This research aims to realize “robust satellite image fusion,” in which multiple image sequences are synthesized while separating degradation, to obtain image data with high resolution and no degradation, making the most of mathematical optimization and deep learning.

Hayato Idei

Constructing neurodynamical models of cognitive autonomy

Grant No.：JPMJAX24C2

Researcher

Hayato Idei

Outline

The construction of brain models (AI) with autonomy is a critical topic for elucidating the principles of human intelligence and realizing a society where humans and AI coexist harmoniously. However, the mechanism by which autonomy emerges within the brain’s computational processes, understood as a specific optimization problem, remains unclear. In this project, I construct a neurodynamical model that integrates spontaneous behavior generation, multimodal sensory integration, and continual learning. From the perspective of cognitive mode switching (dynamic adjustment of the loss function to be minimized) within free energy minimization, I aim to establish a computational theory of cognitive autonomy.

Yu Inatsu

Extending tight optimization method for black-box function maximization to other settings

Grant No.：JPMJAX24C3

Researcher

Yu Inatsu

Outline

In practical applications, black-box function optimization is frequently used. Recently, a Bayesian optimization method called IRGP-UCB has been proposed. However, while this method can be applied to maximization problems, it is unknown whether it can be extended to various other settings. In this study, I develop an extension of IRGP-UCB for typical settings other than maximization problems, such as robust optimization and Pareto optimization problems, and derive theoretical guarantees and verify practical performance through numerical experiments.

Haruhiko Iwaoki

Development of Primate VR Experimental Paradigms for Curiosity-Driven AI

Grant No.：JPMJAX24C4

Researcher

Haruhiko Iwaoki

Outline

Curiosity drives us to action even in uncertain environments with no apparent rewards, much like the real world. This curiosity mechanism has garnered attention in neuroscience and information science fields such as AI and robotics; however, its true nature remains unclear. This research project aims to integrate information science and neuroscience by developing innovative VR experimental systems for primates to elicit curiosity. The study seeks to identify the environmental factors that generate curiosity and the underlying neural mechanisms through this approach. Ultimately, the goal is to contribute to developing next-generation, curiosity-driven AI.

Ryo Ueda

Emergent Communication among LLMs as Multimodal Representation Learning

Grant No.：JPMJAX24C5

Researcher

Ryo Ueda

Outline

In the contemporary era, where Large Language Models (LLMs) are being integrated into real-world robotics, differences in sensations and physicality between humans and LLMs may hinder linguistic communication. This research addresses the problem of allowing LLMs with different input and output modalities to create a common language (as a form of emergent communication in multimodal representation learning), exploring new directions for linguistic communication in the coming era.

Yui Uehara

Unsupervised learning of harmonic analysis incorporating the musical score simplification

Grant No.：JPMJAX24C6

Researcher

Yui Uehara

Outline

Human experts can build simply describable knowledge from limited learning, in contrast to a large-scale, black-box AI. The harmony theory and the harmonic analysis based on the theory in music is a good example of such human knowledge. This research attempts to acquire regularity in harmonic progressions and perform harmonic analysis in an unsupervised manner. The process of removing non-chord tones from the musical score and simplifying them into chord symbols is performed simultaneously and interdependently with the harmonic analysis.

Kento Uchida

Theory and Applications for Safety Improvement of Optimization with Real-World Evaluations

Grant No.：JPMJAX24C7

Researcher

Kento Uchida

Outline

In optimization problems that appear in the medical and engineering fields, solution evaluations are sometimes conducted in real-world environments. However, these evaluations can involve certain risks. The use of simulators has helped to avoid dangerous evaluations; however, challenges such as production costs and dealing with sim-to-real issues have arisen. This project aims to achieve safe optimization that avoids risky evaluations in real-world environments by building theories related to optimization safety and developing relevant algorithms.

Bowen Wang

Diagnostic reasoning for clinical notes using large-scale language models

Grant No.：JPMJAX24C8

Researcher

Bowen Wang

Outline

Large-scale language models (LLMs) have recently demonstrated excellent capabilities across a wide range of tasks and applications, including in the medical domain. Models such as GPT-4 excel at answering medical questions, but lack accuracy and interpretability when dealing with complex tasks in real clinical settings. In this study, we use a structured method based on LLMs to clearly and logically reason about diagnostic results, which can also improve the accuracy and interpretability of the results.

Shunsuke Kando

Building Spoken Dialogue System without Textual Supervision

Grant No.：JPMJAX24C9

Researcher

Shunsuke Kando

Outline

This research examines the way to build spoken dialogue systems with the supervision of speech resources without the need for text resources. Recent studies have pointed out that the existing systems lack the ability to generate meaningful speech utterances. This study aims to solve this problem by extracting word-like units from speech signals in an unsupervised manner. This research will make it possible to extend spoken dialogue systems to various cultural spheres and domains, leading to the realization of an inclusive AI society.

Yusuke Kunimi

Voide Feedforward: Investigating the Acquisition of Additional Voice Quality through Auditory Feedforward.

Grant No.：JPMJAX24CA

Researcher

Yusuke Kunimi

Outline

I aim to achieve voice feedforward which the intended future speech content is synthesized with other voice and presented audibly. Futuremore, I will examine voice feedforward enhances immersion in any given voice quality presented and induces cognitive changes. I will also investigate whether it is possible to aquire a voice quality that triggers such cognitive changes.

Satoshi Kosugi

A Method for Generating Visual Explanations through Interactions

Grant No.：JPMJAX24CB

Researcher

Satoshi Kosugi

Outline

In this research, I aim to develop a method for automatically generating ‘visual explanations,’ where complex concepts and information are illustrated through visual representations. I plan to develop two models: 1) an image generation model capable of producing visual explanations, and 2) a language model that enables interactive, customizable generation of these explanations. By integrating these models, I seek to provide explanations that adapt to the user’s understanding and requests. This approach is expected to drive innovation in information communication across various fields such as education, healthcare, and business.

Kazuma Sawaya

Development of Deep Variable Selection Techniques to Foster Scientific Discoveries

Grant No.：JPMJAX24CC

Researcher

Kazuma Sawaya

Outline

This project addresses the challenging task of inferring relationships between variables from complex real-world observations. Specifically, it aims to refine two main components of variable selection methods—(i) the computation of importance scores for each candidate variable, and (ii) the selection of variables through thresholding of the computed scores—using theoretical insights from deep neural networks (DNNs). The goal is to develop a methodology for scientific discovery that controls the false discovery rate below a preassigned level.

Kazuyuki Shoji

Multimodal AI for Understanding and Retrieving Spatio-Temporal Data

Grant No.：JPMJAX24CD

Researcher

Kazuyuki Shoji

Outline

In this study, we develop a multimodal AI, MobAI, that integrates mobility data and natural language to facilitate the understanding of mobility data through natural language. MobAI supports analysts in interpreting the meaning of mobility data and promotes the utilization of such data by non-analysts. By addressing the challenges of mobility data analysis complexity and privacy concerns in inter-company data sharing, MobAI aims to support data-driven decision-making across various fields, including urban planning and commercial strategy development.

Hirofumi Suzuki

Studies on Model Update for Smooth Collaboration between Humans and Predictive Models

Grant No.：JPMJAX24CE

Researcher

Hirofumi Suzuki

Outline

In decision-making based on predictive models, users gradually become able to guess the reliability of predictions. On the other hand, predictive models sometimes require updates to maintain their prediction accuracy. After updating predictive models, maintaining the productivity of decision-making is not easy because the behavior of the predictive models changes, and the user cannot guess the reliability in the same way as before the model updates. This research aims to realize model update techniques that make it easier for users to revise their reliability guesses and easily maintain the productivity of decision-making by focusing on changes in prediction results and the basis of predictions.

Shuto Takashita

Operating Extended Bodies through Delegation of Unconscious Movements to AI

Grant No.：JPMJAX24CF

Researcher

Shuto Takashita

Outline

In this research, I aim to realize the operation of extended bodies with greater degrees of freedom than innate bodies by allowing AI to control human unconscious movements in place of the subject. This will allow conscious operational inputs to be distributed over multiple parts of agent bodies (avatars). This system combines an AI system that predicts and reproduces subsequent movements based on previous actions that reflect human intentions, with a force-haptic feedback system that assists in switching operations for multiple body parts, thereby enabling the time division of operational inputs.

Takashi Takahashi

Development of variational methods for the explanation and understanding of learning systems

Grant No.：JPMJAX24CG

Researcher

Takashi Takahashi

Outline

The theoretical explanation and understanding of the learning systems is one of the fundamental elements of data science. The theoretical analysis so far has explored fundamental aspects that hold independent of the details of the setting. However, as learning tasks become more complex, giving quantitative theoretical prediction in specific real-world settings has become challenging. In this study, we aim to establish a framework for inferring mathematical models that approximate the behavior of learning systems, which will help develop a methodology to bridge the gap between theory and reality.

Ning Ding

Unraveling Dynamic Coordination Mechanisms for Tactical Scene Analysis in Racquet Sports

Grant No.：JPMJAX24CH

Researcher

Ning Ding

Outline

This research aims to explore and understand the mechanisms of player coordination in response to evolving tactical dynamics in racquet sports. By leveraging reinforcement learning, the study seeks to model the adaptive strategies players employ when faced with tactical changes in-game scenarios. Furthermore, the research develops a framework for tactical scene interpretation, utilizing a multimodal Transformer model to analyze and generalize insights across diverse tactical situations within racquet sports.

Kazuki Nakajima

Inference and visualization of higher-order community structure based on stochastic block models

Grant No.：JPMJAX24CI

Researcher

Kazuki Nakajima

Outline

Computational methods are needed to accurately infer and visualize higher-order community structure (i.e., groups of individuals who interact closely with each other) by representing a social network with a hypergraph that involves high-order interactions among three or more individuals. In this project, I study stochastic block models for inferring higher-order community structure by considering structural characteristics of empirical hypergraphs. I also develop a framework for embedding nodes in a low-dimensional space while accurately preserving inferred higher-order community structure.

Mikito Nanashima

Computational Complexity of Learning and Compression

Grant No.：JPMJAX24CJ

Researcher

Mikito Nanashima

Outline

We investigate the relationships between the description size of hypotheses, computational complexity, and precision in the context of learning and compression. Building on this foundation, we aim to make progress toward resolving the important open quest

Naoki Nishikawa

Statistical Analysis of Attention Mechanisms for Developing Efficient Models

Grant No.：JPMJAX24CK

Researcher

Naoki Nishikawa

Outline

The network architecture called Transformer has achieved remarkable success as the standard for foundation models. The high performance of Transformers is due to the attention mechanism. One challenge with the attention mechanism is its high computation cost. While some alternatives with lower cost have been proposed, their capabilities are inferior to those of the attention mechanism. In this study, we investigate the capabilities of the attention mechanism and alternative methods based on statistical theory. We first identify the situations where the attention mechanism can be replaced by alternative methods without a loss in performance. Then, we aim to establish a methodology for developing efficient and high-performing networks.

Takaaki Nishimoto

Developing compressed data structures achieving an information-theoretic lower bound

Grant No.：JPMJAX24CL

Researcher

Takaaki Nishimoto

Outline

Recently, the size of string datasets has grown rapidly. To process such massive datasets within practical time and memory constraints, it is crucial to develop data structures that efficiently answer queries on the data. Memory usage can be reduced by employing data compression techniques, and there is an information-theoretic lower bound for memory usage, referred to as the δ-optimal space. This project aims to develop compressed data structures that not only achieve the δ-optimal space but also efficiently process various queries.

Akari Haga

Efficient Multimodal Language Model Based on Imitation of Child Speech Errors

Grant No.：JPMJAX24CM

Researcher

Akari Haga

Outline

While current large language models achieved remarkable success, they still required large amounts of data for training. In contrast, children acquired language with limited resources. In the field of first language acquisition, previous research suggested that the errors made by children during the acquisition process are crucial for efficient language learning. This project aimed to improve the learning efficiency of multimodal language models by imitating the typical errors made by children when associating visual information with language.

Kohei Honda

Safe Reinforcement Learning for High-Speed Autonomous Agents Under Uncertainty

Grant No.：JPMJAX24CN

Researcher

Kohei Honda

Outline

This research aims to develop and implement an online safe reinforcement learning framework for controlling high-speed autonomous agents. To address uncertain dynamics and safety constraint models in high-speed domains, we will utilize reinforcement learning combined with probabilistic inference-based optimal control. This approach allows us to acquire adaptive and optimal policies while satisfying safety constraints. Ultimately, this project aims to realize a “trusted high-speed autonomous control AI” capable of safely learning superhuman control in real-world environments.

Koki Madono

Development of a Drawing Path Generation AI Capable of Decomposing Completed Images

Grant No.：JPMJAX24CO

Researcher

Koki Madono

Outline

Research into generative AI for various users, including art enthusiasts, painters, and traditional technique successors, is currently thriving. While generating images through AI is now feasible, generating drawing sequences from images—a task known as drawing sequence generation—remains a challenging problem due to issues with poor settings, and current technology does not yet achieve the precision that satisfies users. This research aims to develop a new AI that optimizes drawing sequence estimation from human drawing data, addressing this as a well-posed problem and achieving more plausible results.

Chiaki Miura

Push Citation: Peer Review Mechanism without Monetary Incentives

Grant No.：JPMJAX24CP

Researcher

Chiaki Miura

Outline

Demand for a quality and efficient reviewer assignment is increasing faster. I suggest a “push” in academic citation, integrating the newer scientific findings into a more extensive, ever-growing literature. The mechanism is built on the recent success of post-publication peer review. Learning from open-source software (OSS) development, I demonstrate how citation metrics can incentivize reviewers. This project will focus on the literature recommendation to tell the authors which literature to push their work. I construct a graph neural network model where each article corresponds to a subgraph of a knowledge graph constructed from the existing literature.

Yamato Miyatake

Personalized Food Fabrication System for Optimizing Taste Based on Individual Perception and Preferences.

Grant No.：JPMJAX24CQ

Researcher

Yamato Miyatake

Outline

In recent years, the demand for personalized food has been rapidly increasing due to the diversification of dietary habits and the growing awareness of health. Traditional systems provide food based on databases, but individual tastes and preferences vary and change over time, making personalized optimization necessary. In this study, we aim to develop an AI that learns individual preferences and a food fabrication system that controls flavor, with the goal of constructing a system that optimizes the taste of food for each individual.

Yoàv Montacute

Symmetry in AI: Combinatorial Topological Methods for Next-Generation AI

Grant No.：JPMJAX24CR

Researcher

Yoàv Montacute

Outline

Artificial intelligence prediction methods excel at mimicking human responses, but formal and logical reasoning remains a challenge. This research explores novel approaches to applying complex mathematical structures to AI and formally verifying their properties, as well as accelerating AI’s inferential computations using these structures. Through this innovative approach, we aim to establish a theoretical foundation for enhancing AI’s logical reasoning capabilities, contributing to the development of more sophisticated decision-making systems.

Ryosuke Yamaki

Exploration of Creative Language Use by LLMs Based on the Integration of Syntax and Semantics

Grant No.：JPMJAX24CS

Researcher

Ryosuke Yamaki

Outline

This study aims to computationally elucidate the mechanisms underlying the creative use of language, a cognitive function unique to humans, by integrating Holographic CCG, a syntactic parsing model based on Combinatory Categorial Grammar, with large language models and enabling interaction between these models. Through this approach, I seek to advance academic understanding of human cognitive functions while also enhancing advanced language use by AI and improving AI alignment.

Taiki Yamada

Extended Learning Theory of Dynamical Systems Considering the Finiteness of Data

Grant No.：JPMJAX24CT

Researcher

Taiki Yamada

Outline

Research on the generalization performance and interpretability of AI, which has contributed to the development of trustworthy AI, can be understood as efforts to ensure reliability while considering the finiteness of data quantity and quality. In this study, we address the meta-question of ‘what kind of behavior can be guaranteed for AI under the finiteness of data’ by analyzing learning tasks of dynamical systems. Through studying the mathematical structures inherent in these tasks, we aim to characterize intelligence that is learnable and interpretable by humans.

Zihui Li

Improving the Trustworthiness of Japanese Medical Text Generation Tasks with LLMs

Grant No.：JPMJAX24CU

Researcher

Zihui Li

Outline

AI is becoming more important in healthcare, helping with tasks like answering medical questions and giving treatment advice. This project aims to make AI better at answering medical questions by using a large language model (LLM) that can give more accurate answers with the help of a medical knowledge graph. The project will also work on improving other types of medical text generation and will create a high-quality Japanese medical dataset to further improve the AI’s performance in medical tasks.