InkluBot: Inclusive voice control for robots in the age of large language models (LLMs)

AI-Based Interaction for Assistive Robotic Systems

The InkluBot project aims to further develop AI-based voice control systems for assistive robotic systems in order to make them accessible to people with varying levels of speech ability. The focus is on the individual adaptation and integration of Large Language Models in order to harness the potential of artificial intelligence to enable intuitive robot control as well as comprehensible and context-sensitive interaction between humans and machines. Particular attention is given to processing non-standard speech and the generation of easy-to-understand feedback for people with disabilities.

Three-Stage Adaptive Interaction Model

The key research objectives are to improve automatic speech recognition by optimizing existing AI methods and to develop new methods for the recognition and processing of vocalizations. To achieve these objectives, a three-stage adaptive interaction model will be developed, with the individual stages defined as follows. The first stage focuses on implementing standardized voice control for people without impairments. The second stage involves optimizing existing speech recognition systems with the aim of making them usable by people with mild to moderate speech impairments. The third stage focuses on developing new methods for controlling robots using vocalizations and non-verbal signals (such as gestures), thereby enabling people with severely limited speech abilities to use assistive robotic systems.

Individualized AI Models and Transferable Solutions

The project’s innovative potential lies particularly in the domain-specific adaptation of existing base variants of Large Language Models as well as in the development of a novel system for interpreting vocalizations and non-verbal signals. By using datasets collected specifically for the project in the participating workshops for people with disabilities, the AI systems to be developed can learn to reliably interpret varying speech and communication patterns. InkluBot thus demonstrates the potential of combining AI and robotics to promote self-determined inclusion, support rehabilitation processes, and create company-wide added value. The results achieved can be transferred to further fields of application, particularly in healthcare and industry.