Intelligent Supportive System for People with Profound Intellectual and Multiple Disabilities

People with profound intellectual and multiple disabilities (PIMD) are a vulnerable and often marginalized group of people, commonly without the ability of symbolic communication. This makes expressing and recognizing their inner state and desires challenging and requires in‐depth knowledge and understanding of each individual. Use of standard assistive technologies is thus infeasible due to lack of personalization. This challenge is tackled by the INSENSION system—a novel intelligent decision support system leveraging state‐of‐the‐art noninvasive audio‐visual sensor technologies together with machine learning algorithms and expert knowledge, to detect and interpret behaviors and communications (nonverbal signals—NVSs) of people with PIMD in challenging real‐world scenarios. The detection of NVSs supports caregivers and allows the people with PIMD a form of communication through a communicator application. Furthermore, the system enables them to control and adjust their environment through a smart room and multimedia player. According to quantitative and qualitative evaluation, good recognition accuracy is achieved, and the system empowers people with PIMD and increases their quality of life.