"We Musicians Know How to Divide and Conquer": Exploring Multimodal Interactions To Improve Music Reading and Memorization for Blind and Low Vision Learners

Despite the potential of multimodal assistive technologies (MATs) to convey visual information, such as music notation, to blind or low-vision (BLV) individuals, we do not fully understand how MATs can be used to improve music reading and memorization. Through ideation and co-design workshops, we explored how modalities, such as sound and vibration, can improve music reading and memorization through hands-free timely interactions and reminders. Our design workshops presented a unique opportunity for BLV musicians and learners to collaborate and actively engage in the research and design process informed by their individual perspectives and lived experiences. We classified the complex challenges of reading and memorizing music into intrinsic (related to the cognitive aspects of music understanding) and extraneous (pertaining to external factors such as interaction and access) complexities and found that specific modalities are well suited to tackle particular problems. We conclude by outlining design implications and future research directions aimed at developing MATs that holistically improve music learning for BLV people.