Beyond LENA: Open-Source NLP & AI Tools for Audio Data

This tutorial introduces Natural Language Processing (NLP) algorithms and Artificial Intelligence (AI) tools for processing long-form audio data. As typical projects on speech can have datasets of up to thousands of hours of audio, researchers cannot rely only on manual annotation. Automatic speech processing tools became necessary to work with very large datasets. Moreover, reliance only on proprietary NLP and AI tools can hinder the reproducibility of experiments by a wide research community. Therefore, in this tutorial, we focus on open-source tools such as VTC (Voice Type Classifier) and ALICE (Adult LInguistic unit Count Estimator). VTC helps segment audio files into broad speaker categories, while ALICE estimates the number of linguistic units (e.g. syllables, words) produced by an adult speaker. We also present open-source Audio and Multimodal Large Language Models (LLMs) that can help extract linguistic features from audio data, in a multilingual context.