Lost in Translation? LLMs, Education, and Linguistic Fairness

Recent advancements in Large Language Models (LLMs) have demonstrated remarkable capabilities, particularly in English prompts. However, lingering questions persist regarding these models’ ability to grasp the subtleties of language beyond mere rule recognition.As these models are increasingly used across various settings, particularly by students and trainees worldwide, a key concern is their ability to understand and accurately translate nuanced English, especially in tasks like translation and academic assistance, where English often dominates as the primary language. In this study, we collaborated as a group of researchers native to diverse linguistic backgrounds to assess LLMs’ aptitude and challenges in translating English idioms and context-rich phrases into 25 languages in a cross-cultural context. Our focus included idioms, gendered sentences, and contextually ambiguous cases. We identified significant patterns in LLMs’ performance on idiomatic expressions across languages. These patterns show substantial disparities and gender bias in translations, particularly disadvantaging low-resource languages. These findings raise concerns about these models worsening educational inequities and call for targeted improvements to ensure fairness and inclusivity. By revealing both limitations and strengths, our work emphasizes the need for a deeper understanding of language nuances to support multiculturalism, expand access to quality models for non-English speakers, and drive equitable progress in education and beyond.