Readability of Patient Information Leaflets Improved by Large Language Model Artificial Intelligence

Introduction: Artificial intelligence (AI) is revolutionizing healthcare. Emergency departments (EDs) are under increasing time pressure, and physicians often rely on patient information leaflets (PILs) to inform/reassure patients with common presentations. International recommendations require PILs to be at an appropriate reading level (that of a 12-year old child), thereby improving comprehension for patients (7, 8). Some PILs are too difficult to read for patients(9, 10), demanding consideration of appropriate lexical complicity (7, 11). Methods: This study assesses whether commercially available large language models (LLMs) can improve the reading comprehension level of PILs. The readability/lexical complexity of 19 PILs currently used in an Irish ED was measured with validated reading level metrics, including the Common European Framework of References for Languages (CEFR) grade, using an online tool against the set standard reading comprehension. Two LLMs were employed with the same instructions to rewrite the PILs, scored using the same metrics, and improvements quantified. Results: Only three current PILs met the required lexical complexity level (median 63.3%, IQR 7.3). LLM generated PIL overshot the improvement in reading level, simplifying it by a median of 4% (1 CEFR level). The first LLM improved the median reading level to 60.25% (CEFR = C1, IQR 6.41), while the second LLM resulted in a median reading level of 56.92% (CEFR = C1, IQR = 5.6), meeting the target in 12 of 19 PILs. Conclusion: The findings demonstrate the importance of a critical review of the reading levels used in EDs. LLMs may be employed to facilitate improving readability, but may not be able to reach the instructed reading level at first iteration, and are more useful in simplifying PILs at an initial college reading level. All AI-generated PIL retained the core message. Currently available LLMs may be utilized by healthcare workers to simplify PIL, improving readability to the recommended reading level.