5PSQ-069 Portability of a text mining algorithm for detecting adverse drug reactions in electronic health records across diverse patient groups in two Dutch hospitals

<h3>Background and Importance</h3> Adverse Drug Reactions (ADRs) pose a significant challenge in healthcare.¹ While structured documentation of ADRs in electronic health records (EHRs) enables automated alerting, many ADRs are recorded as unstructured free-text, limiting detection.^{2 3}Text mining (TM) shows potential for extracting clinically relevant data from unstructured text. However, the portability of TM algorithms across different institutions and departments remains uncertain, due to variations in EHR structures and documentation practices.⁴ To enhance these general-purpose algorithms, evaluating their portability is essential for ensuring effective performance across diverse clinical settings. <h3>Aim and Objectives</h3> To evaluate the portability of a previously developed TM-based ADR identification algorithm by assessing its performance using EHRs from two different departments in two different hospitals. <h3>Material and Methods</h3> EHR free-text data from 62 hospitalised patients in the geriatric and orthopaedic departments of two Dutch teaching hospitals were reviewed for ADRs via manual review and the TM algorithm. Performance was evaluated using F-score, sensitivity and positive predictive value (PPV), with comparisons across hospitals and departments. <h3>Results</h3> Manual review identified 359 unique ADRs. The TM algorithm detected 534 potential ADRs, 286 of which overlapped with manual review, yielding an F-score of 0.64, sensitivity of 80% and PPV of 54%. Performance was consistent across hospitals and departments. Notably, 26 ADRs identified by the algorithm were clinically relevant yet missed in manual review. <h3>Conclusion and Relevance</h3> This study demonstrates portability of the TM algorithm by identifying pADRs across different hospitals and departments without adaptations. These findings support its broader implementation potential for ADR detection in diverse healthcare settings <h3>References and/or Acknowledgements</h3> 1. Lazarou J, 1998. https://pubmed.ncbi.nlm.nih.gov/9555760/ 2. McLachlan G, 2023. https://pubmed.ncbi.nlm.nih.gov/34930047/ 3. Van Der Linden CMJ, 2013. https://pubmed.ncbi.nlm.nih.gov/25083253/ 4. Sun W, 2018. https://pubmed.ncbi.nlm.nih.gov/29849998/ <h3>Conflict of Interest</h3> No conflict of interest