PREFER-IT: A transdisciplinary co-created framework to realise inclusive medical AI

Abstract Artificial intelligence (AI) in healthcare holds transformative potential but risks exacerbating existing health disparities if inclusivity is not explicitly accounted for. This study addresses the disconnected discussions on inclusive medical AI by developing a comprehensive framework, PREFER-IT. This framework is based on the outcomes of a five-day transdisciplinary co-creation workshop that involved 37 experts from diverse backgrounds, including healthcare, ethics, law, social sciences, AI, and patient advocacy. For this workshop, we used design thinking and participatory methodologies to develop a framework for realising inclusive medical AI. We identified three key challenges for realising inclusive medical AI: integrating the lived experiences and stakeholder voices across the AI lifecycle, designing data collection practices that promote fairness and prevent inequalities, and fostering regulatory frameworks to uphold human rights and promote inclusivity. The analysis of participants’ perspectives informed the development of eight key thematic clusters of PREFER-IT: Participatory and co-design approaches (P), Representative and diverse data (R), Education and digital literacy (E), Fairness (F), Ethical and legal accountability (E), Real-world validation and feedback (R), Inclusive communication (I), and Technical interoperability (T). These elements were mapped across structural layers of AI (humans, data, system, process, and governance) and the AI lifecycle to guide inclusive design, development, validation, implementation, monitoring, and governance. This framework fosters stakeholder engagement and systemic change, positioning inclusion as a guiding principle in practice. PREFER-IT offers a practical and conceptual contribution for how to include ethical, legal and societal aspects when aiming to foster responsible and inclusive AI in healthcare. Author Summary Artificial intelligence (AI) is being used more and more in healthcare to improve diagnosis, treatment, and personalised care. However, if not designed carefully, these technologies can unintentionally increase existing inequalities and exclude certain groups from their benefits. In our study, we brought together experts from healthcare, ethics, law, social sciences, and patient advocacy to explore how AI in medicine can be made more inclusive. Over five days, we worked together to identify key issues and come up with practical solutions. We focused on three main areas: 1) Ensuring diverse voices are heard during the development of AI tools; 2) Making data collection fair and representative; and 3) Creating regulations that protect human rights. From the discussions of the workshop, we created the PREFER-IT framework, which outlines eight key principles for inclusive AI: P articipatory and co-design approaches R epresentative and diverse data E ducation and digital literacy F airness E thical and legal accountability R eal-world validation and feedback I nclusive communication T echnical interoperability This framework helps guide developers, policymakers, and healthcare professionals in creating AI systems that are not only effective but also fair and respectful of all users. Our work emphasises the importance of involving patients and communities in shaping the future of AI.