Large language models powered system safety assessment: applying STPA and FRAM

• Evaluated ChatGPT and Gemini in stand-alone STPA and FRAM safety assessments. • The study used LH2 aircraft refuelling as a novel, complex case study. • ChatGPT outperformed Gemini in structured STPA analysis tasks. • Both LLMs struggled with systems thinking and output consistency. • Findings offer lessons for AI integration in system safety assessments. The advancement of large language models (LLMs) shows immense promise in many domains. However, their reliability is still questionable. This study aims to comparatively examine the performance of ChatGPT and Gemini in conducting a stand-alone systems-based risk assessment using System-Theoretic Process Analysis (STPA) and the Functional Resonance Analysis Method (FRAM). Our findings revealed that both LLMs demonstrated weaknesses in their analyses, with ChatGPT generally outperforming Gemini regarding response comprehensiveness and adhering to the prompted format. Specifically, LLMs failed to use systems thinking in their stand-alone applications and failed to follow up on previous prompt outputs. While LLMs can provide substantial amounts of information quickly, the effectiveness of LLMs in system safety assessment is contingent on addressing their limitations and implementing strategies to improve their capabilities.