Analysis of the AI prompt used to improve safety management system accessibility

This paper presents a timely and highly relevant investigation for evaluating the effectiveness of AI prompt engineering in improving the safety management system. The study focuses on upstream oil and gas operations, which frequently struggle with quick access to essential safety information. The study investigated an internal AI system configured to access company health, safety, and environmental documentation repositories. The performance is assessed using 30 safety-related prompts across six distinct operational domains, each evaluated on the criteria of relevance, clarity, and completeness, with scores ranging from 1 to 5. Results revealed a trimodal performance distribution, with 60% perfect scores, 30% complete failures attributed to indexing and contextual interpretation issues, and 10% moderate performance( characterized by partial correctness or some degree of inaccuracy). A particularly insightful observation was that high clarity scores were sometimes paradoxically paired with incorrect citations, suggesting that the AI could convincingly present erroneous information. From a process safety perspective, this paper discusses the significant potential of AI to mitigate human error stemming from information scarcity in critical operations. It also critically examines the inherent risks of AI-generated inaccuracies that could lead to potential events in safety-critical contexts. The study recommends the adoption of a hybrid human-AI system, rigorous technical transparency, expanded evaluation methodologies, and a robust framework for addressing ethical considerations to ensure the responsible and effective deployment of AI in hazardous environments. • Present a detailed analysis of the Artificial Intelligence Prompt for Safety Management. • The study identifies critical gaps in the effectiveness of AI prompt use for Safety Management. • The study recommends key elements to be considered in developing a responsible AI system to support Safety Management