The Qualitative Engine: Creating and Evaluating an Iterative <scp>AI</scp> Modeling Tool

ABSTRACT The rise of generative artificial intelligence (AI) has introduced new possibilities for automating qualitative system dynamics modeling, lowering barriers to entry while raising concerns about methodological rigor and ethical integrity. This paper presents the qualitative engine, an AI‐assisted modeling tool developed as part of the open‐source sd‐ai platform, designed to help users generate causal loop diagrams (CLDs) using large language models (LLMs). Unlike prior systems such as SDBot, the qualitative engine employs a zero‐shot, one‐pass prompting strategy with structured JSON outputs to ensure consistent, parseable results. It supports incremental model building and contextual integration of problem statements and background information, enabling iterative human‐AI collaboration. The paper also describes the results of assessing the engine on evaluation frameworks: causal translation (accuracy in extracting trivially stated causal relationships) meant to assess only the most basic form of competency and conformance (adherence to user instructions on scope and level of complexity) meant to assess basic capabilities related to abstraction. Using the gemini‐2.5‐flash‐preview‐09‐2025 LLM, the engine achieved 100% accuracy in a causal translation evaluation and 78% success in a conformance evaluation. These findings demonstrate the potential of open, modular AI‐powered tools to enhance users' abilities to carry out SD modeling.