From Chaos to Innovation: Understanding Products and People in a Non‐Deterministic World

Abstract In an era tempted by rapid in‐market iteration, this paper demonstrates the critical role of ethnographic methods for understanding complex human interactions with non‐deterministic LLMs. Through a longitudinal Wizard‐of‐Oz study of “Nova,” a simulated AI family wellness assistant, we exposed limitations of traditional usability methods in high‐stakes, multi‐participant contexts. Our methodological approach documented organizational chaos in group AI interactions, identified optimal patterns in human‐AI and Human‐in‐the‐Loop (HITL) collaborations, and traced the nuanced process of AI relationship formation and its impacts on user reflection and behavior. Ethnographic insights led directly to innovations including dynamic topic segmentation technology and multiple patent applications. This work demonstrates the indispensability of ethnographic methods for understanding AI systems within authentic social contexts, where human expertise and support remain vital.