Artificial intelligence generated operative reports for common spine surgeries: how close are they to the real thing?

Study Design: Randomized controlled survey study. Purpose: Evaluate the level of similarity between AI-generated and fellowship-trained spine surgeon-written operative reports. Overview of Literature: Large language models (LLMs) are a subset of generative artificial intelligence (AI) designed to produce human-like text. LLMs have demonstrated an ability to generate contextually accurate, detailed, and coherent writing across various domains. Their potential to enhance healthcare efficiency, particularly in streamlining medical documentation such as operative notes, is of growing interest. Methods: Operative reports for two common spine surgeries—anterior cervical discectomy and fusion (ACDF) and lumbar microdiscectomy were generated using ChatGPT-3 (OpenAI). A fellowship-trained spine surgeon wrote corresponding operative reports. These reports were randomized and presented to attending surgeons, fellows, and residents in orthopedic or neurosurgery, who were asked to identify whether each report was written by AI. Results: Fifty-two respondents completed the survey. For ACDF, 69.2% correctly identified AI-generated reports ( P =0.050); for lumbar microdiscectomy, 61.5% were correct ( P =0.239). In side-by-side comparisons, correct identification improved to 79.2% for ACDF ( P =0.004) and 60% for microdiscectomy ( P =0.317). Accuracy increased with training level, from 55.6% among residents to 100% among attending spine surgeons. Most participants reported that AI-generated reports had human-like language (86.3%), adequate detail (68.6%), essential procedural steps (78.4%), and accurate descriptions (68.6%). Notably, 87.5% expressed interest in incorporating AI into future operative documentation. Conclusions: Differentiation of AI-generated operative reports improves with training level. These reports share many characteristics with surgeon-written notes and are viewed favorably, suggesting a future role for AI in surgical documentation. Level of Evidence: II