Evaluation of the performance of radiologists assisted by AI in detecting colorectal liver metastases on contrast-enhanced CT

Colorectal liver metastasis (CRLM) detection on contrast-enhanced CT (CECT) remains challenging due to low tumor-to-liver contrast. This study aimed to evaluate the performance of 2.5D U-net based artificial intelligence (AI) software for focal liver lesion (FLL) detection on CECT and its added value by comparing radiologists’ performance with and without AI support. This retrospective study included patients with colorectal cancer between January 2008 and December 2011, with available preoperative CECT. Six radiologists consisting of three attendings and three fellows read the CECT images in four review sessions: reporting all FLLs and only suspicious colorectal liver metastasis (CRLM), both with and without AI. The detection rates of FLL and CRLM, diagnostic performance of CRLM, and the reading time were compared between the sessions, using reference standards of pathology, follow-up CECT or gadoxetic acid-enhanced MRI. The study included 277 patients (median age 70 years, 182 male) with 989 FLLs (median size 6 mm, 324 CRLMs). The figures-of-merit of AI and radiologists were 0.82 (95% CI: 0.77, 0.86) and 0.86 (95% CI: 0.81, 0.89) in FLLs ≥ 10 mm (P = 0.188), and 0.53 (95% CI: 0.48, 0.62) and 0.60 (95% CI: 0.56, 0.64) in 5–9 mm FLLs (P = 0.145). In sessions reporting only suspicious CRLM, AI assistance increased pooled sensitivity (62.5% [1215/1944] vs. 66.8% [1299/1944], P < 0.001) while it maintained pooled specificity (89.6% [3574/3990] vs. 89.8% [3584/3990], P = 0.730) in per-lesion analysis. The median reading time decreased with AI when reporting all FLLs from 38.0 s to 30.0 s (P < 0.001). With AI assistance, the reading time gap between senior and junior radiologists decreased from 16.0 s to 9.0 s in sessions reporting all FLLs and decreased from 24.0 s to 14.0 s in sessions reporting only suspicious CRLMs. AI software may improve radiologists’ performance by increasing the sensitivity of diagnosing CRLM on CECT, without decreasing specificity, and reducing the reading time. Not indicated.