Analytical Validation of a Clinical Grade Prognostic and Classification Artificial Intelligence Laboratory Test for Men with Prostate Cancer

Introduction: This is the first study of which we are aware to describe the analytical validation (AV) of clinical grade artificial intelligence (AI) algorithms for a commercially available prostate cancer test performed on hematoxylin and eosin stained specimens that is not dependent on a priori established molecules or a priori semantically meaningful morphology. Methods: We adapted AV methods used in molecular diagnostics and clinical pathology to two AI biomarkers used in a clinical test for prostate cancer biopsy specimens. The two algorithms included one algorithm with prognostic performance and a second algorithm predictive for treatment benefit from short-term androgen deprivation therapy (ST-ADT). We assessed analytical accuracy, intra-operator reliability, and inter-operator reliability, and biopsy set completeness reliability on two AI algorithms deployed into a clinical laboratory setting. Analytical accuracy was measured using intraclass correlation coefficient (ICC). Reliability studies were assessed using ICC for the prognostic algorithm and percent agreement for the ST-ADT classification algorithm. Results: Analytical accuracy ICC was 0.991 and 0.934 for the prognostic and ST-ADT algorithms, respectively. Intra-operator reliability was 0.981 (ICC) and 100% (percent agreement) for the prognostic and ST-ADT algorithms, respectively. Inter-operator reliability was 0.994 (ICC) and 93.3% (percent agreement) for the prognostic and ST-ADT algorithms, respectively. Biopsy-completeness reliability for one versus three prostate biopsy cores was 0.894 (ICC) and 91.67% (percent agreement) for the prognostic and ST-ADT algorithms respectively. For one versus six cores, reliability was 0.857 (ICC) and 95.00% (percent agreement) for the prognostic and ST-ADT algorithms respectively. Conclusion: This study describes a novel approach to AV of AI algorithms in prostate cancer and applies this approach to two algorithms translated for use as a clinical grade AI-based laboratory test, supporting analytical validity of the test.