Evaluation of SOFA-2 Score Performance Across Demographic Subgroups: An External Validation Study Using MIMIC-IV

Abstract The Sequential Organ Failure Assessment (SOFA)-2 score was recently validated for ICU mortality prediction across more than 3 million admissions but was not evaluated across demographic subgroups. We assessed the discrimination and calibration of the SOFA-2 score for ICU mortality across subgroups defined by age, sex, race and ethnicity, primary language, and insurance status. We conducted a retrospective cohort study of adult patients (aged 18 years or older) admitted to ICUs at Beth Israel Deaconess Medical Center between 2008 and 2022 (MIMIC-IV, version 3.1), selecting the first ICU admission per patient. First-day SOFA-2 scores (range, 0-24) were calculated using worst recorded values across 6 organ systems. Discrimination was assessed using AUROC, calibration using intercepts and slopes, and subgroup differences using bootstrap resampling. Among 64,015 ICU admissions (median age, 66 years [IQR, 54-78]; 56.1% male; 66.1% White), overall ICU mortality was 7.2% (n=4,596). Overall AUROC was acceptable at 0.77 (95% CI, 0.76-0.77). Notably, discrimination declined significantly with age: AUROC was 0.85 (95% CI, 0.83-0.87) for ages 18-44 and 0.72 (95% CI, 0.70-0.73) for ages 75 and older (difference in AUROC, −0.14; 95% CI, −0.16 to −0.11), with systematic underprediction of mortality in older patients (calibration intercept, 0.39). Discrimination was also significantly lower among non-English speakers (difference in AUROC, −0.04; 95% CI, −0.07 to −0.01) but did not differ significantly across documented racial and ethnic groups. Patients with unknown race/ethnicity (14.3% of the cohort) had nearly double the overall mortality rate and poor calibration. SOFA-2 demonstrated good overall performance for ICU mortality prediction but with clinically meaningful variation across demographic subgroups, particularly a substantial decline in discrimination with advancing age. These findings underscore the need for routine equity evaluation of clinical prediction tools before widespread implementation.