Widespread reference missingness disparities in open scholarly metadata

Abstract Open scholarly metadatabases are becoming an indispensable component of quantitative studies of science. In particular, reference information from these databases is essential for citation analysis, including impact evaluation and disruption measurement. Recent studies have examined the reference coverage of different scholarly databases; nevertheless, the underlying factors driving the disparity in reference quality across publications remain largely unexplored. Here, we examine the reference quality of OpenAlex, one of the largest open scholarly metadatabases to date, and find pervasive reference missingness. Importantly, we observe that this effect is more pronounced in articles associated with less prestigious institutions, lower impact journals, and authors affiliated with countries of lower economic standing. A striking 57% of articles below the median prestige and journal impact that belong to the Global South are completely missing references. A regression analysis revealed that these disparities are mainly associated with the prestige of the journal and the institution. We discuss how we should address this issue more systematically and exercise caution when using metadata to assess impact.