Analyzing Bias in LLM-Augmented Knowledge Graph Systems: Taxonomy, Interaction Mechanisms, and Evaluation

Large Language Models (LLMs) are increasingly integrated into Knowledge Graph (KG) construction and augmentation pipelines to reduce manual effort and enable scalable knowledge extraction, completion, and reasoning. While this integration offers substantial benefits, it also introduces new forms of bias and unreliability that extend beyond those observed in standalone LLMs or traditional knowledge graphs. In particular, biases originating from language models, such as social and representational bias, hallucination, prompt sensitivity, and domain coverage limitations that interact with structural and content biases inherent to knowledge graphs, result in compounded distortions that propagate across the pipeline. This paper provides a structured and comprehensive analysis of bias in LLM-augmented knowledge graph systems. We first review bias mechanisms in LLMs and standalone KGs, and then examine how these biases interact and amplify during key stages of LLM-based entity extraction, relation generation, graph completion, and reasoning. Based on this analysis, we introduce a unified taxonomy that characterizes bias as a pipeline-level phenomenon rather than an isolated model. We further consolidate recent evaluation metrics adapted for LLM-generated graphs, including semantic and soft lexical measures. Additionally, we survey representative datasets and benchmarks used to study bias in LLMs, KGs, and hybrid LLM–KG systems and identify open research gaps in developing pipeline-aware evaluation frameworks. This work aims to support the design of more reliable, accurate, and fair LLM-augmented knowledge graphs for engineering and domain-specific applications.