DriftBench: Measuring and Predicting Infrastructure Drift in LLM Serving Systems

Production LLM deployments lack systematic methods to assess output consistency risks when infrastructurechanges. We present DriftBench, a measurement and prediction framework comprising 236,985 prompt-responsepairs across 105 configurations spanning 5 models, 4 GPU platforms, 3 frameworks, 3 precisions. We develop thePortability Risk Index (PRI), achieving held-out-dimension generalization of R2=0.909 for unseen hardware andR2=0.763 for unseen precision (R2 ranges up to 1.0; higher is better). We discover a fundamental dichotomy:hardware/precision changes exhibit systematic drift (R2 ≥ 0.76) enabling predict-once deployment, whileframework/model changes show idiosyncratic drift (R2 < 0.48) requiring re-measurement. Production validationblocked a high-drift upgrade where 23.85% of safety prompts flipped between safe and unsafe classifications(nearly 1 in 4 answers changed from safe to unsafe or unsafe to safe), demonstrating operational value. Ourcontribution is measurement and risk assessment; we do not propose drift mitigation techniques, as this remainsan open challenge for future work.