Spectral Robustness Mixer: Cross-Scale Neck for Robust No-Reference Image Quality Assessment

No-reference image quality assessment (NR-IQA) models achieve high correlation with human mean opinion scores (MOS) on clean benchmarks, yet recent work shows they can be highly vulnerable to small adversarial perturbations that severely degrade ranking consistency, including in black-box settings. We introduce the Spectral Robustness Mixer (SRM), a lightweight neck inserted between an NR-IQA backbone and regression head, designed to reduce adversarial sensitivity without changing the dataset, label format, or target metric. SRM couples (i) deep-to-shallow cross-scale fusion via a Nyström low-rank attention surrogate, (ii) ridge-conditioned landmark kernels with ridge regularization, solved via numerically stable small-matrix factorization (SVD/LU) to improve conditioning, and (iii) variance-aware entropy-regularized fusion gates with a bounded gain cap to limit gradient amplification. We evaluate SRM on TID2013 and KonIQ-10k under a white-box l∞/l2 attack ensemble that includes per-image regression objectives and a correlation-aware pairwise inversion objective (a ranking-inspired surrogate for correlation inversion), with expectation-over-transformation (EOT) and anti-gradient masking checks. At ϵ=4/255 (l∞), SRM improves worst-case robust Spearman’s rank-order correlation coefficient (SROCC; defined as the minimum over our fixed attack ensemble) by an absolute 0.06–0.08 SROCC points (i.e., correlation-coefficient units, not percentage gain) across datasets/backbones, while keeping clean SROCC within 0.00–0.01 of the baseline. We observe similar trends for Pearson linear correlation coefficient (PLCC).