Latent Diffusion Model Assisted Image Deblurring

Image deblurring aims at pixel-level prediction with high fidelity, while realistic details are indispensable to high-quality deblurred images. A large latent diffusion model (LDM) trained on large-scale datasets generates images with realistic details, while incorporating a LDM into the deblurring process cannot avoid the non-negligible cost during inference on consumer electronics. To verify the hypothesis that a pretrained large LDM helps a deblurring network learn to improve visual quality without the additional inference cost raised by LDM, this paper proposes a novel two-phase training framework. The pretrained LDM functions as the auxiliary network during training and helps a deblurring network learn to improve quality. For this, the proposed late textual and visual prompt tuning equips the pretrained LDM with prior knowledge of sharp images, enabling the use of a small number of learnable parameters. The LDM helps an existing deblurring network learn to generate realistic details. The inference phase only retains the deblurring network to avoid the cost of the LDM. We utilize synthetic and real image deblurring datasets as main tools for developing and evaluating the proposed method. Test results show that the proposed method enhances most of perceptual indices and keep balance between fidelity and quality.