Rethinking the modeling of the instrumental response of telescopes with\n a differentiable optical model

We propose a paradigm shift in the data-driven modeling of the instrumental\nresponse field of telescopes. By adding a differentiable optical forward model\ninto the modeling framework, we change the data-driven modeling space from the\npixels to the wavefront. This allows to transfer a great deal of complexity\nfrom the instrumental response into the forward model while being able to adapt\nto the observations, remaining data-driven. Our framework allows a way forward\nto building powerful models that are physically motivated, interpretable, and\nthat do not require special calibration data. We show that for a simplified\nsetting of a space telescope, this framework represents a real performance\nbreakthrough compared to existing data-driven approaches with reconstruction\nerrors decreasing 5 fold at observation resolution and more than 10 fold for a\n3x super-resolution. We successfully model chromatic variations of the\ninstrument's response only using noisy broad-band in-focus observations.\n