Decoherence and degradation of squeezed states in quantum filter cavities

Squeezed states of light have been successfully employed in interferometric gravitational-wave detectors to reduce quantum noise, thus becoming one of the most promising options for extending the astrophysical reach of the generation of detectors currently under construction worldwide. In these advanced instruments, quantum noise will limit sensitivity over the entire detection band. Therefore, to obtain the greatest benefit from squeezing, the injected squeezed state must be filtered using a long-storage-time optical resonator, or ``filter cavity,'' so as to realize a frequency-dependent rotation of the squeezed quadrature. While the ultimate performance of a filter cavity is determined by its storage time, several practical decoherence and degradation mechanisms limit the experimentally achievable quantum noise reduction. In this paper we develop an analytical model to explore these mechanisms in detail. As an example, we apply our results to the 16 m filter cavity design currently under consideration for the Advanced LIGO interferometers.