Generalized Analytical Modeling of Filtering Penalties and Equalization in Coherent Optical Links

This paper presents a comprehensive analytical framework for evaluating filtering penalties in coherent optical links. The model accounts for the cumulative effects of cascaded optical filters, amplified spontaneous emission noise due to optical amplification, and transceiver noise, alongside digital equalization at the receiver. By developing a generalized channel representation, we adapt to the optical channel scenario the closed-form expressions for signal-to-noise ratio degradation under various equalization strategies, including Zero-Forcing Equalizer, Minimum Mean Square Error Equalizer, Fractionally Spaced Equalizer, and Finite Length Equalizer. These models capture the impact of colored noise resulting from linear filtering and provide insights and tools to implement the filtering penalty assessment in optical network digital twins. The proposed framework is validated through time domain simulation using accurately modeled optical filters, and extensive experimental analyzes on a complex optical link setup, using commercial transceivers from different vendors. An excellent agreement is obtained between theory and experiments, offering a robust foundation for system-level performance evaluation, with special focus to metro-access networks.