The Effect of Frequency Selective Fading on the Binary Error Probabilities of Incoherent and Differentially Coherent Matched Filter Receivers

The presence of frequency-selective fading in a communication channel limits the maximum data rate capability of conventional communication systems. Surprisingly little analysis has been carried out to determine the effect of frequency-selective fading on digital (or analog) communication systems. This paper considers the effect of frequency-selective fading on the binary error probabilities of incoherent and differentially coherent matched filter receivers employing postdetection diversity combining. In the analysis it is assumed that the fading is slow enough so that over a few bits at least the channel transfer function remains constant. In addition, it is assumed that the amplitude and phase fluctuations on a received carrier have the same statistical character as those of narrow-band Gaussian noise. The general analytical results are specialized to the cases of frequency-shift keying using incoherent detection, and phase-shift keying using differentially coherent detection for the case of a Gaussian frequency autocorrelation function. For these special cases, signal-to-noise degradation curves are given as a function of the ratio of the binary data rate to the correlation (or coherence) bandwidth. Two types of FSK are considered, phase-continuous and phase-discontinuous. In phase-continuous FSK there is no discontinuity in the phase of the transmitted waveform at the markspace or space-mark epochs. Such an FSK system results when the mark and space frequencies are obtained by frequency modulating an oscillator. In phase-discontinuous FSK phase discontinuities exist at the transition epochs. Such an FSK system results, for example, when the mark and space frequencies are derived by switching between two independent oscillators. An interesting result of the analysis is that the differentially coherent PSK system and the phase-discontinuous FSK system degrade considerably more rapidly with increasing (normalized) data rate than the phasecontinuous FSK system. The existence of an irreducible error probability is demonstrated for the incoherent and differentially coherent matched filter receivers. Thus, in general, an increase in transmitted signal power cannot reduce the error probability below a certain value depending upon the ratio of data rate to correlation bandwidth and order of diversity. Theoretical curves of irreducible error probability are given for the incoherent FSK and differentially coherent PSK systems.