Probability and Random Processes with Applications to Signal Processing

(NOTE: Each chapter concludes with a Summary, Problems, and References.) 1. Introduction to Probability. Introduction: Why Study Probability? The Different Kinds of Probability. Misuses, Miscalculations, and Paradoxes in Probability. Sets, Fields, and Events. Axiomatic Definition of Probability. Joint, Conditional, and Total Probabilities Independence. Bayes' Theorem and Applications. Combinatorics. Bernoulli Trials-Binomial and Multinomial Laws. Asymptotic Behavior of the Binomial Law: The Poisson Law. Normal Approximation to the Binomial Law. 2. Random Variables. Introduction. Definition of a Random Variable. Probability Distribution Function. Probability Density Function. Continuous, Discrete and Mixed Random Variables. Conditional and Joint Distributions and Densities. Failure Rates. 3. Functions of Random Variables. Introduction. Solving Problems of the Type Y=g(X). Solving Problems of the Type Z=g(X,Y). Solving Problems of the Type V=g(X,Y), W=h(X,Y). Additional Examples. 4. Expectation and Introduction to Estimation. Expected Value of a Random Variable. Conditional Expectation. Moments. Chebyshev and Schwarz Inequalities. Moment Generating Functions. Chernoff Bound. Characteristic Functions. Estimators for the Mean and Variance of the Normal Law. 5. Random Vectors and Parameter Estimation. Joint Distributions and Densities. Multiple Transformation of Random Variables. Expectation Vectors and Covariance Matrices. Properties of Covariance Matrices. Simultaneous Diagonalization of Two Covariance Matrices and Applications in Pattern Recognition. The Multidimensional Gaussian Law. Characteristic Functions of Random Vectors. Parameter Estimation. Estimation of Vector Means and Covariance Matrices. Maximum Likelihood Estimators. Linear Estimation of Vector Parameters. 6. Random Sequences. Basic Concepts. Basic Principles of Discrete-Time Linear Systems. Random Sequences and Linear Systems. WSS Random Sequence. Markov Random Sequences. Vector Random Sequences and State Equations. Convergence of Random Sequences. Laws of Large Numbers. 7. Random Processes. Basic Definitions. Some Important Random Processes. Continuous-Time Linear Systems with Random Inputs. Some Useful Classification of Random Processes. Wide-Sense Stationary Processes and LSI Systems. Periodic and Cyclostationary Processes. Vector Processes and State Equations. 8. Advanced Topics in Random Processes. Mean-Square (m.s.) Calculus. m-s Stochastic Integrals. m-s Stochastic Differential Equations. Ergodicity. Karhunen-Loeve Expansion. Representation of Bandlimited and Periodic Processes. 9. Applications to Statistical Signal Processing. Estimation of Random Variables. Innovation Sequences and Kalman Filtering. Wiener Filter for Random Sequence. Expectation-Maximization Algorithm. Hidden Markov Models (HMM). Spectral Estimation. Simulated Annealing. Appendices. Appendix A: Review of Relevant Mathematics. Basic Mathematics. Continuous Mathematics. Residue Method for Inverse Fourier Transform. Mathematical Induction A-4. Appendix B: Gamma and Delta Functions. Gamma Function. Dirac Delta Function. Appendix C: Functional Transformations and Jacobians. Introduction. Jacobians for n = 2. Jacobian for General n. Appendix D: Measure and Probability. Introduction and Basic Ideas. Application of Measure Theory to Probability. Appendix E: Sampled Analog Waveforms and Discrete-time Signals.