Table of Contents:

1 Introduction to Probability

• 1.1 Introduction: Why Study Probability?

• 1.2 The Different Kinds of Probability

• 1.3 Misuses, Miscalculations, and Paradoxes in Probability

• 1.4 Sets, Fields, and Events

• 1.5 Axiomatic Definition of Probability

• 1.6 Joint, Conditional, and Total Probabilities; Independence

• 1.7 Bayes’ Theorem and Applications

• 1.8 Combinatorics 38

• 1.9 Bernoulli Trials–Binomial and Multinomial Probability Laws

• 1.10 Asymptotic Behavior of the Binomial Law: The Poisson Law

• 1.11 Normal Approximation to the Binomial Law

2 Random Variables

• 2.1 Introduction

• 2.2 Definition of a Random Variable

• 2.3 Cumulative Distribution Function

• 2.4 Probability Density Function (pdf)

• 2.5 Continuous, Discrete, and Mixed Random Variables

• 2.6 Conditional and Joint Distributions and Densities

• 2.7 Failure Rates

3 Functions of Random Variables

• 3.1 Introduction

• 3.2 Solving Problems of the Type Y = g(X)

• 3.3 Solving Problems of the Type Z = g(X, Y )

• 3.4 Solving Problems of the Type V = g(X, Y ), W = h(X, Y )

• 3.5 Additional Examples

4 Expectation and Moments

• 4.1 Expected Value of a Random Variable

• 4.2 Conditional Expectations

• 4.3 Moments of Random Variables

• 4.4 Chebyshev and Schwarz Inequalities

• 4.5 Moment-Generating Functions

• 4.6 Chernoff Bound

• 4.7 Characteristic Functions

• 4.8 Additional Examples

5 Random Vectors

• 5.1 Joint Distribution and Densities

• 5.2 Multiple Transformation of Random Variables

• 5.3 Ordered Random Variables

• 5.4 Expectation Vectors and Covariance Matrices

• 5.5 Properties of Covariance Matrices

• 5.6 The Multidimensional Gaussian (Normal) Law

• 5.7 Characteristic Functions of Random Vectors

6 Statistics: Part 1 Parameter Estimation

• 6.1 Introduction

• 6.2 Estimators

• 6.3 Estimation of the Mean

• 6.4 Estimation of the Variance and Covariance

• 6.5 Simultaneous Estimation of Mean and Variance

• 6.6 Estimation of Non-Gaussian Parameters from Large Samples

• 6.7 Maximum Likelihood Estimators

• 6.8 Ordering, more on Percentiles, Parametric Versus Nonparametric Statistics

• 6.9 Estimation of Vector Means and Covariance Matrices

• 6.10 Linear Estimation of Vector Parameters

7 Statistics: Part 2 Hypothesis Testing

• 7.1 Bayesian Decision Theory

• 7.2 Likelihood Ratio Test

• 7.3 Composite Hypotheses

• 7.4 Goodness of Fit

• 7.5 Ordering, Percentiles, and Rank

8 Random Sequences

• 8.1 Basic Concepts

• 8.2 Basic Principles of Discrete-Time Linear Systems

• 8.3 Random Sequences and Linear Systems

• 8.4 WSS Random Sequences

• 8.5 Markov Random Sequences

• 8.6 Vector Random Sequences and State Equations

• 8.7 Convergence of Random Sequences

• 8.8 Laws of Large Numbers

9 Random Processes

• 9.1 Basic Definitions

• 9.2 Some Important Random Processes

• 9.3 Continuous-Time Linear Systems with Random Inputs

• 9.4 Some Useful Classifications of Random Processes

• 9.5 Wide-Sense Stationary Processes and LSI Systems

• 9.6 Periodic and Cyclostationary Processes

• 9.7 Vector Processes and State Equations

Appendix A Review of Relevant Mathematics

• A.1 Basic Mathematics

• A.2 Continuous Mathematics

• A.3 Residue Method for Inverse Fourier Transformation

• A.4 Mathematical Induction

Appendix B Gamma and Delta Functions

• B.1 Gamma Function

• B.2 Incomplete Gamma Function

• B.3 Dirac Delta Function

Appendix C Functional Transformations and Jacobians

• C.1 Introduction

• C.2 Jacobians for n = 2

• C.3 Jacobian for General n

Appendix D Measure and Probability

• D.1 Introduction and Basic Ideas

• D.2 Application of Measure Theory to Probability

Appendix E Sampled Analog Waveforms and Discrete-time Signals

Appendix F Independence of Sample Mean and Variance for Normal Random Variables

Appendix G Tables of Cumulative Distribution Functions: the Normal, Student t, Chi-square, and F

Index