chapter 13, Probability and Random Variables

Excerpt

Historical Comments: The origins of probability theory can be traced back to the correspondence between Blaise Pascal (1623–1662) and Pierre Fermat (1601–1665) concerning gambling games. Their theory, considered the first foundation of probability theory, remained largely a tool reserved for use in games of chance until Pierre S. Laplace (1749–1827) and Karl Friedrich Gauss (1777–1855) applied it to other problems.

Further interest in probability was generated when it was recognized that the probability of an event, e.g., in the kinetic theory of gases and in many social and biological phenomena, often depends on preceding outcomes. In Russia, for example, the study of such linked chains of events (now known as Markov chains or Markov processes) was initiated in 1906–1907 by Andrei A. Markov (1856–1922), a student of Chebyshev. Important advances in Markov processes were made by Andrei N. Kolmogorov (1903–1987) in 1931. Kolmogorov is also credited with establishing modern probability theory in 1933 by his use of the theory of measure and integration advanced in the early twentieth century by Henri Lebesgue (1875–1941) and Félix E. E. Borel (1871–1956).

The objective of this chapter is to provide an overview of probability theory that is useful in physical applications. In particular, the theory presented here makes it possible to mathematically describe or model random signals that arise, for example, in the analysis of communication systems. Such random signals, called random processes, are discussed in further detail in Chapter 14.

13.1 Introduction

The mathematical theory developed in basic courses in engineering and the physical sciences is usually based on deterministic phenomena. As an example, the input to a linear filter is often presumed to be a deterministic quantity, such as a sine wave, step function, impulse function, and so on, leading to a deterministic output. However, in practice the input to a filter may contain a fluctuating or “random” quantity (noise) that yields some uncertainty about the output. In general, an unpredictable noise signal always appears at the input to any communication receiver and thus interferes with the reception of incoming radio or radar “signals.” Situations like this that involve uncertainty or randomness in some form cannot be analyzed by deterministic methods but must be treated by probabilistic methods. Probability theory has become an indispensable tool in engineering and scientific analysis involving electron emission, radar detection, quality control, statistical mechanics, turbulence, and noise, among other areas.