Probability Distributions

The One Thing

A probability distribution assigns probabilities to the possible values of a random variable, allowing uncertainty to be quantified. The appropriate model depends on the variable and experimental conditions: binomial for fixed independent success–failure trials, Poisson for event counts over intervals, and normal for continuous measurements with approximately bell-shaped variation.

Definitions and Results

• Random Experiment: An experiment whose outcome cannot be predicted with certainty before it is performed, although all possible outcomes are known.

• Random Variable: A function that assigns a numerical value to each outcome of a random experiment.

• Discrete Random Variable: A random variable that takes a finite or countably infinite set of values, such as the number of heads in three coin tosses.

• Continuous Random Variable: A random variable that can take any value in an interval, such as height, weight, time, or temperature.

• Probability Distribution: A complete description of the possible values of a random variable and their corresponding probabilities. The total probability represented by a probability distribution must equal 1.

• Probability Mass Function: For a discrete random variable $X$,

$$p(x)=P(X=x),$$ which gives the probability that $X$ takes the value $x$. For values $x_i$ with probabilities $p_i$, $$p_i=P(X=x_i),p_i\ge 0,\sum p_i=1.$$ For every possible value $x$, $$0\le P(X=x)\le 1.$$

• Probability Density Function: For a continuous random variable $X$, a non-negative function $f(x)$ whose area over an interval gives the probability that $X$ lies in that interval. It must satisfy

$$f(x)\ge 0,{\int }_{-\infty }^{\infty }f(x)dx=1.$$ For $a\le b$, $$P(a\le X\le b)={\int }_a^{b}f(x)dx,$$ while $$P(X=a)=0.$$

• Cumulative Distribution Function: The function

$$F(x)=P(X\le x).$$ For a continuous variable, $$F(x)={\int }_{-\infty }^{x}f(t)dt.$$

• Mathematical Expectation: The weighted average, or long-term mean, of a random variable. For a discrete random variable,

$$E(X)=\sum x_i{p}_i,$$ provided the sum exists. More generally, $$E[g(X)]=\sum g(x_i)p_i.$$

• Variance: A measure of the average squared deviation from the mean:

$$\mathrm{Var}(X)=E[(X-\mu {)}^2]=E(X^2)-[E(X){]}^2,$$ where $$\mu =E(X).$$

• Standard Deviation: The positive square root of variance:

$$\sigma =\sqrt{\mathrm{Var}(X)}.$$ It measures the typical spread around the mean.

• Linear Transformation: For constants $a$ and $b$,

$$E(aX+b)=aE(X)+b,$$ and $$\mathrm{Var}(aX+b)=a^2\mathrm{Var}(X).$$

• Bernoulli Trial: A random experiment with exactly two possible outcomes, usually called success and failure.

• Independent Trials: Trials in which the result of one trial does not affect the result of another trial.

• Binomial Distribution: The distribution of the number of successes in a fixed number of independent Bernoulli trials, each with the same probability of success. Its probability function is

$$P(X=r)=\left(\genfrac{}{}{0ex}{}{n}{r}\right)p^r{q}^{n-r},$$ where $$r=0,1,\dots ,n,q=1-p.$$ Its mean, variance, and standard deviation are respectively $$np,npq,\sqrt{npq}.$$ The conditions are a fixed number of trials, two outcomes per trial, independent trials, and a constant probability of success.

• Poisson Distribution: A discrete distribution modelling the number of occurrences of an event in a fixed interval when events occur independently at a constant average rate. For parameter $\lambda$,

$$P(X=r)=\frac{e^{-\lambda }{\lambda }^r}{r!},$$ where $$r=0,1,2,\dots ,\lambda >0.$$ Its mean and variance are both $\lambda$, and its standard deviation is $$\sqrt{\lambda }.$$ It can approximate a binomial distribution when $n$ is large, $p$ is small, and $$\lambda =np$$ remains finite.

• Normal Distribution: A continuous, bell-shaped, symmetric distribution determined by its mean $\mu$ and standard deviation $\sigma$. Its density is

$$f(x)=\frac{1}{\sigma \sqrt{2\pi }}{e}^{-(x-\mu {)}^2/(2{\sigma }^2)},\sigma >0.$$ It is symmetric about $x=\mu$, so the mean, median, and mode are equal. Approximately 68% of observations lie within one standard deviation of the mean, 95% within two standard deviations, and 99.7% within three standard deviations.

• Standard Normal Variable: The standardised variable

$$Z=\frac{X-\mu }{\sigma },$$ with mean $0$ and standard deviation $1$.

• Applications: Probability distributions are used for estimating defect rates, predicting demand, analysing examination scores, studying arrival patterns, calculating insurance risk, and making decisions under uncertainty.

Worked Methods

Constructing and checking a discrete probability distribution

• List every possible value $x_i$ of the discrete random variable.
• Assign the corresponding probability

$$p_i=P(X=x_i).$$

• Check that every probability is non-negative:

$$p_i\ge 0.$$

• Check that the probabilities sum exactly to one:

$$\sum p_i=1.$$

• Use the probability mass function to calculate probabilities of particular values.

A named example of a discrete random variable is the number of heads in three coin tosses.

Calculating expectation, variance, and standard deviation

• Multiply each possible value $x_i$ by its probability $p_i$.
• Add the products:

$$E(X)=\sum x_i{p}_i.$$

• Calculate $E(X^2)$ using

$$E(X^2)=\sum x_i^2{p}_i.$$

• Let $\mu =E(X)$, then calculate

$$\mathrm{Var}(X)=E(X^2)-[E(X){]}^2.$$

• Take the positive square root:

$$\sigma =\sqrt{\mathrm{Var}(X)}.$$

For a function $g(X)$, calculate its expectation directly from $$E[g(X)]=\sum g(x_i)p_i.$$

Using a probability density function

• Confirm that $f(x)\ge 0$.
• Confirm that the total area is one:

$${\int }_{-\infty }^{\infty }f(x)dx=1.$$

• To find the probability that $X$ lies between $a$ and $b$, integrate:

$$P(a\le X\le b)={\int }_a^{b}f(x)dx.$$

• Remember that an individual point has probability zero:

$$P(X=a)=0.$$

• To obtain the cumulative probability up to $x$, use

$$F(x)={\int }_{-\infty }^{x}f(t)dt.$$

Applying the binomial distribution

• Check the four binomial conditions: a fixed number of trials, two outcomes per trial, independent trials, and a constant probability of success.
• Identify $n$, the fixed number of trials; $p$, the probability of success; $q=1-p$; and $r$, the required number of successes.
• Substitute into

$$P(X=r)=\left(\genfrac{}{}{0ex}{}{n}{r}\right)p^r{q}^{n-r},r=0,1,\dots ,n.$$

• For summary measures, use

$$E(X)=np,\mathrm{Var}(X)=npq,\sigma =\sqrt{npq}.$$

Applying the Poisson distribution

• Check that the variable counts occurrences in a fixed interval.
• Check that events occur independently and at a constant average rate.
• Identify the parameter $\lambda >0$.
• For $r=0,1,2,\dots$, calculate

$$P(X=r)=\frac{e^{-\lambda }{\lambda }^r}{r!}.$$

• Use

$$E(X)=\lambda ,\mathrm{Var}(X)=\lambda ,\sigma =\sqrt{\lambda }.$$

• A Poisson approximation to a binomial distribution is appropriate when $n$ is large, $p$ is small, and $\lambda =np$ remains finite.

Applying the normal distribution

• Identify the mean $\mu$ and standard deviation $\sigma$, ensuring that $\sigma >0$.
• Use the density

$$f(x)=\frac{1}{\sigma \sqrt{2\pi }}{e}^{-(x-\mu {)}^2/(2{\sigma }^2)}.$$

• Standardise a value $X$ using

$$Z=\frac{X-\mu }{\sigma }.$$

• Use the symmetry about $x=\mu$, where the mean, median, and mode coincide.
• Interpret spread using the approximate percentages: 68% within one standard deviation, 95% within two, and 99.7% within three.

Selecting a model for an application

• Determine whether the variable is discrete or continuous.
• For a fixed number of independent success–failure trials with constant success probability, use the binomial distribution.
• For counts of independent events in a fixed interval with a constant average rate, use the Poisson distribution.
• For continuous measurements showing approximately bell-shaped variation, use the normal distribution.
• Check assumptions such as independence, constant probability, fixed trial number, and an appropriate measurement scale before calculating.

These methods support applications including estimating defect rates, predicting demand, analysing examination scores, studying arrival patterns, calculating insurance risk, and making decisions under uncertainty.

Where It Goes Wrong

• Assigning a negative probability or allowing probabilities to sum to anything other than exactly $1$.
• Treating a continuous probability density value as a probability; for a continuous variable, probability is an area, and $P(X=a)=0$.
• Using the binomial formula without checking the fixed number of trials, two outcomes per trial, independence, and constant probability of success.
• Applying a Poisson model without confirming independent events, a fixed interval, and a constant average rate.
• Confusing variance with standard deviation: variance is $\mathrm{Var}(X)$, whereas standard deviation is $\sqrt{\mathrm{Var}(X)}$.
• Forgetting the conditions for a Poisson approximation to a binomial distribution: $n$ must be large, $p$ small, and $\lambda =np$ finite.

What Gets Asked

• Define a random experiment, random variable, discrete random variable, continuous random variable, probability mass function, probability density function, or cumulative distribution function.
• Check whether a proposed set of probabilities forms a valid probability distribution.
• Calculate $E(X)$, $E[g(X)]$, variance, and standard deviation from a discrete distribution.
• Use the transformations $E(aX+b)=aE(X)+b$ and $\mathrm{Var}(aX+b)=a^2\mathrm{Var}(X)$.
• Evaluate probabilities from a continuous density using

$$P(a\le X\le b)={\int }_a^{b}f(x)dx.$$

• Identify whether a situation should be modelled by a binomial, Poisson, or normal distribution.
• Calculate a binomial probability, mean, variance, or standard deviation.
• Calculate a Poisson probability, mean, variance, or standard deviation.
• Decide whether a Poisson approximation to a binomial distribution is justified.
• Use the normal density, standardise using

$$Z=\frac{X-\mu }{\sigma },$$ and interpret the 68%, 95%, and 99.7% rules.

• Apply probability distributions to defect rates, demand, examination scores, arrival patterns, insurance risk, and decisions under uncertainty.