Sum of normally distributed random variables

Probability theory

Probability theory is the branch of mathematics concerned with analysis of random phenomena. The central objects of probability theory are random variables, stochastic processes, and events: mathematical abstractions of non-deterministic events or measured quantities that may either be single...

, calculation of the sum of normally distributed random variables is an instance of the arithmetic of random variable

Random variable

In probability and statistics, a random variable or stochastic variable is, roughly speaking, a variable whose value results from a measurement on some type of random process. Formally, it is a function from a probability space, typically to the real numbers, which is measurable functionmeasurable...

s, which can be quite complex based on the probability distribution

Probability distribution

In probability theory, a probability mass, probability density, or probability distribution is a function that describes the probability of a random variable taking certain values....

s of the random variables involved and their relationships.

Independent random variables

If X and Y are independent

Statistical independence

In probability theory, to say that two events are independent intuitively means that the occurrence of one event makes it neither more nor less probable that the other occurs...

random variable

Random variable

s that are normally distributed (not necessarily jointly so), then their sum is also normally distributed. i.e., if

and

are independent, then

This means that the sum of two independent normally distributed random variables is normal, with its mean being the sum of the two means, and its variance being the sum of the two variances (i.e., the square of the standard deviation is the sum of the squares of the standard deviations).

Note that the result that the sum is normally distributed requires the assumption of independence, not just uncorrelatedness; two separately (not jointly) normally distributed random variables can be uncorrelated without being independent, in which case their sum can be non-normally distributed (see Normally distributed and uncorrelated does not imply independent#A symmetric example). The result about the mean holds in all cases, while the result for the variance requires uncorrelatedness, but not independence.

Proof using characteristic functions

The characteristic function

Characteristic function (probability theory)

In probability theory and statistics, the characteristic function of any random variable completely defines its probability distribution. Thus it provides the basis of an alternative route to analytical results compared with working directly with probability density functions or cumulative...

of the sum of two independent random variables

and

. is just the product of the two separate characteristic functions:

and

.

The characteristic function of the normal distribution with expected value

and variance

This is the characteristic function of the normal distribution with expected value

and variance

Finally, recall that no two distinct distributions can both have the same characteristic function, so the distribution of

must be just this normal distribution.

Proof using convolutions

For random variables

and

, the distribution

equals the convolution of

and

Given that

and

are normal densities,

Substituting into the convolution:

The expression in the integral is a normal density distribution on

, and so the integral evaluates to 1. The desired result follows:

Geometric proof

First consider the normalized case when

and

,
so that their PDF

Probability density function

In probability theory, a probability density function , or density of a continuous random variable is a function that describes the relative likelihood for this random variable to occur at a given point. The probability for the random variable to fall within a particular region is given by the...

s are

and

Let

. Then the CDF

Cumulative distribution function

In probability theory and statistics, the cumulative distribution function , or just distribution function, describes the probability that a real-valued random variable X with a given probability distribution will be found at a value less than or equal to x. Intuitively, it is the "area so far"...

for

will be

This integral is over the half-plane which lies under the line

.

The key observation is that the function

is radially symmetric. So we rotate the coordinate plane about the origin, choosing new coordinates

such that the line

is described by the equation

where

is determined geometrically. Because of the radial symmetry, we have

, and the CDF for

This is easy to integrate; we find that the CDF for

To determine the value

, note that we rotated the plane so that the line

now runs vertically with

-intercept equal to

. So

is just the distance from the origin to the line

along the perpendicular bisector, which meets the line at its nearest point to the origin, in this case

. So the distance is

, and the CDF for

, i.e.,

Now, if

are any real constants (not both zero!) then the probability that

is found by the same integral as above, but with the bounding line

. The same rotation method works, and in this more general case we find that the closest point on the line to the origin is located a (signed) distance

away, so that

The same argument in higher dimensions shows that if

for

then

Now we are essentially done, because

So in general, if

for

then

Correlated random variables

In the event that the variables X and Y are jointly normally distributed random variables, then X + Y is still normally distributed (see Multivariate normal distribution) and the mean is the sum of the means. However, the variances are not additive due to the correlation. Indeed,

where ρ is the correlation

Correlation

In statistics, dependence refers to any statistical relationship between two random variables or two sets of data. Correlation refers to any of a broad class of statistical relationships involving dependence....

. In particular, whenever ρ < 0, then the standard deviation is less than the sum of the standard deviations of X and Y. This is perhaps the simplest demonstration of the principle of diversification.

Extensions of this result can be made for more than two random variables, using the covariance matrix

Covariance matrix

In probability theory and statistics, a covariance matrix is a matrix whose element in the i, j position is the covariance between the i th and j th elements of a random vector...

Proof

In this case, one needs to consider

As above, one makes the substitution

This integral is more complicated to simplify analytically, but can be done easily using a symbolic mathematics program. The probability distribution ƒ_Z(z) is given in this case by

where

If one considers instead Z = X − Y, then one obtains

which is also can be rewritten with