The central limit theorem is absolutely remarkable.
It is a very deep result, and highly
nontrivial and non intuitive.
There's no apparent reason why this random variable here, a
standardized version of the sum of random variables,
should have an approximately normal distribution.
Furthermore, it is very useful, and one key reason is
that it is universal.
It doesn't matter what the distribution of the X's is.
No matter what the distribution is, still in the
limit, this standardized version of the sum is going to
behave like a normal random variable.
And if we wish to apply it to particular examples or models,
the only thing that we need to know about the distribution of
the X's are the corresponding means and variances, as we're
going to see in multiple examples.
When we apply it, it turns out to be very accurate, and it is
also a very nice computational shortcut.
Even if we knew, in detail, the distribution of the X's,
in order to calculate the distribution of Sn, we would
have to take the distribution of the X's and convolve it
with itself n times, something that can be
computationally tedious.
Whereas the computations that are involved, when we use the
central limit theorem, are very, very simple, as the
examples that will be coming up will show to us.
Finally, at the philosophical level, the central limit
theorem justifies why models involving normal random
variables are very natural.
Whenever you have a phenomenon or an object that's affected
by multiple noise sources, and if these noise sources are
independent, then the overall effect of those noise sources
is going to be well-modeled by a normal random variable, even
if the distribution of each one of these noise sources is
very different from being normal.
And this is a reason why, in many, many applications in
many different fields, normal random variables provide very
useful and accurate models.
Since the central limit theorem is so useful and
important, it is worth making sure that we understand
exactly what it says and to make a few comments on its
mathematical content.
What it says is the following.
We take the sum of independent identically distributed random
variables, we form this standardized version of the
sum, where we subtracted the mean and divide by the
standard deviation of the sum, and then what it tells us is
that the CDF of this random variable, Zn, converges to the
normal CDF.
So what we have is a statement about CDFs.
It does not yet tell us anything specific
about PDFs or PMFs.
So for example, if Sn and the X's are all continuous random
variable, so Zn is also continuous, you might wonder
whether the PDF of Zn converges to a normal PDF.
It turns out that there are results of this kind that
assert convergence of the PDF, or even PMF, of this random
variable to a normal PDF, in some sense.
But these results generally need a few more mathematical
assumptions for the results to be valid.
Nevertheless, when we show pictures of various examples,
we will do this by showing pictures of PDFs and PMFs
because these are easier to visualize.
Now since the result is so general and so important, it
might be worth understanding to what extent it can be
generalized to other contexts.
Our main two assumptions are that the random variables are
independent and identically distributed.
Can we remove those assumptions?
There are versions of the central limit theorem that
apply to the case where the Xi's are not identically
distributed.
One just needs to make certain assumptions on the means and
to the variances of the Xi's.
Some conditions will be needed.
Also, the assumption of independence does not need to
be literally true.
There are versions of the central limit theorem that are
valid when we have just weak dependence.
That is, nearby X's may be dependent, but if you compare
X5 with X of 1 million, then these two random variables are
essentially independent.
In those cases, we can still apply a suitable version of
the central limit theorem.
And finally, you may be curious how
this result is proved.
One way of proving it, which is the way it was originally
established a long time ago, for the special case of
Bernoulli random variables X, in which case S is binomial.
The way it was established was by carrying out algebraic
manipulations on the binomial formulas.
But this was a derivation that would not generalize.
For the general case, the proof is obtained using
so-called transform methods, which is a topic that we're
not covering, but it goes as follows.
We consider this function of the random variable Zn, where
s is some parameter, and we show that this expectation
converges to the corresponding expectation if you have the
standard normal Z in the place of Zn.
And this is true for all s, or at least for all s in some
rich enough set.
And then, one appeals to some deep mathematical results that
tell you that if this kind of expectation converges to that
expectation, then the CDF of Zn must also converge to the
CDF of Z. But this is a proof that involves various steps
and appeals to some deep results from other fields of
mathematics.
And finally, there is the practical side.
What exactly does it say and how do we use it?
Since the CDF of Zn can be approximated by the CDF of a
standard normal, this means that in practice, we can treat
the random variable Zn just as if it were a standard normal.
But now, we notice that Sn is obtained as a
linear function of Zn.
Namely, the definition of Zn gives us this formula.
So, Sn is a linear function of Zn.
If we pretend that Zn is normal, and since linear
functions of normal random variables are normal, this
means that we will also pretend that Sn is normal.
So in practice, the way to carry out calculations is
often to just pretend that Sn is normal with the appropriate
mean and variance.
These are the correct means and variance of Sn, and that's
the only information that we need in order to apply it.
If we know mu and sigma squared, and using the normal
approximation, then we have an approximate distribution for
Sn, and we can go ahead.
Now in practice, can we use the central limit theorem when
n is moderate?
For example, if n is 30, can you apply the
central limit theorem?
This is a relation that's true in the limit of very large n.
How large should n be?
It turns out that the central limit theorem gives us very
good approximations even when n has moderately small values.
Now, these approximations sometimes will be better and
sometimes worse.
It helps if the distribution of the X's that you're
starting with has some common one features with the normal
distribution.
If the X's are already normal, then S will be normal and
there's no approximation involved.
If the X's are close to normal, then for fairly small
values of n, S will be very well modeled by a normal
random variable.
Now, what does it mean that the distribution of the X's
looks a little bit like the normal one?
It helps if the distribution is symmetric around its mean,
and it also helps if it is unimodal, in the sense that it
has a single peak rather than multiple peaks.