In this lecture, we provide a quick introduction into the
conceptual framework of classical statistics.
We use the words "classical" to make the distinction from
the Bayesian framework that we have been using so far.
Recall that in the Bayesian framework, unknown quantities
are viewed as random variables that have a certain prior
distribution.
In contrast, in the classical setting an unknown quantity is
viewed as a non-random constant that just happens to
be unknown.
We can still carry out estimation without assuming a
prior for the unknown quantity.
For example, if the unknown theta is the mean of a certain
distribution, we can generate many samples from that
distribution and form the sample mean.
The weak law of large numbers then tells us that this
estimate will approach in the limit as n increases the true
value of theta.
After going through this argument, we will then take
the occasion to introduce some terminology that is often used
in connection with classical estimation methods.
Now, the sample mean provides us a point estimate for the
unknown theta, but does not tell us how accurate that
estimate is.
To give a sense of the accuracy involved, we
introduce the concept of a confidence interval, which is
an interval that has high probability of containing the
true theta.
In general, it is a common practice to report not just
estimates, but also confidence intervals.
But as we will discuss, one has to be careful in
interpreting what exactly a confidence interval tells us.
We will see that we can easily calculate confidence intervals
using the central limit theorem.
And we will discuss in some detail some extra steps that
need to be taken if we do not know the variance of the
random variables involved.
We will then continue in the direction of greater
generality.
We will see that by repeated use of various sample means,
we can also estimate more complicated quantities, such
as, for example, the correlation coefficient
between two random variables.
We will conclude by introducing a general
estimation methodology, the so-called maximum likelihood
estimation method.
This is a method that applies always, even when the unknown
parameter of interest cannot be interpreted as an
expectation.
It is a universally-applicable method.
And fortunately, has some very desirable properties.