If our objective is to keep the mean squared estimation
error small, then the best possible estimator is the
conditional expectation.
But sometimes the conditional
expectation is hard to calculate.
Maybe we're missing the details of the various
probability distributions.
Or maybe we have the distributions that we need but
the formulas are complicated.
After all, the conditional expectation can be a
complicated non-linear function of the observations.
For this reason, we may want to consider an estimator that
has a simpler structure, an estimator that is a linear
function of the data.
And then, within this class of estimators, find the one that
results in the smallest possible mean squared error.
In this lecture we will formulate this linear least
squares estimation problem and then solve it.
We will see that the solution is given by a simple formula
that involves only the means, variances, and covariances of
the random variables involved.
Because of the simplicity of the method, linear estimators
are used quite often, especially in systems where
estimates need to be computed quickly in real time as
observations are obtained.
We will look into some of the mathematical properties of the
linear least mean squares estimator and the associated
mean squared error, revisit an example from the previous
lecture, and finally close with some comments on the ways
that this estimator can be used.