We have seen so far two ways of
estimating an unknown parameter.
We can use the maximum a posteriori probability
estimate, or we can use the conditional expectation.
That is, the mean of the posterior distribution.
These were, in some sense, arbitrary choices.
How about imposing a performance criterion, and
then finding an estimate which is optimal with respect to
that criterion?
This is what we will be doing in this lecture.
We introduce a specific performance criterion, the
expected value of the squared estimation error, and we look
for an estimator that is optimal under this criterion.
It turns out that the optimal estimator is the conditional
expectation.
And this is why we have been calling it the least mean
squares estimator.
It plays a central role because it is a canonical way
of estimating unknown random variables.
We will study some of its theoretical properties, and we
will also illustrate its use and the associated performance
analysis in the context of an example.