We have previously defined the abstract conditional
expectation of one random variable given
another random variable.
And we discussed that it is, by itself, a random variable.
In particular, it has an expectation, or
mean, of its own.
What is this mean?
This is what we want to find out.
Let us recall our development.
We look at the conditional expectation of a random
variable given a specific numerical value of another
random variable.
This is a number that depends on little y.
And this can be used to define a function little g.
The function little g for any particular little y tells us
the numerical value of the conditional expectation.
Since little g is a well defined function, we can also
now define this particular function, which is now a
function of a random variable.
It's a well defined object.
It's a random variable.
And then we introduced this abstract notation.
We defined this object to be exactly this
particular random variable.
So now we want to calculate the expected value of this
object, which is written this way.
Now this notation, here, may look quite formidable, but
let's see what is happening.
Inside here, we have a random variable.
And we take the expected value of that random variable.
Or, more crisply, think of that as the expected value of
g of capital Y, where g of capital Y is defined through
these correspondences here.
How do we calculate the expected value of a function
of a random variable?
Here we use the Expected Value Rule.
Assuming that Y is a discrete random variable, the Expected
Value Rule takes this form.
And the next step is to substitute the particular form
for g of Y that we have.
g of Y was defined in this manner.
So we're dealing with the sum over all little y's of the
expected value of X, given that Y takes the value little
y, weighted by the PMF of little y.
Now if we look at this expression, then it should
look familiar.
It is the expression that appears in the Total
Expectation Theorem.
We take the conditional expectation under different
scenarios and weigh those conditional expectations
according to the probabilities of those scenarios.
And this just gives us the overall expectation of the
random variable X.
So this step, here, was carried out using the Total
Expectation Theorem.
So we have proved this important fact, that the
expectation of a conditional expectation is the same as the
unconditional expectation.
This important fact is called the Law of Iterated
Expectations.
The proof was carried out assuming that Y is discrete.
So we use this particular version involving a PMF, but
the proof is exactly the same for the continuous case.
You would be using an integral and the PDF, instead the PMF.
As the proof indicates, the Law of Iterated Expectations
is nothing but an abstract version of the Total
Expectation Theorem.
It is really the Total Expectation Theorem written in
more abstract notation.
But this turns out to be powerful and also we avoid
having to deal separately with discrete or
continuous random variables.