We have defined the conditional expectation of a
random variable given another as an abstract object, which
is itself a random variable.
Let us now do something analogous with the notion of
[the]
conditional variance.
Let us start with the definition of the variance,
which is the following.
We look at the deviation of the random variable from its
mean, square it, then take the average of that quantity.
If we live in a conditional universe where we are told the
value of some other random variable, capital Y, then
inside that conditional universe the variance becomes
the following.
It is defined the same way.
Well, in the conditional universe, this is the expected
value of X.
So this quantity here is the deviation of X from its
expected value in that conditional universe.
We square this quantity, we find the squared deviation,
and we look at the expected value of
that squared deviation.
But because we live in a conditional universe, of
course, this expectation has to be a conditional one given
the information that we have available.
So this is nothing but the ordinary variance, but it's
the variance of the conditional distribution of
the random variable, capital X. This is an
equality between numbers.
If I tell you the value of little y, the conditional
variance is defined by this particular
quantity, which is a number.
Now, we proceed in the same way as we proceeded for the
case where we defined the conditional expectation as a
random variable.
Namely, we think of this quantity as a function of
little y, and that function can be now used to define a
random variable.
And that random variable, which would denote this way,
this is the random variable which takes this specific
value when capital Y happens to be equal to little y.
Once we know the value of capital Y, then this quantity
takes a specific value.
But before we know the value of capital Y, then this
quantity is not known.
It's random.
It's a random variable.
Let us look at an example to make this more concrete.
Suppose that Y is a random variable.
We draw that random variable.
And we're told that conditioned on the value of
that random variable, X is going to be uniform on this
particular interval from 0 to Y.
So if I tell you that capital Y takes on a specific
numerical value, then the random variable X is uniform
on the interval from 0 to little y.
A random variable that's uniform on an interval of
length little y has a variance that we know what it is.
It's y squared over 12.
So this is an equality between numbers.
For any specific value of little y, this is the
numerical value of the conditional variance.
Let us now change this equality between numbers into
an abstract equality between random variables.
The random variable, variance of X given Y, is a random
variable that takes this value whenever
capital Y is little y.
But that's the same as this random variable.
This is a random variable that takes this value whenever
capital Y happens to be equal to little y.
So we have defined the abstract concept of a
conditional variance, similar to the case of conditional
expectations.
For conditional expectations, we had the law of iterated
expectations.
That tells us that the expected value of the
conditional expectation is the unconditional expectation.
Is it true that the expected value of the conditional
variance is going to be the same as the
unconditional variance?
Unfortunately, no.
Things are a little more complicated.
The unconditional variance is equal to the expected value of
the conditional variance, but there is an extra term, that
is, the variance of the conditional expectation.
The entries here in red are all random variables.
So the conditional variance has been defined as a random
variable, so it has an expectation of its own.
The conditional expectation, as we have already discussed,
is also a random variable, so it has a variance of its own.
And by adding those terms, we get the total variance of the
random variable X.
So what we will do next will be first to prove this
equality, and then give a number of examples that are
going to give us some intuition about what these
terms mean and why this equality makes sense.